Clip-It, a promising start-up in the field of automated video production, came face to face with a serious challenge when they had to perform a prompt refactoring of their cloud infrastructure in response to the anticipated increase in production load. As the chosen provider of DevOps services for this mission, the Noveo team soon realised that the initial solution, which had been perceived as optimal at first, was not entirely viable in the context of this particular project. Like that, we had to make a quick switch over to an alternative solution and bridge some gaps thanks to the many features and possibilities of Amazon Web Services. 

Our Customer

Clip-It is a technological start-up that reinvents the video creation process. Founded by experienced engineers with a background in machine learning and web development, the company skillfully combines an API-first approach with video production. Clip-It’s innovative stance is built upon the automation and cloud-native principles, enriching all the best practices of the marketing industry.

Our project

Context 

Clip-It had successfully developed a proof-of-concept solution which passed all of the production workload tests. However, later on the customer faced a sudden scale-up challenge brought on by the anticipated growth of their client base and the fact that the existing solution was not able to accommodate this newly increased load. Finding the most suitable approach to the problem proved to be quite a complex task – especially considering a couple of critical factors that came into play:

• Refactoring of the solution’s architecture, review of the existing tech stack;
• Proprietary nature of several components, licensing issues;
• Ease of maintainability and its automation;
• Cost efficiency.

The multitude of options to address all of these issues, combined with the extensive range of AWS services that could potentially form the foundation for the final solution, had only added on to the complexity and uncertainty of the decision-making process.

At the same time, decisions had to be made rather swiftly, since the recent increase in the production traffic was ringing a worrying alarm about the approaching scaling limits.

Noveo’s solution

At the beginning of our joint project with Clip-It, Noveo reviewed the existing POC architecture to detect any and all bottlenecks that could get in the way of our upscaling works. It was established that one of the main system components, the video rendering engine, required an alternative deployment approach in order to facilitate the efficient horizontal scaling. With that in mind, our team considered using the following AWS services for the engine part:

• Elastic Kubernetes Service;
• Elastic Container Service (EC2-based), Fargate;
• Lambda;
• EC2 + Auto Scaling Groups (ASG).

Although EKS is quite cost-effective and greatly scalable by nature, it was quickly crossed off our list of options due to the administrative overhead. Moreover, we discovered that there were some limitations imposed by the proprietary nature of the customer’s engine, which made it difficult to implement the solution via containerisation (ECS) and Lambda functions. In the end, we decided to follow the traditional EC2 + ASG route, but then got stumped by the need to automate our autoscaling and autohealing activities with the least maintenance efforts possible. 

And this is when AWS Systems Manager and Lambda came in handy! Thanks to the latter, we defined the scaling logic where rendering instances would be allocated automatically based on the queued jobs; and thanks to the former, we were able to launch our automated checks and rendering engine on a new EC2 invocation. 

On top of that, ECS allowed us to decouple workloads by deploying the REST job server as an autohealing service behind an Application Load Balancer. Plus, we defined the infrastructure using HCL syntax (Terraform) and split it into the development and production environments.

Results

And thus, the main goal of our refactoring project was achieved: our AWS-based solution increased the efficiency of the customer’s system, and enabled Clip-It to easily scale up their platform without worrying about any limitations. Furthermore, the existing components got decoupled, bringing along all of the associated benefits – better resilience, flexibility of operations and development, as well as improved observability. 

The latter was especially appreciated by Clip-It, with their API workloads being quite sensitive to errors/latencies and requiring constant monitoring and optimisation efforts. In this sense, Amazon’s CloudWatch and Systems Manager made it possible to keep a close eye on the whole job processing chain, effectively eliminating any possible “black box” situations.

The infrastructure-as-code approach and detailed documentation writing guides provided by the Noveo team were designed as part of a preventative controls system. It helped the customer to avoid any configuration drifts in the future, and make their maintenance and further infrastructure development tasks more straightforward.

Noveo has been providing a wide range of IT services since 1998. Our team uses the latest technologies and strives to stay flexible as well as to always adapt to the modern-day market constraints. We promote simplicity, transparency, excellent results and long-term relationships.