At Capria, we continuously strive to enhance our technological infrastructure to better serve our portfolio companies. A crucial part of this effort involves optimizing our JobBot, an application that employs advanced Retrieval-Augmented Generation (RAG) to provide accurate and relevant information. Recently, we undertook a significant migration, moving our Job Bot from Amazon Web Services (AWS) to Microsoft Azure. This article shares the insights and lessons learned from this migration, aiming to provide valuable tech tips for our community.

JobBot was initially deployed on Amazon Web Services (AWS) using EC2 T2 medium instances. AWS provided us with a reliable infrastructure, granular control over our environment, and the flexibility to fine-tune configurations to meet specific needs. This setup allowed us to move quickly in the early stages of development and ensured we had access to a wide range of tools and services. However, as our usage patterns evolved and the focus shifted toward more scalable and containerized architectures, we began evaluating alternatives that could better support lightweight, stateless workloads with reduced operational overhead.

Transitioning to Azure provided significant advantages:

• Cost Inefficiencies for Small Applications: For our use case, the AWS EC2 T2 medium instances were costing approximately $30 – $50 per month. While this seemed reasonable initially, we explored options for cost optimization.
• Scalability Limitations with Fixed Resources: AWS provided fixed resource allocation. The T2 medium instances had a set amount of processing CPU and memory. During periods of high traffic or increased usage of the JobBot, these fixed resources led to delays for users as the system couldn’t automatically adjust. This lack of dynamic scaling was a major concern.
• Stateful Server Complexity: In AWS, we had stateful servers, meaning each server instance retained data and connections. While this offered control over data, it added complexity in managing and scaling the application, particularly when handling high traffic loads.

Benefits of Migrating to Azure

Given these challenges, we explored Microsoft Azure and decided to migrate our Job Bot. This transition brought several key benefits:

• Enhanced Scalability with Azure Containers: Azure containers provide superior scalability. They can automatically adjust the number of container instances based on the user load. This dynamic scaling ensures that the Job Bot performs smoothly even during peak usage times, eliminating the delays we experienced with AWS’s fixed resources.
• Easy to setup: With Azure Container Instances you simply point to your Docker image, specify CPU and memory, set environment variables, and the container starts—no extra proxy or load-balancer to manage. On AWS (EC2), you usually need to configure an Application Load Balancer or deploy Nginx/Apache yourself to handle HTTP routing, as well as set up task definitions, security groups, and health checks
• Stateless Architecture Simplification: Azure containers operate as stateless entities. This means each container instance is independent and doesn’t retain data or connections between requests. This stateless architecture, driven by Docker images, simplifies scaling and deployment. Creating replicas of these images allows Azure to quickly respond to increased demand.
• Potential Solutions for Data Management in a Stateless Environment: Recognizing that stateless containers don’t retain data, we explored solutions for data persistence. We are considering using Redis, a fast caching system, and MongoDB, a NoSQL database optimized for JSON data (which aligns with our data structure). These solutions will enable efficient data storage and retrieval, ensuring data consistency across container instances.
• Cost (For Scalability): Azure containers provide enhanced scalability, automatically adjusting the number of instances based on user load, eliminating delays experienced with AWS’s fixed resources. Setting up Azure Container Instances is straightforward, requiring only a Docker image, CPU/memory specification, and environment variables. The stateless architecture of Azure containers simplifies scaling and deployment by using independent Docker image instances, enabling quick responses to increased demand. While Azure container costs might initially seem higher ($110-$150), the automatic scalability and pay-as-you-go model can lead to cost savings during low load periods. 
• In addition to scalability and ease of setup, potential solutions for data management in this stateless environment are being considered, such as Redis and MongoDB, ensuring efficient data storage and retrieval. Docker images serve as the blueprint for creating server instances in Azure, and the stateless nature of these containers ensures request isolation and simplified scaling, though it necessitates external data management solutions. 
  • For applications with high HTTP request volume, typical pricing falls between $110 and $150. However, Azure’s Pay-as-you-go model suits our situation better. While we currently maintain at least one running container by setting a minimum replica of 1, cost savings can be achieved by setting this to 0, allowing the container to scale down to zero during periods of low traffic. Azure’s free tier plan, as previously indicated, aligns well with our requirements.

Technical Considerations: Docker and Statelessness

A crucial aspect of the migration involved understanding Docker and its role in statelessness:

• Docker for Container Creation: Docker Image provides the instructions for creating our server instances in Azure. Each container is built from a Docker image, which is a blueprint containing everything needed to run the JobBot.
• Stateless Nature of Azure Containers: The stateless nature of Azure containers means that there’s no persistent connection or saved state between requests. Each request is handled by a new instance of the container, ensuring isolation and simplifying scaling. However, this also necessitates separate data management solutions like Redis or MongoDB to maintain data persistence.

Conclusion

Migrating our JobBot from AWS to Azure has proven to be a strategic move to enhance performance, scalability, and efficiency. The shift to Azure’s container-based architecture, with its automatic scaling and stateless nature, addresses the limitations we faced with AWS’s fixed resources and stateful servers. While there are technical considerations, such as managing data persistence in a stateless environment, solutions like Redis and MongoDB provide viable options.

Tech Tip for Portfolio Companies

When evaluating cloud hosting options for your applications, consider the following:

• Scalability: How well can your chosen platform handle fluctuating user loads?
• Cost: Analyze both initial and long-term costs, considering the benefits of scalability and performance.
• State Management: Understand the implications of stateful vs. stateless architectures and choose solutions that align with your application’s data needs
• Operational Visibility : In Azure portal we can see the metrics of http requests etc.