OpsWorks Co

About client: The client is a proven Shopify automation tool developer with 5 successful apps and thousands of users in 175 countries. Driven by the goal to make E-commerce automation available to everyone, no matter how big or established their brand is, they offer features and solutions to manage inventory in a few stores, improve conversion, automate the sales process, auto-calculate loyalty discounts, and more. To align product vision with product capabilities, they started rebuilding one of their apps and sought DevOps help.

Goals of the project: The client wanted to release a new version of the app within a strict deadline, make a new version of the app high-available and scalable, and optimize cost efficiency of the infrastructure and maintenance.

Challenges: The client already had a version of the app for the multi-store synchronization with infrastructure built around on-prem servers with limited capacity and scaling options. Each surge of requests caused crashes and downtimes, which resulted in lost orders and money for the app's users. To prevent that in the future, Egnition decided to make their next version of the app Cloud-native.

Solutions:

• Cloud consulting. Knowing that they wanted their next app version to be Cloud-native and exist within the AWS ecosystem, Egnition started to build it accordingly. By the time the OpsWorks team joined, they already had a solid tech stack and a chunk of code. We started by reviewing it to understand whether it is applicable to the Cloud at all.
• Cloud infrastructure building. Recognizing our client's unique struggles and needs, we set out to build a cloud infrastructure accordingly. We began by securing their network to keep private services private. Next, we implemented encrypted Amazon RDS for MySQL and Amazon ElastiCache for Redis to store customers' data securely. Historical data found a home in Amazon DocumentDB. To integrate their app seamlessly with Shopify, we used Amazon API Gateway. Finally, AWS Fargate allowed us to deploy their dockerized application with ease and scalability.
• Dynamic request handling. With the goal of synchronizing inventory and products across multiple stores in real-time, the Egnition team faced two main challenges: events from Shopify needed to be processed in a strictly ordered way, while the number of such transactions may be close to hundreds of thousands per minute. To overcome it, the client’s team developed a unique algorithm to solve the first obstacle, which allowed the OpsWorks team to configure the Amazon Simple Queue Service to handle the second one. This technology helped us to manage the high volume of requests efficiently, ensuring that the backend processes each query without overwhelming the system.

Tech stack: PHP (Laravel), Docker, Bitbucket, Nginx, AWS Fargate, NewRelic, Amazon RDS for MySQL, Amazon ElastiCache for Redis, Amazon DocumentDB, Amazon API Gateway, AWS SQS

About client: The client is the global AI leader in commercial food waste solutions. Serving chefs in 2,000+ locations, it helps run a more sustainable and profitable business, reducing food waste by 50%. The system captures images of discarded food, training itself to identify items in the bin. Winnow then processes this data, giving businesses insights that influence their decision-making.

Goals of the project: The client wanted to build a fault-tolerant infrastructure, improve the system's reliability, availability, and resilience, and automate deployment process and development routine.

Challenges: Heavily relying on real-time tracking and analysis, Winnow is committed to providing crucial day-to-day insights to its users. Any interruptions or downtimes can impact the data's quality and overall tool's efficiency.

Solutions: 

• Microservice architecture development. Initially, the client hosted their 10-component application on the AWS server. With each container relying on the others, a shortfall in any component risked system failure. To address this, our team transitioned from AWS server to AWS ECS, enabling automated version deployment with a single click on the Jenkins dashboard.
• Infrastructure as a code implementation. Allowing the system to scale up and down depending on its needs, our engineers developed a set of specific scripts. IaC approach to automation helped to make the Winnow infrastructure more flexible, making it easier to validate changes, create versioned releases, and apply a variety of coding best practices.
• CI/CD implementation. To automate routine software development work, our team set up Jenkins and a CI/CD Pipeline. Now, there is no need to stop an old docker image and start a new one manually since this can be done with one button in the Jenkins dashboard.

Tech stack: AWS, Jenkins, Docker, Terraform, Ansible, New Relic, Bitbucket, Grafana

About client: The client is a company that has developed an algorithmic trading software system for quantitative hedge funds. This system allows the automation of complex quantitative trading strategies in the equity, Forex and derivatives markets.

Goals of the project: The goal of the project work was to develop and improve a manageable, predictable, and highly secure infrastructure with an improved CI/CD pipeline.

Challenges: Since several apps are stored on the server, the system must be operable at all times. Therefore, it is important to optimise and maintain the IT infrastructure for reasonable resource management. 

Solutions: 

• IT Infrastructure Maintenance. Our goal was to upgrade the Linux servers, create test environments and reconfigure them, allowing the customer to focus more on the product and attract new customers;
• CI/CD Pipeline Improvement. Optimization of the workflow, so that software testing now proceeds without queuing, allowed us to update and release new versions faster; 
• Routine Processes Automation.There was a need to make it easier moving all the infrastructure bricks between different virtualization systems by containerizing the applications stored on the server. To avoid latency, be more stable and scalable;
• Security Enhancement. We have containerized applications, and with that increased security by isolating services. Thus, a security error in one service does not lead to the collapse of the entire system.

Tech stack: Gitlab, Jenkins, VSphere, Docker, MySQL, Prometheus, Grafana

About client: The client is a company that provides a platform that allows you to store and distribute podcast audio files through their domain. It makes podcast production very fast and easy by taking care of the processing, editing, and publishing of your podcast.

Goals of the project: The main goal was to create an easily manageable and predictable infrastructure, as well as improve security. 

Challenges: The difficulties were to help manage AWS servers, optimize them, and improve uptime. After decomposing high-level tasks, we focused on two main goals: creating an easily manageable and predictable infrastructure, and improving security.

Solutions: 

• CI/CD Pipeline Implementation. Installation and configuration of Jenkins and test jobs for the development server were aimed at making development and testing processes easier and more predictable; 
• Monitoring and Logging Setup. With ELK Stack, we set up a log collection service and consolidated logs from the servers into one centralized repository. This allowed us to simplify the process of sorting and searching logs in the storage; 
• Security Enhancement.Set up of a VPN server with Pritunl allowed to provide reliable remote access with automatic failover; 
• Configuration Management. We used Terraform to create all AWS resources and Ansible for configuration. This gave the client a robust infrastructure configuration repository with change auditing and the ability to instantly copy any part.

Tech stack:AWS, Kubernetes, Terraform, Jenkins, Zabbix

About client: The client is an agency that works tirelessly to create an ideal environment for testing SEO techniques.

Goals of the project: The goals were to increase server security, manage data backups, automatically configure APIs, and improve infrastructure manageability. 

Challenges: Сonsidering the problems faced by the client (low performance, poor infrastructure manageability and security), provide a solution based mainly on moving the main HVSEO server to the AWS cloud. 

Solutions: 

• AWS Cloud Migration. Transfer of the main server to AWS provided better manageability, reliability, and fault tolerance. All servers are now located together, resulting in faster communication and data transfer; 
• Auto Backup System. We created and configured a backup system for all AWS servers and implemented AWS Lambda to remove backup files. This allowed the client to reduce the performance degradation associated with data backup and disaster recovery; 
• Monitoring System Setup. All of our customers' servers were added to our Zabbix-based monitoring system as a security and failure prevention measure; 
• Security System Improvement. There was a need to install Fail2Ban in order to log files and ban malicious IP addresses (IP addresses with a lot of password rejections or looking for exploits). Because too many IP addresses with different names keep trying to connect to the HVSEO system; 
• API System Automation. Creation of a script detected the issue with an IP and notified the HVSEO team when it occurred. Before, the client would change the IP manually. Currently, the team saves a lot of time having the system change instances every day to avoid IP banning; 
• Database. We reconfigured the database parameters to reduce CPU usage. As a result of Nginx server reconfiguration, application and site performance improved to a faster and more stable operation.

Tech stack: Docker, Ansible, RabbitMQ, Ruby  

About client: The client provides a learning management system for corporate onboarding, development and assessment with flexible integration capabilities.

Goals of the project: Provide the platform with horizontal scalability, and automate the infrastructure with high availability and fault tolerance. 

Challenges: The application was implemented on mare-metal Hetzner servers that grew to be an issue with the growth of popularity for e-learning solutions. Since the platform's amount of traffic expanded, the maintenance cost and the need for horizontal scaling arose too.

Solutions: 

• Kubernetes implementation. Kubernetes allowed us to implement an auto scaling solution that prevented infrastructure failure when traffic was high due to load balancing; 
• Instance-based infrastructure. With the instanced approach available from Hatzner, we were able to make the system repeatable. This means that scaling the application took much less time and only one utility;
• Monitoring and alerting system. We supplied a monitoring solution that is based on Prometheus and Grafana for virtualization as the most effective combination for monitoring and alerting among current DevOps practices;
• Centralized logging system. A new logging system for the Locky-based application reduced the number of instances, which improved application performance and allowed us to reduce infrastructure maintenance costs;
• Database. Adopting a clustered database increased the availability of the application because the solution protected the nodes from failure if one of the nodes failed.
• Backup system. We have backed up Elasticsearch and ClickHouse. The backup system has been updated with a policy that removes unnecessary or old backups, saving space and money.
• Cost reduction. It was necessary to reduce the cost of maintaining infrastructure, with a system that automatically deletes unnecessary backups and an infrastructure that uses only the resources it needs.

Tech stack:  Zabbix, Ansible, RabbitMQ, Ruby

About client: The client is Web3, a young game company whose main product is an open-world multiplayer game.

Goals of the project: The company asked OpsWorks Co. to build a scalable and cost-effective infrastructure for Web3 multiplayer on AWS.

Challenges: The company's core application was ready to be released for beta testing. The main concern was the accessibility of the application and its components. So the team decided to use AWS Elastic Kubernetes Service, that out of the box provides high availability, cost efficiency, and has built-in mechanisms for "blue-green" deployments and rollbacks. 

Solutions: 

• Accessibility and scalability. To increase the availability of the application, in addition to the standard solution of having multiple replicas, we configured Horizontal POD auto scaler and cluster auto scaler, which, working together, allow the application to scale depending on the load. As a result, the infrastructure we created is able to serve almost any number of users.
• Security. The current infrastructure has not yet reached the final architecture. In the next iterations, we are going to improve infrastructure security and focus on improving the Docker image of the application, collect more business metrics from the application, and perform fine-tuning based on the feedback collected.

Tech stack:AWS, Kubernetes, Helm, Postgres, Redis, Nginx Ingress, Datadog Vector, Grafana, Github Actions, OpenSearch, Go-lang

About client: Our client is a French сompany whose main goal is to make life easier for drivers. The company has developed a platform that allows car owners to deliver their car to any location in the platform's service area. 

Goals of the project: It was necessary to implement a cost-effective and secure platform for development, testing and production. The solution had to be resilient to any failures and require almost zero operational effort. At the same time, the infrastructure didn't have to limit the fast pace of the project.

Challenges: Due to strict security requirements, our team had to review the current infrastructure and strengthen it. Since client applications use some services managed by third parties, such as MongoDB and ElasticCloud. 

Solutions: 

• Security upgrades. Because the client was using third-party services, our team implemented VPC peering with third-party providers to carry sensitive traffic over the AWS backbone. In addition, we moved their scheduled job to AWS Batch, which reduced the cost of it. For security purposes, we have implemented a strict MFA rule for a wide range of services.
• AWS services. Our engineers make extensive use of AWS services for development. Code is stored in CodeCommit and CodeBuild, CodePipeline and CodeDeploy are used as CI/CD. AWS ElasticBeanstalk was chosen as the platform for the application.
• Development and adaptation. The client application is still evolving, so we continue to adapt it to AWS services by selecting the most cost-effective instance size, customizing the environment, and selecting and recommending a possible architecture to meet the application requirements. It is still using some resources in Azure and is not ready to migrate them. 

Tech stack: AWS EC2, BeansTalk, Application Load Balancer, VPC Peering, CodeBuild, CodePipeline, CodeCommit, S3, CloudWatch, Lambda, SNS, SQS

About client: The client is a company that provides a cloud-based SaaS platform that offers comprehensive and reliable solutions for the retail sector.

Goals of the project: Provision of qualified and proactive system administration, acceleration of Magento sites loading and automation/creation of new Magento instances. 

Challenges: The biggest issue was the fact that Monitoring was not configured, the server performance was insufficient, and therefore required the migration of the infrastructure to the cloud and the integration of Zabbix 6.

Solutions: 

• Linux Servers Configuration. The solution to optimize the servers was to configure them so that the servers have high fault tolerance and do not fail under high load. To balance the high load, our team used Nginx, PHP-FPM;
• Magento Instances Automation.When a new client comes, we use a configured (with Magento) VM template for a new marketplace. This allows the client to avoid overspending human resources when reaching a new client – everything is autonomous;
• Fault-tolerance Enhancement.The customer was struggling with database issues, which was using Galera Cluster. So we switched to a simpler replication solution based on Master-Slave, the system became more stable as it became easier to manage the current setup;
• Monitoring Configuration. As a general monitoring solution, we added Susoft to our Zabbix monitoring system. It helps us to get notifications about every lag in the system. 
• Access Rights Management. While administering the servers, our team encountered an access problem. The reason was the execution of Magento commands, through the wrong user. The problem was resolved with the execution of commands through the correct users.
• New Magento Instances Automation. The solution that is based on developing a script allows store managers to create new instances by simply clicking a single button in the app.

Tech stack: Zabbix, Docker, Ansible, Ruby, Saltslack

About client: The client is a company that has a Magento based e-commerce site and is selling healthy food, vegan snacks, lean meats and muscle supplements across the UK, aimed mainly at fitness enthusiasts and even professionals. 

Goals of the project: The launch of a feature-rich website on the Magento platform, conduction of an in-depth audit and provision of speed optimization. 

Challenges: Problems with the infrastructure and the lack of monitoring to identify the cause of the problem. It turned out that the infrastructure was managed by a third party and no one had access to it.

Solutions: 

• Troubleshooting infrastructure issues. Our goal was to create a new infrastructure using the Infrastructure as Code approach with Terraform and Ansible. The migration went smoothly and caused no downtime. 
• Monitoring and logging. We installed Prometheus, Grafana and Alertmanager, which provide the client with full observability of the infrastructure and early notification of emerging problems.

Tech stack:Kubernetes, MySQL, GitLab, CloudFormation

About client: The client is one of the largest independent providers of financial services in Central and Eastern Europe, which offers a full range of banking services to individuals and corporate clients. 

Goals of the project: Creation of a solution for scalable IT infrastructure, development of redundancy and reduction of downtime.  

Challenges: Since the client works in banking, we were faced with special security measures that had to be followed. 

Solutions: 

• Kubernetes cluster setup. Installation of Kubernetes gave the opportunity to manage the load at all times and ensure sufficient capacity for all bank users. This allowed the client's IT infrastructure to scale automatically.
• Logging and Monitoring. Open Distro and Prometheus were used to create a proactive monitoring solution that helped the client team stay on top of any malicious events. A custom parser was developed so that the client could get the logs in the format they wanted.
• Audit of the cluster security. Our team conducted an audit to strengthen security measures in the bank's IT infrastructure. The audit resulted in documented recommendations for additional security measures.

Tech stack:Ansible, Terraform, Kubernetes, Docker, Gitlab, VMWare, Hashicorp Vault, Prometheus, Grafana, ElasticSearch

About client: The client is a company that provides a sales and marketing automation tool for family entertainment centers that allows centers to manage e-commerce and use the most effective marketing channels.

Goals of the project: Automation and creation of infrastructure using IaC for customer systems, configuration of CI/CD for their application using Jenkins, as well as installation of monitoring/notification system. 

Challenges: The environments for each client were manually created by the company, so it was necessary to set up CI/CD for the app as well as to automate the provisioning of infrastructure.

Solutions: 

• Monitoring and Logging. It was significant to implement a real-time monitoring and alerting solution, including Prometheus and Grafana, and establish a list of technical and business metrics to be tracked; 
• CI/CD Pipeline Creation. A full infrastructure audit gave the chance to define the process flow. Having agreed on all the container metrics, we built several pipelines that allow the client to automate the deployment of the application based on the source code and enabled one-click deployment of the application in different environments with the configuration required for a particular client; 
• Infrastructure as Code. We created Terraform scripts and Ansible playbooks for deploying Jenkins servers, Prometheus servers with Grafana, and ELK Stack. This helped to automate the creation, copying, and deleting of the infrastructure; 
• Cost-efficiency.The aim was to configure the AWS ECR repository image lifecycle as well as AWS Cost Explorer AWS Cost and Usage Reports. As well as setting up cost of service accounting separately for each client and each client environment. This allowed the client to calculate infrastructure costs for each client using resource tags by client name and environment.

Tech stack:Ansible, RabbitMQ, Ruby, Saltslack

About client: The client is a company that has developed a leading e-commerce optimization platform for brands. The company's system helps brands appear first among the millions of SKUs available online.

Goals of the project: Reduction of cost of cloud services, routine processes automation, construction of an easily managed cloud ecosystem, reduction of deployment time and minimization of failures, improvement of website performance and reduction of time-to-market of the released features. 

Challenges: Due to the limited technical documentation of the project, it was not possible to perform a complete system analysis. Therefore, it was necessary to audit the infrastructure, deployment pipelines and monitoring system.

Solutions: 

• Cost Reduction.The client had many AWS tools and services installed but not using them, so there was an overrun problem. So we stopped the services that were in place but not being used by the client and performed an in-depth analysis of the existing servers, which allowed us to optimize processes;
• Routine Processes Automation. We automated copying, creating, deleting, and migrating servers. Creating a new production environment for a client now requires the click of a button in the CI/CD tool;
• Cloud-based Ecosystem Construction. Since our primary goal was to reduce costs, we decided to migrate the infrastructure to Kubernetes, which consequently simplified server management;
• Deployment Optimization. Previously, when testing a feature, engineers would simultaneously deploy the code to dozens of servers. So we used Jenkins to optimize and develop the infrastructure. With this technology, test environments are created and destroyed depending on the needs that arise; 
• Monitoring Flexibility. We use Prometheus to monitor the cluster and Zabbix as a support tool. In addition, the ELK Stack implementation was used for logging.

Tech stack:AWS, Kubernetes, Terraform, MySQL, GitLab

About client: The client is a company that is one of the largest providers of free telemedicine solutions that allow physicians around the world to connect with their patients quickly, easily and efficiently, considering the patient experience. 

Goal of the project: Our main goal was to reduce costs without compromising performance and security, as well as to scale and ensure uninterrupted service. 

Challenges: The company's HIPAA compliance was ensured by a third-party contractor that provided off-the-shelf MySQL servers and databases that were pre-configured for HIPAA compliance. Since most AWS tools are HIPAA compliant by default, we decided to migrate the company's infrastructure to AWS, which would not only ensure compliance and allow the infrastructure to scale quickly and securely, but also optimize infrastructure costs. 

Solutions: 

• Providing fast application scalability. The first stage was conducted without rebuilding the architecture, but in the future we're going to build everything on AWS Elastic Kubernetes Service, which will give us even more efficiency and availability; 
• Configuration setup.After fine-tuning docker-flow-proxy, it was possible to use all processor cores equally and reduce the number of working EC2 instances without sacrificing overall performance.

Tech stack:Kubernetes, Jenkins, Amazon web services, CloudFormation