Optimizing an NGINX Web Server Running On Amazon EKS (Fargate) – Part 3

Optimizing NGINX is not easy, let alone coupling it with resource sizing in Kubernetes. The solution? Automated tuning. This blog post is based on a webinar that was co-hosted by Shashi Raina, a Partner Solution Architect at AWS, and Tomer Morad, Co-Founder of Concertio. It is divided into three parts: In the first post we explored containers, VMs and Kubernetes on AWS, and in the second part, we dove into automatic optimization and how to optimize an NGINX website that is hosted on Amazon’s EKS, or more specifically, Fargate. In this final part, we are ending with integrating Continuous Optimization into the Continuous Deployment process, and with a discussion on post-optimization steps to generate valuable insights.

Continuous Optimization of Kubernetes Deployments

The best way to leverage automatic optimization in a Kubernetes environment is to implement it as a step in the Continuous Deployment pipeline, and this is called Continuous Optimization.

As the software code and components change, so will the optimal resources that are required to run the application. It would be impractical to have a person rerun all the experiments for every commit to find the optimal resources. The solution? Continuous Optimization of Kubernetes Resources.

The way Continuous Optimization works in this case is as follows:

  • After deployment, the CD process branches out to Continuous Optimization.
  • Optimizer Studio selects new Kubernetes resources (e.g. memory and cpu per pod) and new NGINX configuration tunables
  • The new resources and configurations are applied on a staging cluster or in a different K8s namespace
  • Optimizer Studio invokes a performance test
  • Optimizer Studio receives the feedback from the performance test, doing this (select-test-measure) iteratively, to configure the cluster with different configurations
  • When Optimizer Studio finds a better (above a threshold, e.g. 1-2%) configuration or an exit criteria has been met (e.g. a time limit on the Continuous Optimization process), the better-performing configuration is committed to the repository.
  • When a new commit arrives, it will be released with the optimized configurations.

This is just one example of how to implement Continuous Optimization in the CD process, and many other flavors exist.

Visualizing the results

One of the better ways to understand the results of a performance tuning experiment is to visualize the results. Concertio’s experiment management system receives real-time events from the experiment and plots the results in an easy-to-understand manner. Below is the graph of the NGINX requests per second (on the vertical axis) Vs. the number of requests per dollar (on the horizontal axis). Users can choose any metric in the axes to adjust and tailor the view for their specific experiments.

Clicking on a specific sample shows the values of the tunables that were used to generate it. This allows generating insights from the results such as which tunables were most influential and to what extent the system’s performance metrics are stable. Eventually, such insights drive further experimentation that can lead to improvements in the optimized system metrics (performance, cost, etc.).

Refinement: Trimming non-influential tunables

Because of the way optimization algorithms work, we might end up with a high number of tunables that are different from baseline, but do not necessarily contribute to the optimization. It is desirable, therefore, in many situations, to filter out those tunables that do not contribute much.

For these scenarios, we have another processing step that we call refinement. It is a post-processing step where we revisit the knobs that were changed from baseline and try to estimate the effect of each knob on the result. The refinement algorithm keeps the tunables that are influential, while filtering those tunables whose effect on the target metric is small. Optimizer Studio makes sure to sample the minimum number of additional times required to trim the non-influential tunables.

To learn more about Concertio Optimizer Studio, please click here!

Contact us

Have any lingering questions or need a walkthrough? Send us a message and we’ll get back to you.

Download Concertio HFT One Pager

Download Mellanox Solution Brief

Schedule A Demo

Let’s set-up a time to show how to unleash your system’s full potential.