When we think about websites and the infrastructure they run on, we tend to oversimplify. In the web development and web performance industries though, we know better. Gone are the days where even a semi-popular site could be handled by one server. Like the CDN, there are other components that need to be considered, such as the load balancer. A load balancer is a separate server which acts as a reverse proxy and directs incoming user traffic to one of many servers connected to the load balancer. This ultimately eases the load for all web servers.
An analogy I came up with is: think about a single waiting line at a department store. You have customers queued up (vusers/clients) and there is an employee (load balancer) at the front directing customers to different checkout registers (web servers). Of course, the way the load balancer directs traffic depends on its programming, but you get the picture.
At this point, you should see the advantages of having many checkout registers (web servers) and an employee (load balancer) rather than just one checkout register, even if it is a fast one.
I ran a couple of load tests on our large demo Magento webstore that can scale to a 5,000 concurrent user benchmark. For one of the testing iterations, I used an AWS Elastic Load Balancer to distribute load between three large Magento servers. This whole experiment will be detailed in a future post, but for now I wanted to go over what a load balancer is. Since running multiple web servers also costs more, you want to see benefits. Without going into too much detail, let’s take a look at the htop system monitoring daemons for two scenarios:
- A single large Magento server
- A load balancer serving three large Magento servers
Diagram 1. Single Server Monitoring (click to expand)
Diagram 2. Load-balanced Multi Server Monitoring (click to expand)
Simple Analysis of CPU Utilization (in %)
The difference might be difficult to see right away. These snapshots were taken at around the same elapsed test time for two identical load tests: 5 minutes, 1300 vusers. Check out Diagram 1, there are 32 vCPUs averaging at about 40% capacity. Now if you check out Diagram 2, the load balanced one, there are three 32 vCPU systems (each running their own htop) and you can see that a majority of the CPUs are sitting idle while only the first few are working. A quick analysis shows that the overall average of CPU utilization is significantly lower for the load balanced system than the single server.
Load balancing is a method to improve scalability, in addition to using a CDN. If you run an ecommerce website, it is a good idea for you to not only include a CDN distribution, but also a load-balanced multi-server architecture. We’ve gone over CDNs and their benefits extensively, so I was sure to include them in all the load tests performed against our Magento store. What other techniques do you use to optimize your web infrastructure?