I was reading a post about how the performance of Apache wasn’t quite as good as Nginx, but I got sidetracked by a link on the article that lead me to a broader study that was interesting. An informative performance benchmark comparison was published on http://www.trustleap.com/ where the author concludes that G-WAN running on Linux using a C language-based application has unbelievably better performance than any of the other tested combinations.
First, let me say that I love seeing the metrics. Second, I would love to run a test like this with LoadStorm. Third, my inner geek cynic kicked in as soon as I realized that TrustLeap is the producer of G-WAN. Maybe these numbers are suspect because of their source, however I want to share them with you.
Comparing Apache/Linux, GlassFish/Linux, IIS/Windows, and G-WAN/Windows & Linux
The data has some aspects that I cannot fully understand, but I focused on the first graph that shows all five combinations of web server, operating system, and language. It displays performance metrics of concurrent users and requests per second registered at the failure point for each. There is also a graph depicting the CPU utilization for each tested system as well.
I put these in a table to easier see the summary for comparison purposes:
| Server/OS/language | Users | RPS | CPU | Index |
|---|---|---|---|---|
| G-Wan/Linux/C | 1,000 | 7800 | 10% | 10 |
| G-Wan/Windows/C | 750 | 782 | 10% | 6 |
| Apache/Linux/PHP | 340 | 488 | 100% | 5 |
| Glassfish/Linux/JSP | 690 | 2 | 94% | 4 |
| IIS/Windows/C# | 5 | 4 | 88% | 1 |
I realize there can be many factors that affect web performance, and I understand that tuning each could make significant improvements in the ability to support requests. Still, these are astounding numbers! I’ve never seen a web app that can handle nearly 8,000 RPS with 1,000 concurrent users and only use about 10% of the CPU. It would be nice to see how much memory is consumed, but it probably doesn’t matter…if these numbers are accurate. My goodness! That’s tremendously efficient processing.
We use Apache running on Linux, and our LoadStorm application is written in Java. So I guess I should be telling our system architects to do some benchmarking our own load testing tool’s performance on G-WAN. This data shows that Apache/Linux/PHP performs well, or at least about average. Glassfish/Linux/JSP is about average as well. That seems reasonable to me.
This data points to IIS 7.0/Windows/C# as the slowest combination. I wonder if Microsoft has some different data. It just appears to have such an abysmal performance compared to the others that it makes me skeptical of the whole data set. Still, I find the study quite interesting in light of the other comparison benchmarks.
The author submits that G-WAN has such better performance because it does not starve the kernel of the OS. In the article there are points made about saving money and space due to fewer servers, which is more green (energy consumption) as well. The author claims that Java, C# or PHP’s footprint are in the 80-200 MB range, while IIS 7.0+C# is about 1 GB. I’m not sure about that, but if it is true it definitely points to enormous cost savings.
Conclusion
Performance benchmarks are useful information for web geeks like me that want to squeeze as much speed as possible out of our web applications. Making the right choice for the operating system and web server is all about picking the fastest set up. However, if my application code is agnostic to the platform, then why wouldn’t I go with the environment that produces the best performance metrics? Speed is money, and there are performance testing statistics to prove it.