Performance Testing can be viewed as the systematic process of collecting and monitoring the results of system usage, then analyzing them to aid system improvement towards desired results. As part of the performance testing process, the tester needs to gather statistical information, examine server logs and system state histories, determine the system’s performance under natural and artificial conditions and alter system modes of operation. Performance testing complements functional testing. Functional testing can validate proper functionality under correct usage and proper error handling under incorrect usage. It cannot, however, tell how much load an application can handle before it breaks or […]
This post is part of a series: First – Previous – Next Introduction We’re past the halfway mark for our e-commerce benchmarking experiments where we score a bunch of open source e-commerce platforms for performance. Next up is VirtueMart, which is an e-commerce extension for the Joomla content management system. The first time through, I had anticipated VirtueMart to be a standalone e-commerce platform. VirtueMart’s official site also doesn’t mention Joomla on the front page, or that VirtueMart is an extension. It’s funny because VirtueMart’s official Twitter page shows that information but their official website doesn’t. The VirtueMart Sample […]
This post is part of a series: Next E-commerce environments on the internet come in all shapes, colors and sizes. In our next series of experiments, we delve into the world of open source e-commerce platforms. We are already familiar with one of these: Magento Community Edition. I’ve blogged about Magento extensively in the past, but now it’s time to test it against other free e-commerce platforms. Over six blog posts, we are going to run six different e-commerce platforms and score them on performance. The first one covered will be Magento CE. Our slightly modified Magento CE store […]
WebPageTest (WPT) is a free, open source tool that is used to evaluate web page performance. The great thing about WPT is that it can be utilized by people with all levels of expertise, and has many options for advanced settings and customizations. The public WPT instances cover many scenarios, but private instances can be useful, especially when testing sites that aren’t yet publicly accessible. We decided to install our own private instance of WPT using an Amazon Micro EC2. However, as the WebPerfLab, we couldn’t just leave it at that. Come on, you should know us by now! We […]
If you’re looking for insights on how to improve your site’s performance, webpagetest.org is a good place to start. If it’s your first time running tests, the page test results can be a bit overwhelming, especially to someone new to performance analysis. This blog will provide some quick tips on where to start in deciphering the results of a test. Summary Page After running a test with the default settings, webpagetest.org (WPT) provides a summary of the test results. Sometimes looking at this page can be enough to point you in the right direction on how to optimize your site. […]
This post is a continuation of my Scaling Magento testing experiment (see part one). Testing Phase After getting the first testing environment setup, it was already time to test. Luckily for me, the VUser activity script was already finished back before this project was proposed. So running an identical load test was as simple as doing a Copy & Edit to our previous load test. Here is the activity of the two tests. In this case, x refers to the xth VUser in the load test. VUser x hits the homepage VUser x logs in as “wil”, a previously generated […]
Recently we decided to upgrade our WordPress core to version 3.7.1 and we figured we would perform some tests to see how the change affected us. I took our baseline of 11 performance tests (using webpagetest.org) and a single load test (using LoadStorm) to compare to our new set of data after upgrading to the new version. It was fun to see how things would perform, but the results were not shocking. There was virtually no difference between the performance tests. The page load time was on average 1.22% slower, and we’re talking tiny fractions of a second so the […]
When Motorola created and implemented the Six Sigma quality assurance methodology, from 1986 to 2001, they reported saving $16 billion, a 4.5% savings, in revenue. This was attributed to Six Sigma’s ability to cut out waste and optimize efficiency within the company. The figures are also impressive because many methodologies promise results, while some only provide more red tape. Like in manufacturing, the need for a formalized methodology in performance engineering remains the same. In performance engineering, we have our own testing methodology and best practices. A performance testing methodology can be a hybrid of the best parts of other […]
Thoughts on Performance Culture The performance culture has changed drastically since it’s early beginning. Notifying everyone of it’s important isn’t important anymore because everyone knows that it is important. Through events like Velocity, the word has gotten out to the mass that performance matters. It matters to the business and the users. As people are becoming aware of web performance, more questions are raised as how to implement a culture of performance in the industry. To answer that, some companies are integrating web performance team into their workforce. The web performance team is usually tasked with scavenging the web application […]
Tis the season…. Why haven’t large corporations learned the value of load testing yet? How could e-commerce sites not be ready to handle the increased traffic for the busiest selling period of the year? Unfortunately, with the high traffic of Cyber Monday, certain websites were unable to meet customer expectations. Ok, let’s be honest – bad web performance cost some sites money and really made buyers angry. Let’s conduct a review to see what we can learn. First, a few relevant statistics: For the five-day period from Thanksgiving through Cyber Monday, online buying from desktop computers totaled $5.3 billion, up […]