The post Load Testing Mobile Apps appeared first on LoadStorm.

LoadStorm PRO uses HTTP archive (HAR) recordings to simulate traffic, and we normally create these recordings using a browser’s developer tools on the network tab to record all requests sent to the target server. In this article, I’m going to introduce you to two ways to make recordings of user traffic that can be used in LoadStorm. The first involves packet capturing from your mobile device, and the second involves a Chrome app called Postman used in combination with the Chrome developer tools.

Simulate traffic by using Packet Capturing

If you like to keep things simple, then this method should save you the trouble of manually creating requests as shown in the next method. You’ll need a packet capturing mobile app (such as tPacketCapture) that stores requests in the PCAP file format, and allows you to share the PCAP file. The PCAP file generated by the app can then be converted to a HAR file to be uploaded into LoadStorm for use in a load test. To do this, you can use PCAP Web Performance Analyzer for free without needing any setup, or you can install your own converter from the pcap2har project on GitHub.

For Android devices, this method works as follows:

For iOS devices:

At this time, packet capturing mobile apps are only offered on Android devices. Apple products do not support direct packet capture services. However, if you connect your iOS device to a Mac via USB then you can use OS X supported software (such as Cocoa Packet Analyzer) to capture the packets, as described on the Apple developer site.

Record requests made in Postman using Chrome developer tools

To make a recording with the Postman app, follow these steps:

REST

Making RESTful GET or POST requests will rely heavily on your knowledge of how your mobile app interacts with the application server. If you have a network log that shows the incoming requests from the mobile application, that can help simplify the reconstruction of those requests in the Postman app. Postman actually offers 11 methods for interacting with an application server, and of these we’ll typically only need to use GET and POST requests:

• GET – For a RESTful request to interact via a GET request you’ll usually need to include some parameter(s) as a query string that tells the application server what you would like returned. For example, you could request to see your all of your tweets on your twitter timeline.
• POST – POSTs are used to give the target application new information like adding a tweet to your twitter account from your phone. The content that you would like to POST is sent via the request payload in one of three options; form-data used for multi-part forms, x-www-form-urlencoded used for standard forms, and raw used for a variety of content (JSON, XML, etc.).

In some cases you will also need to add some form of authorization token in the request headers to let the application server know you have the rights to GET or POST information. Postman offers a few options to assist with authorization settings (i.e. Basic Auth, Digest, and OAuth 1.0), but you can always manually input the authorization header if you know the name of the header, the value, and format to send it in.

SOAP

Even though Postman is primarily designed to work with RESTful API calls, it can also work with SOAP.

To make SOAP requests follow these steps:

Additional information about creating SOAP requests can be found on the Postman blog.

Video Tutorial

This video is a short guide on recording a HAR using Postman in combination with the Chrome developer tools.

Also check out Ruairi Browne’s Postman tutorial covering RESTful API GETs and POSTs to Twitter as well as OAuth1.0 credentials.

Upload a Recording

To load test a mobile application using LoadStorm PRO, a HAR recording must be made to simulate the traffic. Once you’ve decided on a recording method, all you have to do is upload your HAR file into LoadStorm, and you’ll be on your way to load testing your app and ensuring end-user satisfaction. If you have questions or need assistance, please contact , visit our learning center, or leave a comment below.

Sources:

• http://stackoverflow.com/questions/9555403/capturing-mobile-phone-traffic-on-wireshark
• https://play.google.com/store/apps/details?id=jp.co.taosoftware.android.packetcapture&hl=en
• https://code.google.com/p/pcaphar/
• http://pcapperf.appspot.com/
• https://github.com/andrewf/pcap2har
• https://developer.apple.com/library/mac/qa/qa1176/_index.html
• http://blog.getpostman.com/index.php/2014/08/22/making-soap-requests-using-postman/
• http://youtu.be/fhPb6ocUz_k

The post Load Testing Mobile Apps appeared first on LoadStorm.

The post Super Bowl Ads = Lots of Traffic. Are Sites Ready? appeared first on LoadStorm.

Back in 1999, Victoria’s Secret made a big splash with their Super Bowl Ad. The ad was one of the very first ever to tie in TV and web. It promoted an online lingerie fashion show and over a million viewers logged on to watch. . . crashing the site.

Since then, using Super Bowl ads to drive traffic to a website has become a very popular marketing technique. Many companies use the massive marketing power of a Super Bowl ad to drive viewers to their websites and take a specific action (signing up for some promotion, voting for a favorite team, etc.). The resulting massive spike in traffic that hits the websites, should be very much expected. However, every year there are a handful of websites that have website crashes resulting in massive social media and PR backlash.

In 2013, a study by Yottaa found that over 13 companies had Super Bowl ads that crashed their websites, including Coca-Cola, SodaStream, Calvin Klein, and Axe. Coca-Cola invited viewers to log on to CokeChase.com and vote for their favorite team, Axe offered a sweepstakes to send someone to space. Both resulted in massive numbers heading to the websites to find crashed websites. The result? Viewers took to social media in droves :

One notable crash of Super Bowl 2014 was from Maserati. The ad was rumored to cost between $11 million and $17 million. It announced the new Maserati Ghibli and sent masses to the website MaseratiGhibli.us, which promptly crashed.

Load testing is critical for any website expecting a rush of traffic. Whether it is a rush from a major ad campaign or a launch, it is imperative that the website be able to handle the pressure of the traffic. The only way to know for sure that your site is prepared is to test it.

As a load testing provider, we see all of these examples as avoidable problems. Load testing allows companies to simulate a large volume of traffic hitting the website or application while monitoring the sites responses (response times, throughput, errors, etc.). Therefore, if the companies had load tested, it is quite reasonable that their web development teams would have found the performance bottlenecks, addressed them, and the crashes would never have happened.

Will we see any big website crashes from Super Bowl ads this Sunday? Comment below if you have any guesses of which websites will fail and then check back next week as we analyze the hard data from our very own testing done on Super Bowl Sunday 2015!

Sources:

http://www.usatoday.com/story/money/business/2015/01/21/victorias-secret-super-bowl-ad/22127891/

http://www.yottaa.com/blog/bid/265815/Coke-SodaStream-the-13-Websites-That-Crashed-During-Super-Bowl-2013

http://geekbeat.tv/maseratis-ghibli-superbowl-ad-crashes-maserati-website/

The post Super Bowl Ads = Lots of Traffic. Are Sites Ready? appeared first on LoadStorm.

The post Can load testing help stop hackers? appeared first on LoadStorm.

As we grow increasingly aware of these occurrence, we as developers need to go forward with the mindset that people will be trying to access our data. As the internet and technology permeates throughout physical stores, our information is becoming even more vulnerable to criminals who use online hacking methods against organizations.

What you can do.

There are numerous ways you can be proactive in protecting your website. One practice that is often overlooked is a combination of penetration and stress testing. Stress testing is the practice of determining how well a website functions in deliberately adverse conditions. Penetration testing is actively trying to break down security methods and access forbidden information. Typical actions in this testing test may include:

• Running several resource-intensive processes (consuming CPU, memory, disk, network) on the web and database servers at the same time
• Attempting to access and hack into the system and use it as a zombie to spread spam
• Doubling the baseline number for concurrent attempts to access a single website as a regular user.
• Attempting to gain cross-account access to gain information when logged into a system

When you evaluate and benchmark your system in this way, you can observe how your system reacts and recovers. As the array of different protocols and applications grow increasingly complex, malicious attacks can quickly bring down a site or exploit a lack of security.

Security breaches are camouflaged.

A previous Sony data breach that jeopardized 77 million users was actually disguised as a DDoS attack, which is an attack characterized with very overwhelming amounts of traffic. According to a recent study from RSA Security and the Ponemon institute, 64 percent of IT professionals in the retail sector have seen an increase in attacks and fraud attempts during abnormally high traffic periods. By testing your site’s response to a simulated attack, this type of security gap can be reduced, which is a proactive step towards protecting your site.

LoadStorm can be used as a tool to determine your site’s breaking point, and possibly, your site’s performance at its most vulnerable point. By simulating an attack in conjunction with a load test, you can now evaluate how network and security devices perform under stress, and isolate and repair flaws. After determining the weak points of your site, get to work implementing a more secure infrastructure. The idea is to close the gap between the attack and the response to the attack.

Readiness is Key.

Many companies make the mistake of launching before they are truly ready. It’s easy to get caught up in launch deadlines or the pressure to conserve time and resources that could be spent on testing. However, with the diverse competition that exists today, many customers will only give you one shot. If your site or your customer’s data has been compromised, don’t be surprised if they leave and do not return. It takes a lot of work to build trust with new users. Don’t lose the value of your hard work on a vulnerable system. As millions of transactions take place on the internet every day, it’s up to us to make sure that our systems are prepared for an attack; that security provisions like network firewalls, flood controls, intrusion detection and prevention, and application firewalls have all been tested thoroughly with realistic simulated traffic.

It’s up to us to ensure that our sites are ready for high traffic and that our data is secure.

The post Can load testing help stop hackers? appeared first on LoadStorm.

The post Introducing QuickStorm appeared first on LoadStorm.

What is a QuickStorm?

QuickStorms are small, short load tests that can be used to evaluate site performance. All you have to do is enter your site’s URL. LoadStorm will generate a load test against the URL, scaling from 1 – 10 virtual users over 10 minutes.

How it works

LoadStorm PRO automatically creates a recording of every request made on the URL you provide. Next, we simulate the traffic escalating from one to ten virtual users, targeting that URL.

Try a QuickStorm test of 10 users on your site by entering the URL here:

Test Results Overview:

Summary

The test results summary provides totals for key metrics from the test. The total amount of requests as well as the overall average response times for those requests are displayed.
Since it’s a QuickStorm, there will only be one script. All of the servers used for the URL will be displayed, so if you want to view performance by servers, you can filter for that.

What you want to see here:
In general, you want to see good average response times, peak response times that are near the average, no errors, and low throughput. In addition, low total data transferred is a good sign. It’s a good idea to avoid page bloat.

Response

This includes all the response times data for the site. Every request included on the site is displayed, detailing the amount of time each resource was requested, the size of the request, the average amount of time it took to get a response, and the peak amount of time it took to get a response.

What you want to see here:
Good response times for requests are under 250 ms, while 500 ms is still generally acceptable. Any longer and you’ll be in trouble if the volume of traffic increases.

Errors

If any errors occurred during the test, they will be shown here. The number of errors and the types will be shown here. The Errors by Resource table will list which resources yielded errors. Usually, errors won’t occur at such a low volume of traffic, but there are some times web developers are simply unaware of the problem. Eliminating these errors is an easy way to boost performance.

What you want to see here:
You want to see 0 errors here. Any errors here should be completely unacceptable. You can do better than that!

Pages

In LoadStorm, pages refer to each unique web page hit. Since QuickStorms are only performed on a single URL, you can expect to see only that single page listed here. Statistics for this page include the number of times each page was requested. For QuickStorms, this number usually reaches just over 100 times. This means your page was visited by virtual users just over 100 times in ten minutes!

During a regular load test, you would typically make a recording of several different pages to mimic the traffic you expect to see on your site. For example, a blog would include a recording of a user browsing to the homepage as well as many different posts.

What you want to see here:
You want to see a zero failures for your page. When it comes to average and peak completion times, you should aim for pages completing in under 5 seconds.

Transactions

This is a feature you won’t be able to experience through QuickStorms, so you can expect to not see any data here. A transaction is a collection of requests that you designate in order to monitor specific site transactions. One example of this would be a search transaction. It’s important to have stellar performance in this area, and by selecting to monitor the search transaction, you can get feedback here.

What you want to see here:
Nothing. To use the transaction feature, sign up for a free account with LoadStorm PRO.

Requests by Time

This data includes the results of the different requests, summarized by time. Here you can easily view any differences in response times between the first minute of the test (1 vuser) and the last minute of the test (10 concurrent users).

What you want to see here:
You want to see comparable response times at the beginning of the test, and the end of the test. Drastic differences can point to an underlying problem that’s just waiting to be exposed.

Export

All of the raw test result data is available for download in a CSV file. Like the Requests by Time, this data usually becomes available within 10 minutes of test completion. If you would just like to show off your results, we also have the test statistics compiled for export as a CSV report.

What you want to see here:
Your site’s excellent results in a pretty report you can show off to your friends or enemies! Try it! It’s fun! And you get a URL you can share. Please note that QuickStorms are limited to 10 per day!

The post Introducing QuickStorm appeared first on LoadStorm.

The post Cyber Monday Performance Evaluations appeared first on LoadStorm.

This year, we selected 29 major e-commerce sites and used LoadStorm to run several tests to compare their performance on the Wednesday before Thanksgiving with Cyber Monday. We created scripts for each site to model typical e-commerce user activity. Each script would hit the homepage, search for a product, add a product to the cart, and then visit the cart. Then we ran our performance tests for 10 minutes at a time, scaling from one to ten virtual users (vusers).

Here’s what we found:

Average Page Completion Time remained relatively reasonable.

Out of the 29 companies, 7 slowed down:

Walmart’s average page completion time increased from 2.9 seconds to 7.8 seconds. That’s huge! Amazon, on the other hand, remained consistent, with an average page completion time of just 1.6 seconds on both days. Average Page Completion Time remained nearly the same for the majority of the sites.

Six companies sped up:

Performance Error Rates increased.

Our preliminary load tests revealed zero performance errors across the board, with one exception. Monday, however, was a different story, as we saw five different companies experience performance errors. This included request read timeouts, request connection timeouts, and even some 503 (service unavailable) errors.

The exception to the increase in performance error rates was Best Buy. Interestingly, we saw Best Buy experience seven request read timeout errors on Wednesday (on product pages and search results), but none on Monday. This seems to corroborate the fact that they became overwhelmed with traffic over the weekend, but they appeared to have recovered gracefully by Cyber Monday.

Peak Page Completion Times increased.

Every site we tested experienced high peak completion times. Some of the best performers with the lowest peak page completion time on both Wednesday and Cyber Monday included Toys “R” Us, Brookstone (one of our customer’s – yay!), Ikea, and Amazon, with peak page completion time only deviating 7% from their mean.

Web Page Test Results

Web page tests were on each company’s home page were performed during testing on Wednesday and Monday as well. Surprisingly, the overall trend was a decrease in page load times. The average page load time from our web page tests decreased from 9.3 seconds to 5.6 seconds. We’re impressed!

Overall Performance

Nobody crashed while we were running our performance tests. Most sites appeared to perform reasonably well, but knowing that just one second delay could cost Amazon over $1.6 billion in sales over the course of a year those few errors matter immensely. Just 250 milliseconds, either slower or faster, is the magic number for competitive advantage on the web. So while none of the sites crashed entirely, whether or not they beat out their competition is another story.

Curious about how your site is performing? Try a QuickStorm test of 10 users on your site by entering the URL here:

Please note that none of the companies involved were contacted nor paid for participation in our experiments. These were just for fun. Here’s the complete list of the companies we tested:

The post Cyber Monday Performance Evaluations appeared first on LoadStorm.

The post Tips and Tricks to Troubleshooting PRO Scripts appeared first on LoadStorm.

When a recording is first uploaded, it behaves the same as when it was recorded; That is, it makes all the same requests, with no differentiating qualities. Until you parameterize it, our application will simply repeat all of the GETs, POSTs, and occasional PUT requests the same way they were recorded, but we handle a few things automatically such as cookies and most form tokens.

Often new scripts can have a bundle of requests and only a few of which that need parameterization to work as expected for every VUser that makes that request. These problems are often found when a request’s recorded status code does not match its status code from the last script execution. Problems such as a failed login request can cause subsequent requests to fail, causing a domino effect. For this reason, it’s always a good idea to start from the earliest requests and work your way down, checking that requests are behaving as expected during a single script execution. Even after it’s all fixed, a small load test is a good way to check that it is also working as expected for concurrent users.

When parameterizing a script, here are the key recommendations:

• Manage servers
• Start with first requests and proceed sequentially
• Examine POSTs to make sure you are getting the response you expect (e.g. login)
• Check script Settings such as Think Times, Timeouts, etc.
• Re-execute the script after changes
• Run a small load test to confirm your script is working as expected (debug test)

Tips and Tricks to Troubleshooting

Where to Begin

Switch to the Manage Server sub-tab and ignore all non-essential third-party servers. The goal is to test your web application, not someone else’s. If you feel the service they’re providing you is critical, please contact for help with server verification. After these have been ignored, execute the script again. This will make things easier when determining which requests you need to work on because the unnecessary requests will now show a status of ignored instead of a response code.

Filtering

You can make it easier to find problematic requests by using our filtering options. To quickly bring all of the apparent errors to your attention click the All Errors radio button.

For the less obvious errors you’ll need to look for mismatched status codes by clicking the Recorded vs Last drop down and selecting the Status Code Mismatch option. As you begin looking at the Recorded Status column and comparing it to the Last Status column for each request you should disregard requests if they show cached or ignored for their last status. Typically the less obvious ones to look for change from a status 200 to a 302, or they were meant to be a 302 and now give a status 200.

If you’re unfamiliar with status code these are some of the most common:

• 200 = OK (usually delivers some content)
• 302 = Found (asks the browser to redirect to a response header called Location)
• 400 = Bad Request (often this means the URL has invalid syntax)
• 401 = Not Authorized (you need permission such as a login to proceed)
• 403 = Forbidden (does not allow the target to be viewed with or without authentication)
• 404 = Not Found (missing the file at this target location)
• 500 = Internal Server Error (often means LoadStorm made a request that has something the server wasn’t expecting like a static token that should’ve been dynamic)

Another option to filter down to what could be considered your most crucial requests is changing the All mime-types drop down and selecting the text/html option. These requests usually represent each page of text content that your users would need to read and interact with.

Request Details Window

Once you’ve identified a request that you feel is causing problems double-click that request to open the details window. From here you can compare all of the details of a request from its recorded values with the values in the last execution. Any values that differ in the last execution will appear in red. Usually there are values that must appear in red because they need to be dynamic per user. Often these are tokens of some kind such as authentication, userIDs, sessionIDs, viewstates, incremental values, etc. If you find that something appears like a randomly generated token, but is colored black for the last execution then that indicates it was repeated as a static copy of the recorded value. This can often be the cause of problems with a request and cause a domino effect of problems with subsequent requests that rely on passing that token around or other chain of a events that need to occur. To fix this you’ll need to look at prior requests to see where this token is being set, and then modify the problematic request to grab the token from the prior request that would assign it.

Response Content tab in the Details Window

You can often find important information regarding the behavior of a request in the response content. This information could let you know many things such as why your login attempt failed, if you received a stack trace from .NET, or if the page contains some dynamic tokens that are needed in later requests.

Previewing Response Content in a New Browser Tab

We also provide you with a hyperlink near the top-right above the last execution half of the response text. This link will let you preview the text of the last execution in a new browser tab. It is especially useful if you’d prefer to read what’s on the page instead of scrolling through HTML code.

Response Content Validation

If you’re concerned that a request is going to deliver a custom error page that shows a status code 200 which isn’t shown as an error during a load test, then fear not. We have a feature that will allow you to put in place a validation that will flag it as an error under the conditions that you specify. Let’s say you’re expecting a login to send you back to the homepage with some message at the top welcoming the user, but in this case the login failed and now the server is still giving us a status 200 delivering us to the homepage without a welcome message such as “Welcome to our store, Michael!”. You can select the homepage request that comes after the login POST request, and then click the Validate Response Content button. This will display a modal window that allows you to choose text that you expect to see, or that you do not wish to see. So following my example we would expect to see the words Welcome to our store in every response we get for this request, and that’s what you would enter in the expected string field. Now even though this request isn’t appearing as an error under normal conditions we can flag it as an error during a load test.

Utilizing Custom Data to find Dynamic Tokens, URLs, and Strings

Whenever you identify a request that needs something changed to a dynamic value you’ll need to parameterize it to make use of our custom data selector. Depending on the need you’ll have to select the request and click the appropriate modification button to get started. Then in the next modal window that appears you’ll have to select the parameter that you wish to modify and change the modification type to custom and click the Select Data button. Now you’ll be in the custom data window where we gather all kinds of data for you to parameterize with from a convenient location. I’ll leave out the User Data and Generated Data tabs for now because this is really meant to focus on dynamic data that we’re grabbing from the responses to requests in your script.

• Form Input tab – Here we offer you all extracted names and values of hidden input fields that we find from the responses of requests that come prior to the selected request.
• Response Headers tab – This provides you with all of the unique response headers from older requests and if there were any duplicate response headers we’ll only display the most recent ones prior to the selected request.
• Response Text tab – While this is the most complicated one, it also allows the most control over what you wish to use. We have a drop-down that allows you to select the response text from any request prior to the selected of course. Once selected the preview box displays the appropriate content. Now comes the hard part, finding what you need. My trick for to make this go quickly is to use the browser’s find feature which can be opened by pressing the Command and F keys together, or the CTRL and F keys for Windows users. Now you should see a miniature search bar appear within the browser window. Begin typing some identifying text that will lead you to the string, but remember not to enter the string itself since it will be different each time. An example of this would be something like ASP.NET_SessionId=, JSESSION=, __VIEWSTATE=, form_id=, etc. I’ve also seen URL paths that are built dynamically into the anchor tags of links based on your session with the server. That could look like href=”/test/user/12JzntR93/profile” in which case you would want to look for the parts just before the string. Once you find it you’ll need to define a unique start string delimiter and an ending string delimiter to let our application now what the string is between. A quick example of this would be an anchor tag like <a href=”/some/path/uniquestring”>link</a> which we would then enter a Start String Delimiter of [<a href=”/some/path/] and an Ending String Delimiter of [“>link</a>] which I’m using the [ ] brackets to represent the input box itself.
• Cookies tab – The contents found here are rarely used, but in one occasion I needed to help someone parameterize a URL that was taking the JSESSION token set in the cookies and passing it allong with a URL of a later request. So in that case we needed to select the request and click the Modify URL button. Selected the token portion of the URL as the substring we wished to replace and copied it into the substring input field. Then using the custom data we selected the JSESSION token from the Cookies tab.

The post Tips and Tricks to Troubleshooting PRO Scripts appeared first on LoadStorm.

The post Cyber Monday and the Impact of Web Performance appeared first on LoadStorm.

In fact, the previous statement is much too modest. Cyber Monday is the biggest day of the year for ecommerce in the United States and beyond. As the statistics show, Cyber Monday has become a billion dollar juggernaut since 2010 – and it has only continued to grow. Last year alone, Cyber Monday was responsible for over $1.7 billion spent by online consumers in the US, a shocking 18% jump from the year before!

Since its inception in 2005, the Monday after Thanksgiving has become a potential goldmine for those with an online presence. It has helped to significantly boost revenue during the Christmas period for savvy businesses that have taken advantage of using the promotion. The “cannot-be-missed” deals are important to any Cyber Monday campaign, but having the website ready to maintain consistent and fast performance with the traffic rush is absolutely critical.

An unprepared business would expect an increase in business on Cyber Monday, but overlook the fact that more visitors = more strain on the performance-side of their website. And the more strain on a website, the more it will begin to falter when it matters most.

How web performance can cause your Cyber Monday to crash

During the mad rush of consumers looking to snap up some bargain deals, your website has to be prepared for the sudden visitor increase – otherwise your Cyber Monday will crumble before your eyes.

Last year Cyber Monday website crashes cost several large companies thousands of dollars in revenue. Motorola was offering a special price on their new MotoX, but the site was not prepared for the rush of traffic it would bring. Many customers experienced a very slow website, errors showing prices without the discount, and then the website crashed entirely.

In addition to losing customers who would have otherwise purchased that weekend, Motorola also had to deal with the PR aftermath. Unhappy would-be customers and the tech media took to social media, posting tweets such as:

In an effort to mitigate the damage, Motorola CEO issued the statement:

Moral of the story? Motorola lost out on thousands of dollars of sales and lost thousands of potential new customers forever, all of which could have been avoided if load and performance testing had been performed early. If they had load tested, Motorola would have been aware of the problems, found the causes, and fixed them before real users experienced them.

While many companies didn’t see full website crashes like Motorola, the rush of traffic still led to painfully slow websites and therefore a loss in revenue. A website must not only remain up and available, but also remain fast to navigate around. Just think of the amount of pages a potential customer might have to go through on your website. Now imagine if there were delays in between each page loading. Internet users are an impatient bunch, a one second delay can cause a 7% decrease in conversions and 11% fewer page views. And 74% of people will leave a mobile site if the delay is longer than five seconds!

Clearly, ensuring your website is constantly up and stable is imperative to maximizing profits for your business this Cyber Monday. Because the last thing you want to do is miss out on the most important day of the year for ecommerce and present your competitors with an opening to snag that business.

The stakes are high; you’ve got to make sure you are suitably prepared for the rush.

LoadStorm wants to help you prepare for the storm of traffic with a Cyber Monday special. We are offering one free hour of consulting and one free load test of up to 5,000 users. Check out our Cyber Monday page to learn more and to request your free test today!

The post Cyber Monday and the Impact of Web Performance appeared first on LoadStorm.

The post Stress Testing Drupal Commerce appeared first on LoadStorm.

Promet Source is a Drupal web application and website development company that offers expert services and support. They specialize in building and performance tuning complex Drupal web applications. Andy’s team worked with our Web Performance Lab to conduct stress testing on Drupal Commerce in a controlled environment. He is skilled at using LoadStorm and New Relic to push Drupal implementations to the point of failure. His team tells me he is good at breaking things.

In this presentation at Drupal Mid Camp, Andy explained how his team ran several experiments in which they load tested a kickstarter drupal commerce site on an AWS instance and then compared how the site performed after several well known performance tuning enhancements were applied. They compared performance improvements after Drupal cache, aggregation, Varnish, and Nginx reverse proxy.

View the below slideshare to see the summary of the importance of web performance and to see how they used LoadStorm to prove that they were able to scale Drupal Commerce from a point of failure (POF) of 100 users to 450 users! That’s a tremendous 450% improvement in scalability.

The post Stress Testing Drupal Commerce appeared first on LoadStorm.

The post E-Commerce Benchmark – Drupal Commerce appeared first on LoadStorm.

Introduction

Drupal Commerce is the last stop on our tour of e-commerce platforms. Drupal is a free content management system, and Drupal Commerce is an extension that allows you to build a web store with Drupal. This setup is similar to WooCommerce and Virtuemart, which both rely on their own content management systems.

If you missed our previous posts, we are load testing six different e-commerce platforms and scoring them based on web performance. We continue our e-commerce benchmarking series by giving you a glimpse of out-of-the-box Drupal Commerce and scoring it against the same load test standards as our out-of-the-box Magento store in the first blog post.

Setup

To streamline setup, I decided to use a bundle called Commerce Kickstart which contains a single package for both Drupal and Drupal Commerce. So you don’t have to worry about the in-betweens of installing the content management system and extension. How easy is that!

Commerce Kickstart is one of the easiest e-commerce packages to install, much like the Magento and osCommerce packages. There was one blaring difference about Drupal Commerce: there were no example products! This was unexpected because the front page (shown below) clearly shows a fake product slideshow.

Every other store had some sort of example product, even if it was just clipart garden tools. I ended up having to spend time adding sample items. I used an array of image sizes and qualities to mimic the variety in a typical web store.

Strangely enough, the example store came with categories. So I was sure to “stock” the items into those categories. I was ready to load test after an acceptable amount of products were added.

Testing Phase

At 3 minutes into each of the three tests performed, we consistently see a spike in performance error rates. A majority of those performance errors are request connection timeout errors which means the load engines are not even able to establish a connection with the target server, Drupal.
Amazon Web Services could be blamed for this. The server is then crashing early on at around 350 VUsers. We expect higher scalability for a large Amazon EC2. So it’s like we’re missing a piece of a larger puzzle that needs investigating.

Score

Drupal Commerce gained a score of 55.7 out of 100. This puts Drupal Commerce in the lower end of the of ranks. The score would be better for Drupal Commerce if the WebPageTest repeat view metrics were not missing. We could not re-test due to time constraints, but we are assuming a value of 60 seconds for all the repeat views. This stops Drupal Commerce from unfairly being ranked as the worst in performance.

At this point, we’ve tested six different open source e-commerce platforms.

• Magento Community Edition
• WooCommerce
• osCommerce
• VirtueMart
• OpenCart
• Drupal Commerce

Since they are all free and open source, you can give them each a try on your own as well. A deeper analysis of the experiments will be done to make a final ranking of all tested e-commerce platforms. Come back next time and see how each platforms ranks!

This post is part of a series: First – Previous

The post E-Commerce Benchmark – Drupal Commerce appeared first on LoadStorm.

The post E-Commerce Benchmark – OpenCart appeared first on LoadStorm.

Introduction

OpenCart is yet another free, open source e-commerce solution for the typical user. When you search for “OpenCart” on Google, you will be surprised by what comes up. The first result is under the domain opencart.us, which is not the one we will test because it is proprietary. The correct OpenCart is the second entry: http://www.opencart.com/

If you missed our previous posts, we are scoring six different e-commerce platforms based on web performance. We continue our e-commerce benchmarking series by giving you a glimpse of OpenCart by scoring it against the same standards as our Magento store in the first blog post.

Setup

The installation of OpenCart was more difficult than I anticipated. I ran into issues right at the end of OpenCart setup phase. For a typical PHP site, you get an indication that your installation was successful. I received no indication of a success or failure.
All I was given was the dreaded browser white screen with no PHP errors shown.

After a few more installation attempts and looking through the Apache error logs, I found out what was missing. There was a function in the php-mbstring package that was missing from the server. Once that issue was resolved, I could proceed to checking out the live demo store.

The OpenCart Demo Store

OpenCart’s demo store is easy on the eyes like WooCommerce’s demo store. There is an immediate and modern appeal to the out-of-the-box sample store and theme. Funny, as this also isn’t the first time there’s been a Galaxy S tab in a sample store. I created a customer account to generate the scripts necessary for testing. Afterwards, it was time to load test the store.

Testing Phase

Test Results for all OpenCart load tests (click to expand)

The test selected to score OpenCart turned out to be test #1. It had an average response time of 9.6 seconds and an estimated scalability of 912 concurrent VUsers. WebPageTest results for first load time were dispersed as follows:

• 1.51 seconds at the 0 minute mark
• 21.0 seconds at the 30-minute mark
• 104 seconds at the 55-minute mark

How did these metrics contribute to OpenCart’s score?

OpenCart is currently second to last in performance overall. The two culprits that attribute to OpenCart receiving such a low score were the following:

1. WebPageTest First View loading time average
2. Estimated Scalability (using the scalability criteria)

I was anticipating for OpenCart to score well because of its design approach, but that was not the case. If you are looking for web performance right out of the box, then OpenCart should not be your choice.

The last testable e-commerce platform is coming up. Check in next week for the final web store, Drupal Commerce, and the web performance winner!

This post is part of a series: First – Previous – Next

The post E-Commerce Benchmark – OpenCart appeared first on LoadStorm.