This post is part of a series: First – Previous
Introduction
Drupal Commerce is the last stop on our tour of e-commerce platforms. Drupal is a free content management system, and Drupal Commerce is an extension that allows you to build a web store with Drupal. This setup is similar to WooCommerce and Virtuemart, which both rely on their own content management systems.
If you missed our previous posts, we are load testing six different e-commerce platforms and scoring them based on web performance. We continue our e-commerce benchmarking series by giving you a glimpse of out-of-the-box Drupal Commerce and scoring it against the same load test standards as our out-of-the-box Magento store in the first blog post.
Setup
To streamline setup, I decided to use a bundle called Commerce Kickstart which contains a single package for both Drupal and Drupal Commerce. So you don’t have to worry about the in-betweens of installing the content management system and extension. How easy is that!
Commerce Kickstart is one of the easiest e-commerce packages to install, much like the Magento and osCommerce packages. There was one blaring difference about Drupal Commerce: there were no example products! This was unexpected because the front page (shown below) clearly shows a fake product slideshow.
Every other store had some sort of example product, even if it was just clipart garden tools. I ended up having to spend time adding sample items. I used an array of image sizes and qualities to mimic the variety in a typical web store.
Strangely enough, the example store came with categories. So I was sure to “stock” the items into those categories. I was ready to load test after an acceptable amount of products were added.
Testing Phase
At 3 minutes into each of the three tests performed, we consistently see a spike in performance error rates. A majority of those performance errors are request connection timeout errors which means the load engines are not even able to establish a connection with the target server, Drupal.
Amazon Web Services could be blamed for this. The server is then crashing early on at around 350 VUsers. We expect higher scalability for a large Amazon EC2. So it’s like we’re missing a piece of a larger puzzle that needs investigating.
Score
Drupal Commerce gained a score of 55.7 out of 100. This puts Drupal Commerce in the lower end of the of ranks. The score would be better for Drupal Commerce if the WebPageTest repeat view metrics were not missing. We could not re-test due to time constraints, but we are assuming a value of 60 seconds for all the repeat views. This stops Drupal Commerce from unfairly being ranked as the worst in performance.
At this point, we’ve tested six different open source e-commerce platforms.
- Magento Community Edition
- WooCommerce
- osCommerce
- VirtueMart
- OpenCart
- Drupal Commerce
Since they are all free and open source, you can give them each a try on your own as well. A deeper analysis of the experiments will be done to make a final ranking of all tested e-commerce platforms. Come back next time and see how each platforms ranks!
This post is part of a series: First – Previous
Curious – there should certainly be demo products in any demo store installation of Commerce Kickstart 2.x. Did you see any relevant messages in your PHP or Drupal logs? For example, if you created a new Commerce Kickstart 2.x demo site on Pantheon, a Drupal Platform-as-a-Service offering, you’d see the store full. (It’s certainly not the only environment in which this would work, just the environment I’ve used the most to train new users where I know there are no system constraints preventing the full installation process from functioning.)
It would be relevant to talk through any configuration you’re doing before load testing these sites. For example, many applications – Drupal included – are configured for development out of the box but offer very simple optimization settings for taking the site live. In Drupal, this involves caching pages for anonymous users and aggregating and compressing CSS / JavaScript to both reduce the processing time and number of HTTP requests required per page load. If you didn’t at least toggle these on, I would expect to see poor results. : )
There are certainly other opportunities to configure a hosting environment for Drupal usage that are documented in a variety of places. I don’t see a documented Apache / PHP / MySQL configuration anywhere – so we’ll assume the testing is good as a relative metric – but it would be relevant to see how the tests perform in a vanilla environment vs. an environment optimized per either best practices or common software-specific platform configurations. How well Drupal Commerce performs relative to other applications on a vanilla environment is interesting, but any developer worth his or her salt wouldn’t deploy Drupal in a vanilla environment. : D
Definitely appreciate your time and insight – feel free to reach out if we can help in the future. : )
It’s a little unfair I didn’t even do minimal optimization techniques like JS/CSS aggregation. This gets complicated if we’re doing it for six different platforms. So we wanted to just keep it simple and try every app out of the box. There’s also no additional LAMP stack configurations aside from what is required to make the app “work”.
We want to simulate what sort of performance an average Joe might expect after they install their web store. They won’t not know about these easy optimization techniques! The first thing on their mind is adding products and raising their conversion rates, not having a highly scalable web store.
That’s where we come in. Like any successful website, there will be growing pains.
Thanks for your comment, Ryan. I still believe Drupal Commerce is a great option for newcomers or veterans to e-commerce.