Sign up in 30 seconds.
No credit card.
No risk.
No download.
Load Testing Feedback Contest
We at LoadStorm are excited to announce that we now have 2,000 customers!
In order to celebrate this milestone, we are hosting a contest for the best LoadStorm feedback. WOULD YOU LIKE TO WIN $250?
We are eager to hear from all of our customers so that we can make LoadStorm a better fit for everyone's load testing needs. Please submit your best LoadStorm recommendations, frustrations, wish list, or any type of feedback that you believe will help our developers improve our product.
The best ideas will win gift cards to Best Buy and the Apple Store.
1st Prize: $250 BestBuy.com gift card
2nd Prize: $100 Apple Store gift card
3rd Prize: $50 Apple Store gift card
You can buy a new netbook for $250. There are more than 50 models of digital cameras for under $200 at Best Buy. The prices on iPods are coming down. Spend the money on whatever geek gadget you desire - all we ask is for your thoughtful input, complaints, or suggestions regarding our load testing tool.
Our team will evaluate all entries and the winners will be chosen at the end of October. Entries will be evaluated on several factors including:
- The value to LoadStorm of the feedback provided in the entry
- Clarity of the feedback
- Innovativeness of the suggestion
- Depth of understanding of an issue
- Overall impact on the quality of the product
To submit your feedback and enter the contest, make sure you have at least a free Breeze account to try the tool, then enter your input as a comment to this blog post by October 15, 2009. We will be reviewing the comments throughout the process.
Please read all contest terms and conditions before entering. Your saved comment to this blog is your indication that you accept the terms and conditions of the contest.
We look forward to hearing your suggestions and ideas on how to make LoadStorm a better tool!
Bookmark/Search this post with:
Comments
1. A non branded reporting
1. A non branded reporting link that can be sent to customers.
A few things that will be
A few things that will be helpful:
1. It will be great if Loadstrom can give the Server and Network response time of a page.
2. The measurement of Windows resources like CPU usages and memory usage.
3. Does Loadstrom support Flas and Flex application? It will be a good to have feature.
Hello, 1. I would like to
Hello,
1. I would like to have the option to duplicate a plan/step etc', I guess it is quite easy to implement and it will be very helpful for users. It will shorten the time it takes to build a new test.
2. It will be very nice to have the option to set several test to run (at the same time) by checking a checkbox and not having to go into every build and adding a run for it.
That's all for now folks ;-)
Barak.
Use/integrate Selenium IDE to
Use/integrate Selenium IDE to create the scenarios and allow the simple ingestion of Selenium test scripts into LoadStorm.
For users Selenium IDE provides:
For LoadStorm the Selenium IDE provides:
Using Selenium scripts for scenario planning makes the step to providing real browser testing and functional testing as further offerings by LoadStorm. The Selenium scripts can be combined with Selenium-RC and Selenium-Grid.
It would be a good feature to
It would be a good feature to upload some custom data for any fields that we want to use for entry into our forms. We do not really have a particular list of fields that we need because our forms allow for users to create their own custom fields, which we would like to test and fill.
However what would be useful is to have a larger number of default fields (say 10 or more) because then we could still fill these with our data, as we do not need the names of the fields to match so much as we need to be able to fill data in.
We would also like to mention that the ability to delete and/or archive old load tests would be very useful as well.
.
the peak response time really
the peak response time really scares our customer as it is super high comparing to the ART. Do you have a statistics like the ApacheBench, which tells me how many percentage of pages are loaded in that peak time? I highlighted the sample at the bottom for your reference.
[nwang@server1 conf]$ ab -k -c 2000 -n 10000 http://127.0.0.1/index.html
This is ApacheBench, Version 2.0.40-dev <$Revision: 1.146 $> apache-2.0
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Copyright 2006 The Apache Software Foundation, http://www.apache.org/
Benchmarking 10.32.62.115 (be patient)
Completed 1000 requests
Completed 2000 requests
Completed 3000 requests
Completed 4000 requests
Completed 5000 requests
Completed 6000 requests
Completed 7000 requests
Completed 8000 requests
Completed 9000 requests
Finished 10000 requests
Server Software: Apache-Coyote/1.1
Server Hostname: 127.0.0.1
Server Port: 80
Document Path: /index.html
Document Length: 35056 bytes
Concurrency Level: 2000
Time taken for tests: 37.933800 seconds
Complete requests: 10000
Failed requests: 0
Write errors: 0
Keep-Alive requests: 0
Total transferred: 353310000 bytes
HTML transferred: 350560000 bytes
Requests per second: 263.62 [#/sec] (mean)
Time per request: 7586.760 [ms] (mean)
Time per request: 3.793 [ms] (mean, across all concurrent requests)
Transfer rate: 9095.56 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 43 215.5 0 8999
Processing: 126 7146 5288.8 6964 34984
Waiting: 124 7146 5288.7 6964 34983
Total: 133 7190 5334.1 6966 35223
Percentage of the requests served within a certain time (ms)
50% 6966
66% 7621
75% 8188
80% 8398
90% 9183
95% 13908
98% 33272
99% 33669
100% 35223 (longest request)
I would like LoadStorm to be
I would like LoadStorm to be able to set and retrieve variables from pages.
Supporting open source testing tools file formats (Apache Flood, Apache JMeter...) would also be nice.
There is a MAJOR BENEFIT to
There is a MAJOR BENEFIT to your product that is missing from your Home page: testing for broken links.
Yes, setting up a test with random clicks does not guarantee coverage. The test could certainly miss broken links.
On the other hand:
- It only takes a few seconds to set up the test, so 1/2 hour of free banging on the visitor's site by 50 users, with clear reports on the links that failed to respond is very valuable.
Your site should have an option to email the report to the user, with an option to attach a CSV export of the results.
- You could consider implementing a feature in the test that simply parses ALL the links on a page, then calls them one at a time.
There are options here for recursion, uniqueness checking, etc. I'll leave those to you.
**** Rather than hiding this feature, you could make a feature on your web site that allows a user to PUSH ONE BUTTON and have your site collect a set of links, then exhaustively follow them all.
The user gets performance info AND finds broken links.
The user can take advantage of code coverage tools on the site code itself to determine code coverage, which is not the same as link coverage, but is also beneficial.
really been enjoying your
really been enjoying your service, a few suggestions:
- add "description" as a column on the Analyze tab, this helps us keep track of previously run load tests.
- allow for custom HTTP headers (useful for mocking security)
- allow for steps to be "inserted" between steps
- better AJAX support
- Allow for sending variable user data as Querystring parameters (currently it is only available for form posts)
- Allow for more fields in user data (custom fields named Field1, Field2, Field[N] would be fine) currently it is limited to username, password, city, zip... it would be awesome if we could put custom data in there so we can populate user input to form posts
- being able to post a selenium test recording as a scenario http://seleniumhq.org/
- providing a CSV download or allowing for column sort on the details screen for show pages by elapsed time, show pages by error code and show pages by response time in the Analyze section
ok, back to load testing... thanks
Your "build a load test" web
Your "build a load test" web UI works well, but is pretty slow and it's tedious to build a scenario with several steps. It would be nice to have the ability to upload a load test script (CSV or other format) to help add several steps to a load test scenario at one time.
A potential future enhancement could be to upload a web server log file so that it could be "played back" through LoadStorm.
The best of this type I've
The best of this type I've ever seen.
A couple of comments. 1) Did
A couple of comments.
1) Did the system stoppage condition change from Sep 16 - Sep 17? Because prior to Sep 17 the loadstorm system seemed to stop within 2 minutes of Error Rate > 50%. Any test run on or after Sep 17 seemed to keep running and didn't stop even with 100% error rate for a sustained 5+ minutes.
It would be great if we supply stoppage error condition
IE. Error Rate > 50% for 2 sustained minutes.
2) As Adam mentioned it would be nice if we could download the raw data of the results as CSV or some other format. The flash charting is nice for summary but if we want to compare different versions would be nice to have access to the raw data. Time vs. (Users / Request per Sec / Throughput / Response Rate / Error Rate). Also it would be nice if we could define sampling interval (with the understanding there is a absolute minimum - maybe 15 sec)
3) Although we can build virtual user file with sample data and we can pass it into form variables we can't pass it into QueryString variables. It would be real useful to be able to do that.
Also would be nice to be able to use dynamic URLs. We have web sites where we pass back db id values in the query string.
AJ
To compete with others the
To compete with others the following things should be taken care of
1. Load Storm should support all types of applications - Web, Web2.0, Oracle aps etc.
2. Licensing policy should be flexible
3. Not just tool , all the assignments right from tool to testing
Thanks,
Govind
Final word. LoadStorm is the
Final word.
LoadStorm is the future of load and performance testing. The cloud-based architecture makes it highly scalable and extremely low cost.
Could you add some basic
Could you add some basic statistics to the "Analyze" tab?
For example:
average response time
max response time
for the whole test run?
If they're already there and I'm not seeing them, I apologize in advance.
I'm looking for a quick way to compare two versions of a site.
Also, export to CSV would be nice, but less important than summary stats.
Thanks!
Adam Leffert
Web Developer
http://www.leffert.com
Overall we are very happy
Overall we are very happy with the system and the flash charting is a great tool for monitoring the jobs while they are running. A couple of things we would like to see changed are as follows:
1) The fifteen minute wait period can be annoying, especially when we are runnning tests during off hours and you know that it is unlikely anyone else is running tests.
(a) It would be great to see a dashboard widget that showed the number of
jobs in process across customers. This at least gives customers a feel
for the queue and confirms that we are competing for resources.
(b) It would also be great if no other jobs were running that you go straight
into staging mode. And if there are other users running jobs have the
timeframe vary based on the number of jobs.
2) Once again, the charting is great, but it would be helpful to get more detailed information on each http error. It is easy to go lookup what each error is, but it would be even easier if we could simply click on the error number and get a description of the error.
3) I like the payment teirs for LoadStorm, but the reality is that there are months when customers are between release cycles and aren't going to use the system, so I wish there was a usage based element to the model.
As far as you're looking for
As far as you're looking for paying customers, you need to help as much as possible your potential customers get money to pay you. Alone, your customer can't get ahead. With you, they can. You have 2000 potential traffic generators on your side now to help each of your potential customers.
Let say, you find some customer tools performing very good under your tests. You can detect them, write a best practice for this customer (customer's architecture, Web servers, load balancers and so on...). This customer should be happy to be cited in a best practice from you. You can forward the best practice to the others 1999 (with the news letter ?). These others can go to the cited site, experience a new service and become engaged users of this service. One happy user will talk to his friends and the virtuous circle develops for your customer with you at the center. Your customer will likely get money to pay you for more testing.
Basically, you helped your customers and yourself with a very small investment...
Good luck guys. Xavier
The main thing holding back
The main thing holding back my client from using this tool as a load tester is the lack of Selenium and/or javascript support. For example, we have an Ajax page that is heavy on the js, but LoadStorm cannot capture the clickable image. Clicking on the image has load implications for our app, and as of right now, we can't use LoadStorm to test this vital functionality.
Being able to read and check
Being able to read and check dynamic content that has been inserted on the web page using javascript would be a real bonus!
Bullet pointed to increase
Bullet pointed to increase the size of your smile :)
* Tests that are run from a vast range of IP addresses.
* Tests that can keep track of cookies sent by applications (make it optional).
* Payment package for 2|5|10 tests as well as monthly.
Some of us might only use it once but don't want the low limit on users.
Test monkey ;-) Thanks for
Test monkey ;-)
Thanks for the feedback. Why would you need a vast range of IP addresses? I can certainly understand needing multiple IP addresses, and for larger tests we often have 5-20 or more. The only way I can envision a vast range would be with either a vast infrastructure (not going to happen) or a vast partnership of clients. My early plans for the system actually leveraged users' machines voluntarily as test machines. Think of SETI@home for testing, using the spare cycles and bandwith of client machines. Seems more practical than searching for aliens.
We are currently tracking cookies within a scenario, which can maintain login state or session info like a browser. We were thinking of showing these when building a scenario, since they are invisible to users. Did you have something else in mind?
Roger
Something like 50 IP
Something like 50 IP addresses would be decent enough. The behaviour of some applications is very much dependent on the number of unique IPs using the system. The requests wouldn't nessecarily have to be from different physical servers.
It would also be nice to configure the request headers sent by your system. Specifically the cache related headers. It should be possible to have different headers on a certain % of requests. An application I am developing currently uses a server based cache but that cache is cleared if a client requests the page with no-cache or max-age=X in the request headers. As such if your system sent say 2% of its requests aimed at clearing my cache.. it would make the test a lot more realistic. Of course this 2% should be configurable.
Hi, I was impressed by the
Hi,
I was impressed by the concept of LoadStorm. After trying it out myself as a developer I then bought it to the attention of my project manager, who tried it too. We'll probably be using it more intensively in the coming year as our projects mature.
The instant feedback and load type makes it very useful for testing conditions that previously we wouldn't be able to test at all. We've always looked at load testing tools, and LoadStorm is the best we've seen.
I do think that the form entry interface could be improved. For example, one of our projects relies on the browser returning the x and y coordinates on an image submit button click - we had to make changes to allow for LoadStorm not filling in this information.
Great work anyway, I hope to see it evolve over time as it already has since I first tried it.
Cheers,
Jeremy