This may not work for non-English speakers, but Cambridge University presents the following paragraph:
Olny srmat poelpe can raed this.
I cdnuolt blveiee that I cluod aulaclty uesdnatnrd what I was rdanieg. The phaonmneal pweor of the hmuan mnid, aoccdrnig to a rscheearch at Cmabrigde Uinervtisy, it deosn’t mttaer in what oredr the ltteers in a word are, the olny iprmoatnt tihng is that the first and last ltteer be in the rghit pclae. The rset can be a taotl mses and you can still raed it wouthit a porbelm. This is bcuseae the huamn mnid deos not raed ervey lteter by istlef, but the word as a wlohe. Amzanig huh? Yaeh and I awlyas tghuhot slpeling was ipmorantt! If you can raed this psas it on !!
Could you read it? Impressive! It would at first appear to be unintelligible, but it surprisingly makes sense. While this is an interesting mind trick, what does it teach us about performance testing? Well, it has been my observation that most owners of websites can understand performance metrics intuitively. Their brains can overcome the potential obstacles from their not being experts in our industry. They can figure out the scrambled world of performance engineering without knowing the code (pun intended).
Although the details about every aspect of the web application under test may not be available, with the key measurements for what a user experiences, a savvy business person can fill in the blanks rather easily. This mental connection does not require specialized training or weeks of working with a performance engineering team. They may not be able to translate Throughput in Kilobytes per Second into a meaningful correlation to the 100 megabits per second network device, yet they can tell where the “pipe is clogged” from a graph of bandwidth with very little problem.
Managers Are NOT Into Details
Professional software testers have my respect. It is difficult to be a good tester, and they usually are not recognized for the value they bring to a software development team – under-appreciated, under-paid, and sometimes overlooked. That said, many times professional testers, especially high-end performance engineers, have a tendency to describe the performance test results to executives/managers/site owners with too much complexity. The engineers want to explain every arcane detail of the process. They want the business person to see all the granularity possible, and I don’t believe that is necessary.
Business people can fill in the gaps. They don’t want to become experts on performance testing. Most executives just want to know that the site will withstand the load expected by the marketing department. Online Product Managers may be more interested in the performance metrics and want to see reports showing the failure points measured in Requests per Second (RPS). A site owner probably only wants to know if the number of Concurrent Users is some multiple (e.g. 10x) the normal peak volume in a day. Simplicity is key in communicating with those stakeholders that aren’t directly involved on the load testing team.
Summary Version Please
My recommendation is to keep your measurements easy to understand. Provide graphs with only the most important metrics shown. Don’t try to explain the test scenarios. Present very few reports. If you are going to show them request/response measurements, you should limit the rows of information. For instance, only provide the top 5 or 10 poorly performing pages or resources. It won’t get you a bonus if you try to become a teacher and waste an executive’s time instructing them in how the script simulates every possible user type on your site. They will just think you are boasting, or worse, they will think you are trying to bamboozle them with all your technical jargon. Either way, they won’t like it.
Furthermore, when the topic turns to remediation or application tuning, be careful to start at the 30,000 foot level. Summarize the data. Identify the onerous bottlenecks, but don’t try to get them to understand how the settings in the JVM or IIS/Apache or mySQL correlate to sudden performance failure. They don’t really care that we are running out of database connections – they really only care that you know what to do about it.
Conclusion
The human brain is an amazing entity. It is a fascinatingly powerful creation. It can perceive activity unconsciously that most sensors could never detect. It can interpret paragraphs of words that are all jumbled and flawed. The brains of successful site owners, business executives, product managers, project managers, CTOs, and marketing department leaders are capable of seeing the big picture of performance engineering. They can see what is germane to getting the web application performing well for the required load.
Give them only what is needed and keep it brief. Communicate to them succinctly and with a focus on the primary metrics. Provide summary reports rather than spreadsheets full of raw data. Don’t try to educate them on how brilliantly you crafted the load tests and found the hidden configuration setting that instantly increased RPS by 10,000%. Yeah, it’s a great feeling to do your job well, however only us geeks really care about the way SQL Server slow query logs work.
Executives don’t respond well to it. Let their minds fill in the gaps.