Novell’s site has a great article about load testing describing how The Washington Post Company implemented an identity access management system by Novell.
The article posts this as a primary challenge of their project: How to quickly, reliably and easily conduct system load testing for Novell Access Manager 3.x (Hereafter referred to as “NAM”). You might be surprised at their solution, but I think it is very, very cool.
In the article, Corbin Links states that load testing is not important – it is critical to access management:
Arguably, one of the most crucial tasks of implementing any form of identity access management (IAM) framework or SSO system is the ability to verify that the system can withstand large amounts of concurrent load. Authentication (authN), Authorization (authZ) and Auditing operations though lightweight and inexpensive from a transaction standpoint, can negatively impact system performance when occurring in large quantities over short periods of time. How much or how little load your NAM system can or should handle will be defined by the requirements of your application and any corresponding Service Level Agreements (SLA) you may have in place.
I often mention in our blogs that load testing is too often overlooked or skipped intentionally. Ignorance and apathy are common excuses for a development team bailing out on testing their web application’s performance under heavy traffic. The Washington Post found out there are additional reasons:
- Comprehensive and relevant load testing can be extremely expensive, costing anywhere from thousands of dollars, well into the tens of thousands.
- SSO deployments are extremely time, resource and cost intensive. It is not uncommon to be running behind project schedule and start dropping task items considered to be “lower priority.” These often mis-categorized items include documentation, load testing, and training – just to name three.
- Load testing tools can be very difficult to implement and set up and reuse. Slight changes in the testing platform or the particular version of NAM you are running may require brand new test cases for each scenario.
- Lack of meaningful internal benchmarks or external usage data which can guide system relevant testing requirements. Generally this scenario exists for one or more of the following reasons:
- NAM/SSO is new to the organization. No hard SLA requirements yet exist.
- There is little or no application analytics data to determine usage patterns, peak and valley usage, raw transaction numbers, errors, daily and per-transaction bandwidth statics and many others.
- Future growth patterns are unknown or inadequately planned
For the IAMS project, Corbin outlines their load test requirements – including a basic transaction that is common to all of their test scenarios. There were 4 key infrastructure components in the target environment:
- NAM LAG
- NAM Identity Server
- Windows 2003 Active Directory Authenticator
- PeopleSoft HR Portal
We are proud to acknowledge The Washington Post picked LoadStorm as their load testing tool for this implementation. While my company has no affiliation with Novell, nor expertise with their NAM product, obviously Corbin Links is an expert. Ease and cost-effectiveness were the reasons we were selected. Corbin mentioned several times in our phone conversations how their team was pleasantly shocked at how fast he got the tests executed. He joked about spending more time in the first few meetings discussing what tool should be used than actually just building the test plan in LoadStorm.
Of course that makes me happy, and I’ll not-so-humbly brag about the testimonial. Corbin goes way beyond that though…in his article, he describes how he actually used our tool to do his testing. We appreciate him taking the time and effort to share a LoadStorm tutorial with his Novel Access Management colleagues. There is also a PDF version attached to the article if you want a download.
Many thanks Corbin!