Analyzing Data in Load Testing: What You Really Need to Know
Most load testing tools collect exhaustive information for each load test run. Traditionally (if we can say that about load testing), you needed to have pretty extensive knowledge in order to interpret the majority of load testing data in any kind of meaningful way. This, unfortunately, has resulted in countless load tests that generated tons of useless data.
More often than not, developers and QA managers come away from a trial of load testing software with little more than the number of users that will crash their system. Unless they have a professional tester on staff, most development teams don’t have the time, resources, or knowledge to garner all they could from their load tests.
When I think about all that data just sitting there, unable to fulfill its life purpose of Web application betterment, well… it makes me sad. So! My final blog of this four-part series is dedicated to vindicating neglected load testing data. Luckily, improvements in graphics and user interfaces like those in LoadUIWeb have made interpreting data much easier—if you know what to look for.
Here’s my list of the most important results in load testing and how you should be working with them.
Page Load Time
You absolutely want to know the average page load time for each page in your scenario. You might have a strict Service Level Agreement (SLA) that mandates how quickly pages must load, or may just want to know what this number is. It is also important to know if one page takes longer than others to load—this indicates a bottleneck in your application.
Response Load Time
Just knowing page load time, is not enough. If a page is slow, you need to know why. Being able to look at average response times for each response really gives you a detailed look into where the time is being spent.
Errors and Warning
You need to know what errors and warnings were generated and at what level of load. This is especially important information to see in chart format. It is important to see which errors and warnings are generated and be able to see how that changes as load increases. A common error at high levels of load is “Server Error 500’s.”
Request and Response Throughput
It's important to see the amount of data going to and coming from the tested system. This is especially important in a case where load is increasing, but bandwidth reaches and maintains a plateau. In this case, it becomes apparent that bandwidth is being throttled at some point in the process, possibly at the firewall, and often this can be fixed by changing a setting.
Because so many of today’s websites call out to a plethora of additional hosts for things like content delivery networks, ad servers, analytics servers, social media and syndicated content, it is important for these sites to be enumerated in your reports. It’s equally important to be able to view all of the calls to a particular host. If a host is called from your pages, the response time for those requests will add to the time it takes your pages to render. You must be aware of and possibly take action in the case where a certain call takes a long period of time to respond.
These are the basic data points I feel must be understood in order to gain the most from load testing. I’d be interested to know how you use your load testing data, and if you’ve found any other metrics especially important.