Why You Should Define Your Own Load Testing Statistics
Test and Monitor | Posted September 10, 2012

The default set of statistics

loadUI comes with roughly 30 statistics per component and about 100 statistics per server monitor. This is unless you're using the soapUI Runner, then there are eight more statistics per soapUI TestStep. Of course, if you run a distributed test, each agent holds its own set of statistics, and when comparing two results the total amount of statistics double. Overall, a typical load test gives you access to roughly 1,000 statistics.

It is not enough

No amount of default statistics will ever be enough. I know this after having many
conversations with users and customers; service level agreements are often not, and should not be, written based on which values a specific load testing application can spit out.

I recently got a question from a user who wanted to make sure that their web service handled 98% of the requests correctly within two seconds. I discussed this with my colleague, Dain, and we realized that this would be possible if the condition component had statistics – actually, that would let the user define any statistic on their own!

Define your own

I decided to add the following statistics to the condition component:

  • True occurrences – The number of times that the condition has passed.
  • False occurrences – The number of times that the condition has failed.
  • True percentage – The percentage that passed.

This makes it able to meet the user's requirements by creating the following scenario:

 soapUI Runner condition component

As you can see, I have configured the condition component to pass all requests that took a maximum of 2,000 milliseconds to complete. I also created an assertion that will fail if the true percentage is lower than 98%. Finally, I can set the entire load test to fail if any assertion failures occur, using project limits.

 Set Run Limits

Go advanced

If you have used the condition component before, you might know that it has an advanced mode. This means that you can now get statistics for, and therefore assert, anything that you can express as a Groovy condition!

Advanced Condition





By submitting this form, you agree to our
Terms of Use and Privacy Policy

Thanks for Subscribing

Keep an eye on your inbox for more great content.

Continue Reading

Add a little SmartBear to your life

Stay on top of your Software game with the latest developer tips, best practices and news, delivered straight to your inbox