API Performance Testing for Success

As the rate of delivery for applications and services continues to rise, performance testing often takes a backseat in the race to delivering apps on-time. This 2022 holiday season is expected to be bigger than ever, with holiday shopping growing 15.5% to $236 billion. During this make-or-break time of year, companies can’t afford for their apps to become unreliable under heavy load.

Now more than ever, companies need to prioritize the performance of their APIs. Web application performance directly affects online sales. If a website is unable to process an unprecedented number of orders during the Christmas season, revenue will ultimately be lost. Performance, especially the perceived load time, is also key to good user experience. 53% of people would abandon a page that loads for longer than 3 seconds, and one-second lag might cost online stores millions in sales per year. Gartner once estimated the average cost of network downtime at $5,600 per minute. The darkest scenario would be for your website to crash on Black Friday.

We would hate for that to happen…. In this blog, we’ll go over the different types of API performance tests and how to build them using ReadyAPI Performance, so you are prepared for this holiday season.

What is API Performance Testing?

Performance testing is about creating product level load simulations within an application or system that is as near as possible to being a finished product to deploy, or an existing application that’s already being used by customers.

Let’s trace it back to the business case…. Performance testing is about understanding the end user experience under load and ensuring your application can meet its SLAs.

Different Types of Performance Testing

Outside of functional testing, there are five types of performance tests: load, baseline, smoke stress, soak, peak, and spike.

Load Tests

The term Load Testing refers to a standard version of the performance test. You specify the number of VUs, the test script and the time to run the test and all VUs hit your API continuously during the specified time while the testing tool records the performance. Then you can compare the performance metrics against the service-level agreement (SLA).

You can use ReadyAPI for load testing. ReadyAPI uses functional test cases to simulate a massive load on a tested web site or web service. It helps you check how your server works under the load and estimates its limits.

For load simulation, ReadyAPI uses virtual users. Running load tests on your local machine allows using only a few virtual users. The maximum number of virtual users you can simulate simultaneously depends on the license you have. If you need a large number of users to simulate at a time, you can run distributed load tests and distributed cloud tests.

Here is a sample view of a load test in ReadyAPI:

Baseline Testing

Baseline testing is a load strategy used to assess how your server performs against the SLA. Your test should help you determine what response is normal for the server. You can use these values to set up the assertion in other tests. You may also find server or target Test Case problems that would prevent longer load tests from running correctly. This is also a good way to check if your server performs according to the SLA.

To create baseline tests in ReadyAPI, we recommend using the Baseline load test template. By default, it will create a test with the following parameters:

If you simulate the Fixed load profile with the VUs load type, you specify the constant number of virtual users to be simulated. If you simulate the Fixed load profile with the Rate load type, you define the constant speed at which new users arrive to the server.

To estimate the server performance, add assertions and server monitors that will check the response time, the size of transferred data and other values to which your SLA can refer.

Here is what your test results may look like:

Smoke Tests

Smoke tests are a load testing strategy to make sure both your server and test work fine. This is a load test you run to prepare for later load testing. Unlike the Baseline test, you do not intend to hit the service-level agreement (SLA), just make sure that there are no errors.

After confirming that everything works fine, you create other load tests.

To create smoke tests in ReadyAPI, we recommend using the Smoke load test template. By default, it will create a test with the following parameters:

If the assertion fails, the load test stops.

Here is what your test results may look like:

Stress Tests

Stress testing is the simulation of heavy load on the server to find the maximum number of user the server can handle. This number is also called a crash point. The crash point does not necessarily mean that the server crashes or hangs. It can mean that errors start happening or that the server performance or response time is below the level that your service-level agreement (SLA) defines.

To create a stress test in ReadyAPI, use the Ramp Up load profile with a large number of users.

In addition, you need to create assertions and server monitors to check the server response against the limits specified in your SLA. When the responses take longer than the SLA specifies, your server runs out of the processing power, or errors start happening – you have reached the maximum capacity of your server.

You can quickly create a stress load test by using the stress load test template. By default, it will create a test with the following parameters:

Soak Tests

A soak test is a load test that runs with a high but non-critical number of VUs for a long time, from several hours to multiple days. The idea behind Soak Testing is that sometimes a system seems to perform as expected under a particular load and then, suddenly, it stops working without any changes in traffic. Ideally, at the end of a test run, the server performance should be the same as it was at the beginning of the test. The decrease in performance can indicate that the server code has some issues.

To create a soak test using ReadyAPI, we recommend using the Baseline load test template. By default, it will create a test with the following parameters:

As an option, you can configure server monitors. ReadyAPI will create two statistics charts, one for the load test and another for the server monitors.

Alternatively, you can create a load test with similar parameters manually.

With ReadyAPI, you can track various server-side metrics like memory and CPU usage, the number of database requests per second, and so on. To do this, you need to configure server monitors in your load tests.

Peak Tests

Peak testing checks how your server works during the busiest periods. Peak testing is like soak testing, but with a much heavier load and shorter duration. These test cases can be helpful to test the scalability of your API, especially if you run it on serverless or auto-scaling infrastructure.

You can create a peak test in ReadyAPI by using the peak load test template. By default, it will create a test with the following parameters:

As an option, you can configure server monitors. ReadyAPI will create a statistic chart for the load. However, you can create a load test with similar parameters manually.

Spike Tests

Spike testing helps you check how the tested server responds to a sudden spike in the number of users. The server should be able to stabilize and return to normal functioning after the spike passes.

To create a spike test in ReadyAPI, use the Burst load profile. It helps you simulate a short burst of users well above the expected maximum load. After that or in parallel, run another load test to check how the server handled the spike. If you simulate the Burst load profile with the Rate load type, the test will create a large number of virtual users coming to the server. This will simulate the sudden spike that can easily overwhelm the server. If you simulate the Burst load profile with the VUs load type, you can specify the exact top number of users that will work with the server. The test will simulate the specified number of users, even if server responses slow down.

We recommend using the Spike load test template. By default, it will create a test with the following parameters:

Your test may look like this:

For more information on API performance testing and an in-depth overview of different scenarios, check out our on-demand webinar, Are you Ready for the Peak: Performance Testing for Success.