Announcing Ready! API TestServer, Developer Testing Gone Wild!
We’re so excited to launch a new product today – something that moves our API testing solutions in a totally new direction; away from desktop and manual visual tools towards code and automation driven counterparts. It’s pretty cool – I’ll jump right in with some examples, code, and everything else you love!
But let’s start with an authentic Q&A to get the basics out of the way:
Q: What is it?
A: An API testing engine exposed through a REST API
Q: How do I use it?
A: By POST:ing API tests to it via its REST API and getting the results back once the tests have executed
Q: In what format are API tests submitted to it?
A: By either a simple JSON format which we call “Test Recipes” – or any existing Ready!API / SoapUI project files
Q: Ok… I think I get it – so what kind of API testing can I do in this “Test Recipe” format?
A: Well, you can test any REST API – assert responses in a variety of ways, chain multiple requests and transfer values between them, add scripts and even add a variety of DataSources to drive your tests, like excel files, csv files, etc.
Q: Hm…What if I want to run API tests as part of my JUnit tests? Do I really have to write JSON files?
A: Nope! We’ve actually created a bunch of open-source components to make it easier to integrate the TestServer into existing code and development workflows, including a Java library that you can use in your existing JUnit/TestNG/whatever setup to generate and post recipes through code.
Q: Ok – I get it! I think. Do I have to install this in my network or can I try it online somehow?
A: If you want a full-blown installation you’ll have to download and install it locally – but to try it out now you can simply head over to https://testserver.readapi.io which hosts a free but somewhat limited version of the TestServer – presented via a trusty Swagger-UI. The Java client is eagerly waiting for you on GitHub (and maven central) – with code samples and all.
Q: Ok now leave me alone so I can give this a go!
A: Sure! While you do that I’ll walk our readers through some of the basic steps to get this working – and I’ll circle back to you once I’m done.
Getting Started
Hopefully that Q&A gave you the basic idea; lets submit our first test recipe to the TestServer running at testserver.readyapi.io.
Navigate to the https://testserver.readyapi.io website and scroll down to expand the POST /recipes action:
Scroll down further to the parameters section, click on the example value to the right so it gets pasted into the request body and set the “async” parameter to false
Now scroll down even more and press the “Try it out!” button – the Swagger-UI will POST the Test Recipe in the request body to the server and you’ll eventually see the response in the “Response Body” section.
As indicated by the page – you could just as well have submitted this via cURL or any other HTTP client – using any platform/language of your choice.
Looking at the response you’ll see both the assertion status for the test and the time it took to execute.
Test Recipes
Let’s have a look at the actual JSON Test Recipe that was submitted to the Server:
As you can see – the root object is a list of TestSteps, which can be any of a number of different steps; the one in the example is of type “REST Request” – which further has properties for the actual HTTP request and an “assertions” array containing a list of assertions to perform on the returned response to our request to https://api.swaggerhub.com/apis. If you’re curious about all different types of TestSteps and their corresponding properties (such as assertions) head over to our documentation website for all the details.
If we wanted to expand on this test we could add some more assertions to assert the actual content of the response – and a couple more requests to further validate the functionality of the SwaggerHub API.
Asynchronous test execution
Before we executed the recipe above we set the “asynchronous” property to false – let’s set it back to true and resubmit the test. Now the response is immediate (not that it took a long time before) and as follows:
To see if our test has finished we’ll use this executionID returned in the response as a parameter to the /readyapi/executions/{executionID}/status resource:
Submitting this we get
which indicates the recipe executed ok – as expected.
If you don’t want to poll for the result of a test execution you can provide a webhook as the “callback” parameter to the original executions POST; the specified endpoint will be called with the result JSON once the test has finished executing.
Using the Java Client
If you’re not a fan of hand-coding JSON recipes we’ve got a couple of alternatives for you. Let’s have a quick look at how you would create the above test using the java client library available at https://github.com/SmartBear/ready-api-testserver-client
Create a simple maven project and add the testserver-client dependency:
Create a method that will execute the test – it could be a JUnit test method or any other regular method – I’ll create a JUnit test;
Here you can see the corresponding Java code for assembling the same test recipe that we had crafted “manually” in the Swagger-UI example. The constructed recipe is executed using the executor created in the @BeforeClass initializer – and the result is asserted using standard assertion code. Please not that this entire unit-test is available on GitHub together with maven project and all.
Running this as part of our general unit-test execution will build the recipe and POST it to the server in the same way we did manually via Swagger-UI. Using the integrated JUnit support of IntelliJ we get the following result:
The Java client contains convenience methods/builders for all the different TestSteps and assertions, integrating it with your existing JUnit tests should be as straight forward as shown above – and since its open-source please don’t hesitate to suggest or even contribute your own improvements directly on the GitHub repository.
TestServer Authentication
The TestServer requires you to specify user credentials using basic authentication (yes – you can secure it with SSL…). The demo Swagger-UI has been configured to do so automatically with a demo account (demoUser/demoPassword) – but if you download and install the server locally you will have to configure an initial user yourself (as described at https://readyapi.smartbear.com/testserver/install/configuring.
Public server limitations
The public TestServer is strictly for demo purposes and does not support the DataSource and Scripting TestSteps – so if you want to give those a try you’ll have to download and install the TestServer in your local environment. Also please note that it is not multi-tenant in any way; any tests you run with it can be seen by others also.
Other tools for creating and running test recipes
As indicated earlier, we’re determined to provide you with a smorgasbord (I’m Swedish you know) of tools and libraries to help you integrate the TestServer into your existing technology stack and development/testing workflow. For now we have:
- A maven plugin for running test recipes and existing SoapUI/ReadyAPI projects during a maven build, which will also generate surefire reports that will show up nicely in your test-result dashboard.
- A basic nodejs CLI to easily execute tests in a node build/dev environment
- A test-recipe from Swagger generator which can help you bootstrap the creation of test recipes for your REST API if you’re using Swagger as a descriptor format
We’re also working on a .NET client in line with the Java client shown above, and the Swagger definition for the TestServer is available on SwaggerHub and can as such be used to generate clients in any of the 20+ languages that is supported by the Swagger-CodeGen.
Special Thanks
We released TestServer a while ago to a select group of customers who were excited to get going with this early, but we wouldn’t have been able to do any of this without an amazing team of API testing experts. They would normally shrug off the praise, but it’s important to give credit where credit is due.
Final questions
Ok – now that I hope I’ve gotten the basic idea across – let’s finish the authentic Q&A.
Q: So – how are things going?
A: Pretty good – I’ve managed to run a few recipes against my own API and also found the maven-plugin on GitHub which allowed me to run existing SoapUI tests together with my new “hand-coded” tests.
Q: Fantastic – looks like you’re getting the hang of it! Do you have any questions for me before I head off to write another pretentious blog-post?
A: hm… nothing immediate – I guess my thoughts are more around how this fits into an agile devops approach to building APIs
Q: Yeah, how could we use this with a Cucumber/BDD approach to API testing?
A: Let’s save that for future blog-posts – I need to draw some charts and stuff to explain all that!
Related Links: