Introduction to API Testing
Gone are the days when testing was just one step in the Software Development Lifecycle (SDLC), nowadays testing has become an ongoing activity that runs throughout the entire SDLC. Tests are now being written right at the design phase of the SDLC (BDD & TDD) and executed whenever there is a change to code, which happens quiet a lot. Tests when written and executed early, highlight fundamental design issues and reduce the risk of finding these critical flaws later, when they can be very expensive to fix. Tests written early not only eliminate design flaws, but they also tie quality to activities at every stage of the software value chain.
APIs enable the digital world. Increasingly in the connected world, more and more APIs are being used for all variety of applications and devices. Whether you are working on a website, ordering grocery on your phone, paying for it from your smartwatch: you are consuming APIs. APIs are enabling organizations to expose their internal data and operations to the outer world and provide enhanced capabilities to their own products and teams. APIs are enabling businesses to grow, and earn by monetizing services and applications enabled by APIs.
Need for API Testing
APIs are the connectors that enable services and applications for end users and businesses. APIs that do not work correctly can cause chaos and suboptimal experience for end users. Recall all the times you have felt frustrated waiting for an app to load the home page or a picture, in such cases, it does not take long for a user to give up and switch to another competitive app. Similarly, poorly designed and poorly implemented APIs will cause software applications to frequently fail. Not to mention the security vulnerabilities, small things like wrongly formatted JSON have the potential to cause hacks, data leaks and erosion of the company’s goodwill.
APIs should define a robust contract on functionality, performance and security and they need to be tested thoroughly for these. API testing is different from UI testing as it requires understanding of the underlying API contract, data formats, transmission mechanism and protocols. To manage this immense complexity, it’s recommended to factor out data formats and schemas into an API definition document that defines the APIs and acts as a single source of truth throughout the lifecycle of the API. The OpenAPI initiative has defined the OpenAPI Specification (OAS) to do exactly this. API testing is a good candidate for testing right from the design phase, especially for APIs defined in OAS. The Initial API design (definition) can be imported into an easy to use API testing tool and automated for repeated testing, every time the design is updated or code is changed.
Start Testing Your API with a SoapUI Pro Free Trial
The world’s most trusted SOAP and REST API Testing Tool for Over 10 Years
Need for Speed in API Testing
With increased pressures on organizations to deliver faster, teams are increasingly adopting Agile practices, and delivering software in iterations spanning a short period of time. Multiple iterations require repeated testing, this often calls for the teams to explore automatic test creation, execution and reporting. With automatic test creation, teams can create tests from design artifacts, like API definitions. When the APIs get updated, the team can bring in the change with very little effort. Automatic test execution involves plugging in tests into automation tools like Jenkins. These automation tools provide scheduling services and triggers for events like code commit. Test execution just gets plugged into this automation pipeline. Thus, teams can build advanced functional, performance and automation tests and subsequently automate them. Automated testing tools also enable easy generation of reports from the test execution data.
Automation increases the speed of delivery; more tests can be executed in short period of time and whenever the software is updated, it can be regression tested with ease. Automated tests can also be run on multiple environments, hence ensuring that the code works fine irrespective of the underlying scenario. Automated creation of tests from API definitions also makes it easy for new practitioners to get started with API testing and scale up their testing efforts with minimal effort.
The command to do that will look something like this:
DevOps is increasingly coming into focus for teams which are moving towards adopting Microservices. Microservices are small independently deployable web-services that encapsulate functionality in a scalable architecture, hence this means that any microservice implementation tend to have a large number of services that need to be developed and tested. Automation tools make this easier to accomplish as most of the work around building, testing and deploying can now be accomplished without much human intervention. This thread of automation runs from design and development phase to the deployment and monitoring phase. Tools like Jenkins can trigger automation jobs, these jobs can start the checkout of the code, trigger build processes, deploy these builds on containers, these containers can then be tested multiple times on different test environments, finally if everything looks good with the build, the container can be deployed on the production environment. Adoption of DevOps is a huge undertaking that requires immense investment in infrastructure and skillset. However, many of SmartBear clients have succeeded in these bigger endeavors by implementing smaller chunks of automation first. They typically start with automation of their build process, then their testing process and finally they can connect the pieces together, creating a comprehensive and valuable automation pipeline.
API Test Automation Tools
Test automation is achieved by integrating created tests with automation tools. For APIs, advanced tests can be created on a tool like SoapUI, or defined in Gherkin for execution by BDD frameworks. These tests can then by plugged into automation tools like Jenkins to be executed or triggered automatically. These automated tests run along or after the code is checked out and build is created. In case of BDD these tests will first be run even before the code is written, subsequently they will be run during different stages of the SDLC.
There are many test automation tools available in the market, the most popular amongst them are Jenkins, Atlassian Bamboo, TFS, Chef and AWS CodePipeline. These applications can be used for continuous testing, integration and delivery. These applications enable you to schedule or trigger various tasks automatically, these tasks may include development, testing, deployment or monitoring tasks. Majority of these tools integrate with code versioning systems, testing systems and deployment systems, which can be plugged into the automation tool easily.
Automating API Tests
API tests are the best candidates for test automation as API design can be controlled through a central definition and used throughout the software lifecycle and the automation pipeline. This is true for SOAP APIs as they are defined by WSDL definitions, and REST APIs which are defined as OpenAPI definitions. Once you have an API definition, you can easily create tests and automate the execution by plugging in the tests into automation tools like Jenkins.
You API tests are ripe for automation once you find that you must execute these tests multiple times within a short period of time. Depending on the time to market for you software, this short period of time could vary from a few hours to a few days. Executing the same test cases manually, is a waste of time. Automated test execution is as good as manual execution and is compatible with test environments and reporting.
It’s important to keep in mind that automated tests do not work in a silo. For effective automated testing there needs to be robust integrations between tests and the appropriate environments (dev, test, production etc.). The automated tests should be dynamic enough that you can switch execution environments fast. The test execution needs to funnel results data in actionable reports. These reports also need to be insightful and portable. There is no point on having reports that are just data dumps that require specialized processing before they can be used for any purpose.
What's Next in Test Automation?
The future is hard to predict, but there are many existing trends that strongly indicative of the future for test automation. First more and more tests will be created automatically from definitions, this will be a positive externality from the proliferation of the API definitions and their use as a single source of truth. Cloud test automation tools like AWS CodePipeline will become increasingly prominent and will be used heavily for automation, continuous testing and DevOps. With advancements like machine learning and Artificial intelligence, focus will shift more towards training models and neural networks by testers. However, these advancements will still be focused on getting more (and better) testing done in less time, which in turn will serve to accelerate test automation.