The Secret Sauce to Cooking Up High Quality Applications
A couple of months ago, one of SmartBear’s technical experts spoke at the StarEast conference about bridging the gap between GUI and API Testing. The presentation was focused around two key questions:
- How can QA do a better job helping development?
- How can development do a better job helping QA?
The presentation touched on these two questions while also providing some other key information about testing best practices. This included the idea that many people should contribute to both the development and testing efforts such as developers, automation engineers, and testers, and why it is necessary to test across all the environments your customers use for both web and mobile applications.
The presentation began by acknowledging that there are two main types of applications: customer-facing applications and internal applications. These applications often need to be compatible with multiple browsers at different resolutions, but you will find that internal applications allow more of a leeway to guidelines, such as “you can only use chrome for this application.” However, when providing an application that will interface with a customer you need to test on many more environments to ensure your customer does not run into problems due to an environment error. Automated testing allows you to rapidly test different user interfaces and APIs on a variety of environments to ensure your application is running at the highest quality.
Most testers are familiar with the three types of software development: Waterfall, Agile/CI, and DevOps/CD. While there are many who argue one or more software development styles are better to use than others, it is important to know when certain types of software development styles are appropriate and when they are not. For example, it does not make sense to use Agile/DevOps development in medical device firmware, aerospace/defense software, financial software, theme park ride safety, or any other mission critical software. But it does make sense to use Agile/DevOps development in ecommerce, social media, games, a theme park app, and other consumer software.
When posing the question, how can QA teams do a better job helping development, we find there are a couple of ways this question can be answered. First and foremost, it is important for QA teams to provide fast feedback to the developers. No one is great at multitasking, and if a developer has developed an application for you to test, they give you the application for feedback, and you get it back to them a couple of weeks later, they have to go back to update the app and switch their mindset from the project they’re currently working on. Switching between different projects and stories can be a large waste of time. In short, the faster you give them feedback, the easier it is for them to fix the problem. Meaning, developers can fix more bugs in a release cycle or fit more stuff into enhancements or stories that are coming out. By providing fast feedback, you ensure developers are using their time most efficiently. It Is important to get fast detailed feedback back to developers as quickly as possible because at the end of the day there is always a need to maximize the use of scarce resources.
The next topic discussed was why is it important to complete both GUI and API testing. Today we find that there is an increase in the number of APIs involved in applications. These APIs may be internal to a company or exposed to your customer. It is important to test both the GUI and the API because there could be problems with either. API testing can catch a lot of problems early on, but you still have to complete GUI testing since that is what the customer is interacting with. Running both GUI and API tests means there are many tests that need to be completed. With a large number of tests needing to be completed, it is nice to have an automated testing platform to complete a large volume of tests. While it is nearly impossible to completely automate all of your testing, most of the time manual testing will still need to be completed; however, if you can complete even 80% of your tests through automated testing, you will greatly reduce the amount of time spent on testing that could be used towards other tasks. This reinforces the importance of have reliable automated processes and tools to complete a large number of tests and ensure the highest quality of output.
Now, another component of testing that needs to be considered are the various environments that tests need to be run in. In some cases, users come from all over the world using different browsers, with older versions, on a variety of operating systems and resolutions. Because of this, it is important to have testing completed on many different environments to ensure that users are getting the same experience on all potential environments that they could be using. When looking at possible environments, the factors that should be considered are the browsers, versions back, operating systems, and resolutions. When you multiply all these factors together, you can have thousands of combinations. While this can be overwhelming, it is important to prioritize what environments you think your user will be using and possibly some extras as a safety blanket. Having a tool that can integrate with an automated testing platform to test on multiple environments can be a worthy investment for a company to ensure that no environments lead to an unwanted expectation.
After discussing the possibilities for QA to help development, i.e. through fast feedback and pinpointing of issues, the next question that was discussed was how can developers help QA? The first tip was to make automation easy. For developers, it is easy to add hooks for quality engineers to pick up on. Hooks or tags can be easier for testing automation and are also an easy thing for developers to add to code, as long as they are made aware of it. Building the need for hooks into the requirements can save a lot of time for the QA team during the testing phase. Additionally, developers may find they need to create custom controls. In this case creating a playbook on what the custom controls are will provide insight to the QA team as QA teams are not as familiar with these features. When developers are working to build an app, they have a set of standard controls they can work with. Sometimes developers have to put in a custom control, for example, combining two features. When this happens, it is tough for the automation tool to recognize the custom control as most automation testing tools have rules for following standard controls. If developers are able to provide script or sample pieces to use for these custom controls, they can discuss with the QA team some methods available for approaching the custom controls that can be used in testing later.
All in all, the presentation at StarEast taught us the importance of aligning teams, processes, and tooling through the entire development and testing process. We covered some tips-and-tricks for where certain types of software development styles are appropriate, why GUI and API testing should both be completed, and the need for testing on multiple environments. If you are interested in learning about testing on multiple browsers check out Environment Manager, a feature of TestComplete that allows you to test on thousands of real devices, on-demand and in the cloud.