A Dashboard is Worth a Thousand Tests

  May 24, 2017

Data driven dashboards are providing users access to buried insights and a holistic picture across the testing process.

Development and testing, like any other digital activity, leaves a trail of data and developers and testers don’t realize it.

Every time code is executed in the IDE or every time a test scenario is run, data is generated and collected. Traditionally in software development, this data (aka logs) has only been used for debugging and monitoring. When the software encounters issues, testers scroll through logs to dig deeper into transactions and analyze the cause of failure.

While development and testing logs and execution data is considered a great source of information when things go wrong, it is rarely used to get the holistic picture of the testing or the development process. But data collected during the testing and development process can be a valuable source of information. This data contains:

  • status of the software
  • results: pass or fail
  • timing information i.e. when a piece of code was run
  • details about the identity of the software or module that was executed

If we think about it, all this data can be used to glean extremely useful insights about that state of testing or development. From data fields like status, timeline, and module name we can figure out how one module is performing over time. If the tool you use for testing or development has been cleverly designed, you will be able to see the data in a visualized fashion, allowing you to see trends in your test results over time, compare results in one timeline over another, and drill down into results for a specific module.

The information provided by dashboards can be extremely valuable for a tester or a developer working on a piece of software. A trend of continuous failures over a course of few months may indicate that the development team has consistently failed to fix a specific issue. This, in turn, could be a good trigger for the tester to raise a red flag to the management team.

As developers and testers work on hundreds of software modules at one time, it’s necessary to have an easy way to generate such insights. Without this information, it’s hardly possible for testers to have any inkling of such glaring issues. The traditional way of doing this has been for the developer or tester to go in, download the test result data in excel and run some analytical procedures to find the trends. This is often very cumbersome and un-intuitive – so developers and testers rarely end up doing it. This results in missed opportunities of finding risky and potentially disastrous trends.

Data driven dashboards can solve this problem. Such a dashboard provides users with high level, holistic insights on the test results and presents the data in a way that is easy to interpret, often in the form of graphs and charts. These dashboards provide users with the ability to filter data based on different fields and compare trends over different times. For testers and developers to adopt these dashboards, they must be attractive and interactive.

At SmartBear, we have understood this need for easy, quick and end-end insights and added a new interactive dashboard in latest version of Ready! API that provides users with an end-to-end view across functional, performance and security tests at a glance. Users can drill down into test results, filter information and export key insights as reports to share. The dashboard also offers 'one click' access to creation of functional, security and load tests. The dashboard is available for the popular SoapUI NG Pro, LoadUI NG Pro and ServiceV Pro products.