Earlier this year, Gartner released the Critical Capabilities for Software Test Automation Report. This report was created to help application and QA leaders, who are modernizing software development, evaluate test automation tools across common testing cases. The movement towards a digital business means there is a growing need to optimize the delivery of apps to be a continuous process. Additionally, with the growth of DevOps and Agile development, there is a need for a more fully automated delivery pipeline that includes automated functional testing, as noted in the report.
As with any automation tool, the hope is to create a set of effective tests to be maintained with minimal effort and human intervention. The costs to be considered are the costs of creating and maintaining automation compared to the cost and time of performing tests manually. This report looks to rank different software test automation vendors based on 14 critical capabilities and five use cases such as responsive design, mobile applications, packaged applications, etc.
Recently there has been a large increase in the use of mobile applications as many companies view it as an important channel to have. The Mobile Applications use case evaluates the additional challenges involved in testing mobile apps such as the different versions of mobile operating systems, differing form factors, screen sizes, security settings, network connectivity challenges, and alternative means of inputs such as gestures, voice, and other sensors. Gartner evaluated products based on their ability to cross-platform tests with platform or device-specific feature-sets, and their ability to handle differing User Interfaces of Android, iOS, and Windows.
Gartner evaluated 9 vendors’ products for the Mobile Application use case, and SmartBear was included in the report. SmartBear’s TestComplete Mobile product is an automated testing tool for native, web-based, or hybrid mobile apps that includes testing on real devices, emulators, device cloud, and virtual machines. TestComplete supports iOS, Android, Windows Mobile, and Blackberry platforms using image and object recognition.
SmartBear achieved the second highest score with a score of 3.66 out of 5 for the Mobile Application use case. SmartBear also achieved the highest score in the Responsive Design use case.
The UI testing products of SmartBear evaluated as a part of this report included TestComplete, TestLeft, and Cross BrowserTesting. As always, SmartBear will look to continue adding new capabilities across our automation tools to make it an easier transition for QA and development teams to deliver quality applications quickly while reducing the cost of the DevOps journey.
Gartner, Critical Capabilities for Software Test Automation, 10 January 2017
Gartner does not endorse any vendor, product or service depicted in its research publications, and does not advise technology users to select only those vendors with the highest ratings or other designation. Gartner research publications consist of the opinions of Gartner’s research organization and should not be construed as statements of fact. Gartner disclaims all warranties, expressed or implied, with respect to this research, including any warranties of merchantability or fitness for a particular purpose