Image Based Automation Tools Don't Make The Cut
Three types of technologies are commonly used for automated test recording. The first and foremost is the technology that records user actions at the object level, essentially meaning that while recording, the technology records object information for the elements presented on the screen. The other two technologies create automated scripts based on the coordinates (e.g. x=55, y=45) or image of the element.
The next part of this article will examine why object-based tools, such as TestComplete, offer high test stability and reduce maintenance costs as well as rework. It will also touch upon how tests created at the object level are much more robust and why these tests do not need to be re-recorded whenever an application under test changes.
Firstly, writing and executing cross-platform tests using an image-based automation testing solution can become challenging in the longer term. For instance, in case of web applications, different browsers render pages depending the screen resolution, rendering engine, and screen size. Images of different UI controls such as text boxes or buttons can vary across various web browsers, thereby causing image based automated tests to break. Image based tests recorded on one browser may also fail when replayed on other browsers given the way 3rd party controls are rendered in different browsers.
The brittle nature of tests recorded using image-based recording automated testing solutions is not just limited to cross browser testing scenarios. These challenges are even experienced when recording tests within the same browser. For example, as a tester, let’s say you want to test if a button with “Check it out” label works as expected when the UI is zoomed in and out.
[caption id="attachment_21140" align="aligncenter" width="500"] Image at 70% Website Zoom[/caption]
[caption id="attachment_21139" align="aligncenter" width="500"] Image at 100% Website Zoom[/caption]
As seen above, image size and resolution varies depending upon the images zoom level. The image of “Check It Out” button looks totally different at 70% zoom than at 100% zoom. In such a scenario, the automated scripts created using an image-based recording automation would fail as the layout of the image used while recording is different than the one it is being run on. Testers would need to record tests for each and every scenario (in this case at every zoom level) they desire to test.
Similarly, with the zoom, the X and Y coordinates for the “Check it out button” button have also changed. Hence, in case a coordinate-based automated recording tool is used, the scripts would also need to be rewritten for the new scenario. As evident in this simple example, it is safe to say that the costs and time required to maintain test scripts created using image and coordinate based solutions can be high.
Object-based recording tools such as TestComplete would work perfectly in the above mentioned situations and a single script can be used to test functionality across browsers and at different screen zoom levels. This is primarily because object mapping property for the “Check it out Button” remains the same irrespective of zoom and the browser test is running on. Developing and maintaining object-based tests can thereby be easy and less costly.
Secondly, image based automation tools necessitate large amount of rework. For instance, if the application under test changes, in case of image or coordinated based automated testing solutions, the tests need to be rerecorded as the change affects the layout and appearance of the image. Take the example of developer changing the image pixels/color of the “Check it Out” button in the above example. In such a scenario, with an image or coordinate based recording tool, the layout and appearance of the image must be recaptured by re-recording the entire scenario from scratch. However, with object-based recording automation tools, no changes to the script are required as the object properties remain the same irrespective of image pixels/color.
Lastly and most importantly, the challenge with image-based recording automation tools becomes even more pronounced with mobile testing. While testing mobile applications, many times, the correctness of the UI is to be dynamically determined as it is dependent upon various factors such as the screen size and resolution. Additionally, validating the correctness of the UI can be challenging, especially when the user interface changes based on the device orientation. The same “Check It Out” button will have different size, resolution, and coordinates based on the way phone is held - landscape or portrait.
The time saved by recording test at object level can be really critical when operating in an agile environment. Object-based recording ensures that existing regression tests don’t fail, which in turn allows testers to spend more time on creating test cases and testing scenarios that have recently been added to the agile sprint.