Next-Gen AI-Powered Visual Testing: Catch the UI surprises before your customers do

  October 31, 2022

The slightest change of code can break the functionality of an existing code structure that was previously working just fine.

Your application’s user experience can be ruined if a bug occurs after a single line of code is changed. To avoid errors in production, you must run continuous tests at the end of every modification.

What is visual testing?

Visual testing compares visual changes to ensure only intentional changes to the UI are being pushed to production. Many companies have the misconception that visual testing is not necessary when robust and broad functional tests are being run.

Visual testing catches defects not detected by functional testing. Whereas functional testing validates whether your application works, visual testing validates your application appears as it should. Both are critical aspects of the software development lifecycle.

The UI is usually the first thing end users see and the ideal opportunity to give a good impression. It is essential to ensure your application looks and works as intended before it impacts a user’s experience.

Visual defects not found with functional testing

  1. Pixel to pixel
  2. Alignment
  3. Layout
  4. Rendering
  5. Overlap
  6. Responsive layout
  7. Font

Importance of automated visual testing

Visual testing can either be executed manually or through automated tools.

In manual visual testing, the tester must manually take screenshots to compare them, by eye, with baseline screenshots to detect the differences on various devices, browsers, and environments.

Identifying differences between two similar photos can be tedious. Imagine inspecting several screenshots in your application after every code change in different environments.

Take a quick look at the images below and try to spot all the differences. Your eyes can play tricks on you, making some changes difficult to spot.

Manual visual testing can be time-consuming and impossible for the human eye to test the application pixel by pixel.

visualtest automated testing

Automated visual testing goes through the process of generating, analyzing, and comparing screenshots for you. It allows you to test various screen configurations at once to decrease test-time execution. Since it automatically detects any visual changes, it naturally saves time and is more accurate, for more effective results.

Introducing VisualTest by SmartBear

At SmartBear, quality is a priority. We are always looking for ways to streamline the software development lifecycle. So we are excited to announce the beta launch of our new product, VisualTest.

VisualTest ensures that your applications look and feel as intended. It helps attain developer visibility with your applications during the design and build phase, with next-generation machine learning and artificial intelligence.

Track several types of visual changes while ignoring false positives to focus on what is needed. Find visual differences faster so that you can deliver the best experiences to your customers.

VisualTest allows you to Integrate your existing functional UI test scripts to deliver an integrated QA design to catch bugs in every test run, commit and build.

Simply add a line of code in your existing functional test to detect and notify your team of any visual changes by capturing full-screen, viewport, and component-level screenshots.

visualtest screenshot

VisualTest with BitBar

For developers or testers to ensure their application looks as intended, they must inspect changes at the end of every modification.

With so many test environments to validate, you might have to run thousands of UI tests. With VisualTest and BitBar combined, you have the added benefit of not only meet your visual testing needs, but scaling it as well. . BitBar provides instant access to the latest and most popular real devices to test on what your customers are currently using. This ensures your application will work on every browser, mobile device, and environment.

Getting Started with VisualTest

For both Selenium Java and Python, you can install the wheel from the release page or download the VisualTest library using native Java and Python methods.

To get started, replace both the API_KEY (with your BitBar API Key) and PROJECT_TOKEN (with your VisualTest project token) in the sample scripts below:

This will run a selenium test using a BitBar browser, and take some screenshots to VisualTest

Python

import unittest 

import time 

from selenium import webdriver 

from sbvt.visualtest import VisualTest 

API_KEY = 'BITBAR_API_KEY' 

PROJECT_TOKEN = 'VISUAL_TEST_PROJECT_TOKEN' 

class TestDiscoverHomePage(unittest.TestCase): 

    @classmethod 

    def setUpClass(cls): 

        print('---- Starting Discover Home Page Test ----') 

        import warnings 

        # turn off tracemalloc socket warnings 
        warnings.simplefilter("ignore")

        caps = { 

            'platform': 'Linux', 

            'osVersion': '18.04', 

            'browserName': 'firefox', 

            'version': 'latest', 

            'resolution': '2560x1920', 

            'bitbar_apiKey': API_KEY, 

        } 

        hubUrl = 'https://us-west-desktop-hub.bitbar.com/wd/hub' 

        cls.driver = webdriver.Remote(command_executor=hubUrl, 
        desired_capabilities=caps) 

        print( 
            f'Launching webdriver for {caps["platform"]} {caps["osVersion"]} 
            {caps["browserName"]} {caps["version"]} '
        ) 

        # load the url 

        url = 'https://discover.com' 

        print(f'Opening URL: {url}') 

        cls.driver.get(url) 

        time.sleep(5) 

    @classmethod 

    def tearDownClass(self): 

        print(f'closing webdriver') 

        self.driver.quit() 

    def setUp(self): 

        # create the Visual Test instance 

        settings = { 

            'projectToken': PROJECT_TOKEN, 

        } 

        self.visualTest = VisualTest(self.driver, settings) 

    def testDiscoverHomePage(self): 

        print(f'testDiscoverHomePage method running') 

        # take a fullpage screenshot 

        print(f'Taking fullpage screenshot') 

        screenshot = self.visualTest.capture('Home Page') 

        print(f'Fullpage screenshot result: {screenshot}') 

        # currently stored locally, but will be uploaded to a server 

        # where image regression will process to display visual bugs 


    def testDiscoverProductMenu(self): 

        print(f'testDiscoverProductMenu method running') 


        # open products link and take element screenshot 

        print(f'Finding nav menu element') 


        menuLink = self.driver.find_element_by_css_selector( 
            '#basepage-159a114a2f > div.main-container.mobile-responsive
            > div > div > div > div:nth-child(1) > div > div > div 
            > header > div > div.header-content > div.navbar-wrapper 
            > div > a'
        ) 


        print(f'Clicking nav menu element') 

        menuLink.click() 


        print(f'Finding product menu') 

        menuEl = self.driver.find_element_by_css_selector( 
            '#basepage-159a114a2f > div.main-container.mobile-responsive 
            > div > div > div > div:nth-child(1) > div > div > div > header 
            > div > div.header-content > nav'
        ) 


        print(f'Taking element screenshot') 

        self.visualTest.capture('Product Menu', {'element': menuEl}) 

if __name__ == '__main__': 

    try: 

        unittest.main() 

    except Exception as e: 

        print(f'Error starting test {e}') 
            
        

Java

import models.ScreenshotResponse; 

import models.ScrollMethod; 

import org.openqa.selenium.remote.DesiredCapabilities; 

import org.openqa.selenium.remote.RemoteWebDriver; 

  

import java.io.IOException; 

import java.nio.file.Path; 

import java.nio.file.Paths; 

import java.util.HashMap; 

  

public class Main { 

    public static void main(String[] args) throws IOException { 

    URL hubURL = new URL("https://us-west-desktop-hub.bitbar.com/wd/hub"); 

    DesiredCapabilities capabilities = new DesiredCapabilities(); 

    capabilities.setCapability("platform", "Linux"); 

    capabilities.setCapability("osVersion", "18.04"); 

    capabilities.setCapability("browserName", "firefox"); 

    capabilities.setCapability("version", "104"); 

    capabilities.setCapability("resolution", "2560x1920"); 

    // Setting the api key 

    capabilities.setCapability("bitbar_apiKey", 
    ""); 

    driver = new RemoteWebDriver(hubURL, capabilities); 

     

    driver.get("https://www.smartbear.com"); 

  

    VisualTest visualTest = new VisualTest(driver, 
    Main.buildSettings("smartbear"), Main.buildLimits()); 

    try { 

        ScreenshotResponse result = visualTest.capture(
            String.format("Test__%s__%s__2", "fullpage", "firefox")); 

    } catch (Exception e) { 

        e.printStackTrace(); 

    } 

  

    driver.quit(); 

    } 

    public static HashMap buildLimits() { 

    HashMap limits = new HashMap<>(); 

    limits.put("scrollMethod", ScrollMethod.CSS_TRANSLATE); 

    return limits; 

    } 

  

    public static HashMap 
        buildSettings(String suiteName) { 

  

    HashMap settings = new HashMap<>(); 

    settings.put("testrun", "Testing java"); 

    settings.put("debugImages", true); 

    settings.put("projectToken", ""); 

    Path currentPath = Paths.get(System.getProperty("user.dir")); 

    Path filePath = Paths.get(currentPath.toString(), 
    String.format("src/test/resources/screenshots/%s", suiteName)); 

    settings.put("saveTo", filePath.toString()); 

    return settings; 

    } 

}

It’s in beta – and you can help

Users have little patience for a poor digital experience. And it’s no secret companies need to ensure a consistently high-quality experience. SmartBear has a proven track record of providing products to improve every aspect of the SDLC.

And you can be part of that process. Want to help us ensure this is a tool that can help everyone? Invest your time and knowledge and try the beta version for VisualTest so developers and testers (and you) can be confident their customers get the experience they planned on.

Join the SmartBear VisualTest beta