Companies today deploy new releases constantly, some even daily. This constant deployment means regular updates are occurring or existing lines of code are being altered, which can cause problems. The slightest change of code can break the functionality of an existing code structure causing the user experience to suffer.
Maintaining a visually perfect UI is important for retaining customers. In fact, a Stanford Study shows that 94% of people say web design impacts their first impression of a site and 75% of people will base a business’s credibility on visual elements.
That’s why successful companies include visual tests in their suite.
What is Visual Testing?
Visual testing, a method of software testing, evaluates the visual fidelity of your user interface (UI) or graphical user interface (GUI). Visual tests tackle the look of your application and help you capture defects that other types of tests might miss. These other UI tests include:
- Functional tests
- Performance Tests
Many believe that functional tests are enough to catch visual bugs – but that is not true. Functional tests validate that your application works but do a poor job catching visual defects. At the end of the day, both are essential to ensure customer satisfaction.
Limitations of Functional Tests
Functional tests validate that your UI works. For example, if someone were to click “pay now”, you’d want the link to correctly charge the customer’s credit card. But what happens when you want to ensure the “pay now” button is properly aligned with invoice information? It would take considerable effort to validate that with a functional test.
Here are some examples of what a functional test might miss:
- Pixel-by-pixel differences
- Alignment shifts
- Page layout
- Rendering problems
- Element overlap
- Responsive layout
- Font differences
- Color differences
So, what happens if you try and validate these with a functional test? To properly capture these changes over releases, you might begin to chain lengthy assertions to each other in the hopes of capturing visual differences. This leaves you with flaky scripts that break easily and is hard to maintain over releases.
Fortunately, automated visual testing can solve these problems.
Why is Visual Testing Important?
Visual testing captures defects that other types of UI tests cannot. It is essential for ensuring a positive user experience with your application. There are many methodologies for conducting a visual test that range from fully manual to fully automated.
Some of these can be:
- Manual
- Pixel-by-pixel
- DOM-based
- AI-Driven
Teams can choose to follow a single methodology or incorporate multiple types into their workflows. But as release frequency increases, it’s important to find ways to build in more automation. AI-driven automation has especially become a major driver of running efficient and comprehensive tests.
Why You Should Use AI In Your Automated Tests
A recent World Quality Report by Capgemini suggests the key to delivering greater customer satisfaction is to help SDETS achieve more flexibility and agility in their workflows. Naturally, automation is the first step to achieving this, but there are multiple ways to employ automation.
Take the previously listed visual testing methodologies for example. Automation can be used in pixel-by-pixel tests, DOM-based tests, and AI-driven tests. As you run these tests, though, you’ll find that pixel-by-pixel comparisons leave you with too many false positives. This leads to extra work, and DOM-based tests miss crucial elements of your UI.
Only AI-tests are advanced enough to review all elements of your GUI while ignoring false positives to save time in your day-to-day.
AI-Driven Automation with VisualTest
At SmartBear, our goal is to help teams ship the highest quality releases. That is why we built VisualTest, to make it easy for QA teams to add AI-driven automated visual testing to their existing selenium and cypress scripts.
By adding a single code snippet to your script, you can use the VisualTest machine learning algorithm to scan your entire webpage for visual defects and highlight all relevant changes. This allows for you to efficiently identify bugs before your customers catch them.
VisualTest has an easy-to-use dashboard, surfacing only relevant changes so you can focus on what matters, and not waste time reviewing false positives. With a series of customization tools, you can:
- Pick and choose which diffs to review
- Change your views
- Add comments for better team collaboration
- Ignore regions
VisualTest is also able to address the more difficult challenges of effectively capturing lazy loaded content without needing to write any additional script.
And once you have run your regression tests, you can also quickly test those views across real browsers and devices using BitBar.
VisualTest with BitBar
VisualTest and BitBar enable teams to run the most comprehensive automated UI tests to achieve the best end-to-end results. Under a single platform, testers can run automated functional and visual tests, and check them across hundreds of real browsers and devices, ensuring you deliver the best user experience no matter how your customers find you.
How to Get Started?
Check out these examples for both Selenium Java and Python.
You can install the SDK from the release page or download the VisualTest library using native Java and Python methods. To get started, replace both the API_KEY (with your BitBar API Key) and PROJECT_TOKEN (with your VisualTest project token) in the sample scripts below:
This will run a selenium test using a BitBar browser, and take some screenshots to VisualTest.
Python
import unittest
import time
from selenium import webdriver
from sbvt.visualtest import VisualTest
API_KEY = 'BITBAR_API_KEY'
PROJECT_TOKEN = 'VISUAL_TEST_PROJECT_TOKEN'
class TestDiscoverHomePage(unittest.TestCase):
@classmethod
def setUpClass(cls):
print('---- Starting Discover Home Page Test ----')
import warnings
# turn off tracemalloc socket warnings
warnings.simplefilter("ignore")
caps = {
'platform': 'Linux',
'osVersion': '18.04',
'browserName': 'firefox',
'version': 'latest',
'resolution': '2560x1920',
'bitbar_apiKey': API_KEY,
}
hubUrl = 'https://us-west-desktop-hub.bitbar.com/wd/hub'
cls.driver = webdriver.Remote(command_executor=hubUrl,
desired_capabilities=caps)
print(
f'Launching webdriver for {caps["platform"]} {caps["osVersion"]}{caps["browserName"]} {caps["version"]}'
)
# load the url
url = 'https://discover.com'
print(f'Opening URL: {url}')
cls.driver.get(url)
time.sleep(5)
@classmethod
def tearDownClass(self):
print(f'closing webdriver')
self.driver.quit()
def setUp(self):
# create the Visual Test instance
settings = {
'projectToken': PROJECT_TOKEN,
}
self.visualTest = VisualTest(self.driver, settings)
def testDiscoverHomePage(self):
print(f'testDiscoverHomePage method running')
# take a fullpage screenshot
print(f'Taking fullpage screenshot')
screenshot = self.visualTest.capture('Home Page')
print(f'Fullpage screenshot result: {screenshot}')
# currently stored locally, but will be uploaded to a server
# where image regression will process to display visual bugs
def testDiscoverProductMenu(self):
print(f'testDiscoverProductMenu method running')
# open products link and take element screenshot
print(f'Finding nav menu element')
menuLink = self.driver.find_element_by_css_selector(
'''#basepage-159a114a2f > div.main-container.mobile-responsive
> div > div > div > div:nth-child(1) > div > div > div
> header > div > div.header-content > div.navbar-wrapper
> div > a'''
)
print(f'Clicking nav menu element')
menuLink.click()
print(f'Finding product menu')
menuEl = self.driver.find_element_by_css_selector(
'''#basepage-159a114a2f > div.main-container.mobile-responsive
> div > div > div > div:nth-child(1) > div > div > div > header
> div > div.header-content > nav'''
)
print(f'Taking element screenshot')
self.visualTest.capture('Product Menu', {'element': menuEl})
if __name__ == '__main__':
try:
unittest.main()
except Exception as e:
print(f'Error starting test {e}')
Java
import models.ScreenshotResponse;
import models.ScrollMethod;
import org.openqa.selenium.remote.DesiredCapabilities;
import org.openqa.selenium.remote.RemoteWebDriver;
import java.io.IOException;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.util.HashMap;
public class Main {
public static void main(String[] args) throws IOException {
URL hubURL = new URL("https://us-west-desktop-hub.bitbar.com/wd/hub");
DesiredCapabilities capabilities = new DesiredCapabilities();
capabilities.setCapability("platform", "Linux");
capabilities.setCapability("osVersion", "18.04");
capabilities.setCapability("browserName", "firefox");
capabilities.setCapability("version", "104");
capabilities.setCapability("resolution", "2560x1920");
// Setting the api key
capabilities.setCapability("bitbar_apiKey","");
driver = new RemoteWebDriver(hubURL, capabilities);
driver.get("https://www.smartbear.com");
VisualTest visualTest = new VisualTest(driver,
Main.buildSettings("smartbear"), Main.buildLimits());
try {
ScreenshotResponse result = visualTest.capture(
String.format("Test__%s__%s__2", "fullpage", "firefox"));
} catch (Exception e) {
e.printStackTrace();
}
driver.quit();
}
public static HashMap buildLimits() {
HashMap limits = new HashMap<>();
limits.put("scrollMethod", ScrollMethod.CSS_TRANSLATE);
return limits;
}
public static HashMap buildSettings(String suiteName) {
HashMap settings = new HashMap<>();
settings.put("testrun", "Testing java");
settings.put("debugImages", true);
settings.put("projectToken", "");
Path currentPath = Paths.get(System.getProperty("user.dir"));
Path filePath = Paths.get(currentPath.toString(),
String.format("src/test/resources/screenshots/%s", suiteName));
settings.put("saveTo", filePath.toString());
return settings;
}
}
Try VisualTest for Free Today!
Users have little patience for a poor digital experience. The only way to keep up with increasing release frequency is to incorporate more automation into your QA workflow.
VisualTest makes it easy for test engineers to do just that. With a single line of code, you can add AI-driven automation to your selenium and cypress scripts, and ensure your UI always looks as you and your customers expect it to.
Try VisualTest for free (no credit card necessary) and see how you can level-up your UI testing today!