This is the 22nd blog in our Things You Should Know About Appium blog series and this time we’ll be focusing on how to use image recognition in your Appium tests when testing mobile apps, games and even websites.
The image recognition has been lately one of the most prominent approaches for enabling test automation for mobile apps and games that have graphics content or other content that regular test automation frameworks may not be able to identify with IDs, descriptions or object characters.
Download Our Appium Step-by-Step Tutorial to Properly Set Up Your Appium Testing Environment for Image Recognition
We have recently updated our Image Recognition sample to be more diverse and robust. There are instructions on how to launch the image recognition tests in your local environment, as a Client-Side Appium test run and as a Server-Side Appium test run. The dependencies are now also using the latest version of the Appium-driver, which includes the usage of Appium java-client 3.2.0.
In other words, the underlying libraries are now more up to date and should help you get your tests running more fluidly and with fewer problems from old known bugs!
The README may seem a bit more complicated now with instructions to all these three types of test execution included, so do take your time when looking through it. It also includes information on how to install the Image recognition libraries to your own machine, so make sure to follow it until the end before attempting to run the tests locally.
Running Appium Image Recognition Tests from Command Line
Once you have installed all the dependencies, you’re ready to set up the project for testing! For local tests, remember to set your Appium server ready as well. Copy the
testdroid.properties.sample.properties file into a
testdroid.properties file in the project root directory and uncomment the example lines depending on the type of test you’ll be running:
Once the testdroid.properties are set, use a maven test command with correct test class specified in “-Dtest” flag to launch the tests:
mvn -Dtest=iOSSample test
mvn -Dtest=AndroidSample test
Uncomment the Client-Side example lines of testdroid.properties and also set your Bitbar user credentials:
testdroid.username=<add username here>
testdroid.password=<add password here>
Configure the following parameters based on which platform and device you’ll be using for testing:
testdroid.project=Client side Android
testdroid.device=Samsung Galaxy Nexus GT-I9250 4.2.2
testdroid.project=Client side iOS
testdroid.device=Apple iPhone 5 A1429 9.2.1
Finally, run your tests with maven using the same logic as with local tests:
mvn -Dtest=<TestClass> test
In addition, if OpenCV is not installed using Brew, the
java.library.path needs to be provided on the command line. In order to get this done, execute the following line with OpenCV java library path in place:
mvn -Djava.library.path=<java-lib-path> -Dtest=<TestClass> test
To get any additional properties configured, check out our Appium-Driver’s
To select the device, you can use either the
testdroid.device variable in the properties file or alternatively give the device name via command line parameter: (e.g. testdroid device name = “Apple iPhone 6 A1429 9.2.1”)
mvn -Dtestdroid.device="testdroid device name" -Dtest=<TestClass> test
After the test run is finalized (you need to wait until the device run is finalized) you’ll get the results, screenshots and all the other assets from the test run under
Running Appium Image Recognition Tests on Server-Side
You will need a Bitbar Testing project of type CALABASH ANDROID or CALABASH IOS, which we have pre-configured for you to act as a Server-Side Appium project.
Once you have a properly configured project available, use the provided scripts to create the test zip from your project:
## For ANDROID - createAndroidZip.sh
cp run-tests_android.sh run-tests.sh
zip -r android-test.zip pom.xml run-tests.sh src akaze/linux akaze/LICENSE lib/testdroid-appium-driver-1.2.0.jar queryimages
## For iOS - createiOSZip.sh
cp run-tests_ios.sh run-tests.sh
zip -r ios-test.zip pom.xml run-tests.sh src akaze/mac akaze/LICENSE lib/testdroid-appium-driver-1.2.0.jar queryimages
These lines are included as shell script files and both
run-tests_ios.sh can be found from the root of the project.
- After you have executed these scripts, you should have a properly built test zip file and the application .apk/.ipa file ready for a test run.
- Then create a new test run in your pre-configured Bitbar Testing project, upload your APK or IPA as well as test zip file using the test run creation wizard.
- Choose a device group where your Appium test will be executed (you can include any number of devices for your test run and the test will be executed simultaneously on these devices).
- The last page of the test run creation wizard includes the setting of max timeout for your tests, so make sure it is set high enough for your tests (default is 10 minutes).
Changing the Test Class Name
At any time if you change the name of your Android or iOS test class, you will need to update it to the
run-tests_ios.sh TEST and JUNIT variables as appropriate:
# Name of the desired test suite and optionally specific test case, eg: AndroidSample#mainPageTest
# JUnit file wont have the #caseName ending
Bitbar’s Appium driver is a Java project, which utilizes the Bitbar API client for java and Appium’s java-client. The major benefits of using the driver are the ready-made logic for app upload to Bitbar Testing, taking of Screenshots as well as simplifying the editing of desired capabilities. The testdroid.properties, which is located at the root of the project, is the only file you need to edit when desired capabilities need changing.
Getting Started With Your Own Tests for Image Recognition
When you’re ready to start tweaking the actual tests, you will mostly work on the test files in
Our collection of helper methods in
src/main/java/TestdroidImageRecognitionIos.java should also be a tremendous help when looking for different ways to make use of the image recognition data.