To cope with the increasing testing workload, it has become a common practice to rely on a cloud-based Android app testing solution to automate scripted tests on a large scale of real devices for extensive QA. Despite that multiple Android testing frameworks are available for choose, it’s critical to understand the basics and how each framework performs so the selected tool can help you meet your testing needs and ultimately achieve your business goals.
Today we are going to behold 5 most used Android testing frameworks and break down the basics and code examples of each. The way how each framework works determines the speed and efficiency of test execution. Should you be working on iOS apps at the same time, you can also check out the top 5 iOS automated testing frameworks with code examples here.
1. Appium
Appium is a mobile test automation framework (and tool) for native, hybrid and web apps for iOS and Android. It uses JSONWireProtocol internally to interact with iOS and Android apps using Selenium’s WebDriver. It supports Android via uiautomator (API level 16 or higher) and Seledroid (API level lower than 16), iOS via UI Automation, and mobile web as Selenium driver for Android and iOS. Looking for a complete Appium tutorial for mobile app testing?
One of the biggest advantages of Appium is that you can write your Appium scripts in almost any programming language (e.g. Java, Objective-C, JavaScript, PHP, Ruby, Python or C#, etc), freedom from having to select tools, compatibility across the most important platforms (Android and iOS), freedom from having to install and configure devices to test and more. Also if you are familiar with Selenium, then it’s easy for you to use Appium in mobile app testing. They use the same WebDriver and DesiredCapabilities is used in the same way. Configuring an application to run on Appium has a lot of similarities to Selenium.
Appium sample code:
# wait for hello
sleep(3)
textFields = driver.find_elements_by_tag_name('textField')
assertEqual(textFields[0].get_attribute("value"), "Hello")
# click sign-in button
driver.find_elements_by_name('Sign in')[0].click()
# find the text fields again, and enter username and password
textFields = driver.find_elements_by_tag_name('textField')
textFields[0].send_keys("twitter_username")
textFields[1].send_keys("passw0rd")
# click the Login button (the first button in the view)
driver.find_elements_by_tag_name('button')[0].click()
# sleep
sleep(3)
# click the first button with name "Compose"
driver.find_elements_by_name('Compose')[0].click()
# type in the tweet message
driver.find_elements_by_tag_name('textField')[0].send_keys(”#Bitbar is awesome!")
# press the Send button
driver.find_elements_by_name('Send')[0].click()
# exit
driver.quit()
2. Detox
Detox is a JavaScript mobile testing framework that is built into the application and the test execution starts with app launch. This makes test execution really fast and robust as no external additional tools are needed to orchestrate and synchronize during the test execution. The only issue is that Detox is not supported to run on real iOS devices yet. Bitbar supports Detox tests on Android devices, though.
Key characteristics:
- Much more debuggable than other test automation frameworks.
- Integrable into apps written with React Native
- Faster than Appium in test execution
- Easy to pick up with the help of great API documentation
Detox sample code:
describe('HelloDetoxTest', () => {
beforeEach(async () => {
await device.reloadReactNative();
});
it('should have welcome screen', async () => {
await expect(element(by.id('welcome'))).toBeVisible();
});
it('should show hello Rect after tap', async () => {
await element(by.id('hello_react')).tap();
await expect(element(by.text('React!!!'))).toBeVisible();
});
it('should show Detox screen after tap', async () => {
await element(by.id('detox_button')).tap();
await expect(element(by.text('Detox!!!'))).toBeVisible();
});
});
3. Espresso
Espresso is the latest Android test automation framework that got open-sourced by Google, making it available for developers and testers to hammer out their UIs. Espresso has an API that is small, predictable, easy to learn and built on top of the Android instrumentation framework. You can quickly write concise and reliable Android UI tests with it. It is supported on API level 8 (Froyo), 10 (Gingerbread), and 15 (Ice Cream Sandwich) and onwards.
It’s quite reliable, synchronizing with the UI thread and fast because there is no need for any sleeps (tests run on the same millisecond when an app becomes idle). But it does not have support for webviews as well.
Espresso sample code:
public void testEspresso() {
// Check if view with the text 'Hello.' is shown
onView(withText("Hello.")).check(matches(isDisplayed()));
// R class ID identifier for 'Sign in' - and click it
onView(withId(getInstrumentation().getTargetContext().getResources()
.getIdentifier("com.twitter.android:id/sign_in", null, null))).perform(click());
// R class ID identifier for entering username
onView(withId(getInstrumentation().getTargetContext().getResources()
.getIdentifier("com.twitter.android:id/login_username", null, null))).perform((typeText("username")));
// R class ID identifier for entering password
onView(withId(getInstrumentation().getTargetContext().getResources()
.getIdentifier("com.twitter.android:id/login_password", null, null))).perform((typeText("password")));
// R class ID identifier for clicking log in
onView(withId(getInstrumentation().getTargetContext().getResources()
.getIdentifier("com.twitter.android:id/login_login", null, null))).perform(click());
// Activate the text field to compose a tweet
onView(withId(getInstrumentation().getTargetContext().getResources()
.getIdentifier("com.twitter.android:id/menu_compose_tweet", null, null))).perform(click());
// Type the tweet
onView(withId(getInstrumentation().getTargetContext().getResources()
.getIdentifier("com.twitter.android:id/edit", null, null))).perform((typeText(”#Bitbar")));
// Tweeting!
onView(withId(getInstrumentation().getTargetContext().getResources()
.getIdentifier("com.twitter.android:id/composer_post", null, null))).perform(click());
}
Check out the Android Espresso tutorial for more information.
4. Calabash
Calabash is a cross-platform test automation framework for Android and iOS native and hybrid applications. Calabash’s easy-to-understand syntax enables even non-technical people to create and execute automated acceptance tests for apps on both of these mobile platforms. Calabash’s tests are described in Cucumber and then converted to Robotium or Frank in run time. It supports about 80 different natural language commands (controllers), and new controllers can be implemented in Ruby or Java. To learn more, this Calabash 101 ebook will give you an extensive understanding.
Calabash sample code:
Feature: Login feature
Scenario: As a valid user I can log into my app
I wait for text "Hello"
Then I press view with id "Sign in"
Then I enter text "username" into "login_username"
Then I enter text "password" into "login_password"
Then I wait for activity "HomeTabActivity"
Then I press view with id "menu_compose_tweet"
Then I enter text "Bitbar" into field with id "edit"
Then I press view with id "composer_post"
5. UI Automator
While Robotium is a good yet basic framework, UI Automator allows you to do more UI testing against Android apps and games. Google’s test framework allows you to test user interface (UI) of your native Android apps on one or more devices. Another advantage of UI Automator is that it runs JUnit test cases with special privileges, which means test cases can span across different processes. It also provides five different classes for developers to use, including
com.android.uiautomator.core.UiCollection;
com.android.uiautomator.core.UiDevice;
com.android.uiautomator.core.UiObject;
com.android.uiautomator.core.UiScrollable;
com.android.uiautomator.core.UiSelector
Similar to its time of birth, it only works on Android devices with API level 16 or higher. Another downside of UI Automator is that it doesn’t support Webview, with no way to directly access Android objects.
UI Automator sample code:
// Public void for the operation
public void testSignInAndTweet() throws Exception {
// Starting application:
getUiDevice().wakeUp(); // Press Home button to ensure we're on homescreen
getUiDevice().pressHome(); // Select 'Apps' and click button
new UiObject(new UiSelector().description("Apps")).click(); // Select 'Twitter' and click
new UiObject(new UiSelector().text("Twitter")).click(); // Locate and select 'Sign in'
UiSelector signIn = new UiSelector().text("Sign In"); // If button is available, click
UiObject signInButton = new UiObject(signIn);
if (signInButton.exists()) {
signInButton.click(); // Set the username
new UiObject(new
UiSelector().className("android.widget.EditText").instance(0)).setText("username");
new UiObject(new
UiSelector().className("android.widget.EditText").instance(1)).setText("password");
new UiObject(new UiSelector().className("android.widget.Button").
text("Sign In").instance(0)).click(); // Wait Sign in progress window
getUiDevice().waitForWindowUpdate(null, 2000); // Wait for main window
getUiDevice().waitForWindowUpdate(null, 30000);
}
new UiObject(new UiSelector().description("New tweet")).click(); // Typing text for a tweet
new UiObject(new UiSelector().className("android.widget.LinearLayout").instance(8)).
setText("Awesome #Bitbar!"); // Tweeting!
new UiObject(new UiSelector().text("Tweet")).click();
Bonus: Robotium
Undoubted, Robotium was once the most widely used Android testing framework in the early days of the Android world. We’ll have a recap on this framework as well.
Robotium is an open-source library extending JUnit with plenty of useful methods for Android UI testing. It provides powerful and robust automatic black-box test cases for Android apps (native and hybrid) and web testing. With Robotium you can write function, system and acceptance test scenarios, and test applications where the source code is available.
Robotium sample code:
// Public void for the operation
public void testRecorded() throws Exception {
// Wait for the text 'Hello!' to be shown for newbie
if (solo.waitForText("Hello!")) {
// R class ID identifier for 'Sign in' - and click it
solo.clickOnView(solo.findViewById("com.twitter.android.R.id.sign_in"));
// R class ID identifier for entering username
solo.enterText((EditText) solo.findViewById("com.twitter.android.R.id.login_username"),"username");
// R class ID identifier for entering password
solo.enterText((EditText) solo.findViewById("com.twitter.android.R.id.login_password"),"password");
// R class ID identifier for clicking log in
solo.clickOnView(solo.findViewById("com.twitter.android.R.id.login_login"));
// Wait until log in is done
solo.waitForActivity("HomeTabActivity");
}
// Activate the text field to compose a tweet
solo.clickOnView(solo.findViewById("com.twitter.android.R.id.menu_compose_tweet"));
// Type the tweet
solo.enterText((EditText) solo.findViewById("com.twitter.android.R.id.edit"), "Bitbar");
// Tweeting!
solo.clickOnView(solo.findViewById("com.twitter.android.R.id.composer_post"));
}
Wrapping up
Here we have listed the top 5 testing frameworks for your daily Android builds, creation, and correction. Certainly, each of them has its pros and cons. And it depends on your goal and how comfortable you are with each framework. In general, Appium and Calabash are good cross-platform frameworks in testing both your Android and iOS versions at the same time. Espresso is a good choice if you are purely testing Android apps. Therefore, think about your testing need – functional testing, compatibility testing, UI testing, etc. – and pick up the right and best Android testing framework(s).