Most developers are familiar with running Selenium
or Appium tests on their local machine (or sending them to a local Appium server). While these capabilities are
essential early on, many organizations need help scaling this approach. Building and maintaining internal device
labs and infrastructure can quickly become a nightmare.
Here, you'll learn about the differences between client-side and cloud-side tests and how to choose the right option
for your project.
What is Client-Side Testing?
Client-side testing is the process of executing your tests locally (often directly on a real device or browser)
rather than on a remote server. For example, a mobile developer working on a fitness app might connect their iPhone
and run an Xcode test
on the device. That way, they receive immediate feedback and can easily diagnose issues by running quick tests.
Some of the benefits include:
Device-level Access – Testers can use specific devices with specific configuration
settings. In addition, they can access in-depth logs and other metrics to diagnose problems.
Quick Set Up – Testers can launch tests from their local development environment without
uploading and scheduling a cloud-side test. This makes it easier to spot-check new features.
Security – Organizations can secure data by keeping test hardware offline or behind a
firewall while retaining full control over PII or other data for maximum compliance.
These benefits might appeal to healthcare companies that collect and compare apples-to-apples data and ensure no
data leaks occur for HIPAA reasons. Or, to cite another example, companies developing internal apps may want to
ensure they work in specific factory settings involving private cellular or Wi-Fi networks.
Despite these benefits, client-side testing comes with a unique set of challenges:
Time and Test Latency – Client-side testing may take longer because it must account for
many client environments. Moreover, tests cannot typically run in parallel.
Slower Agile Cycles – The additional time spent on testing can delay software releases,
which slows down Agile development cycles and can hurt app competitiveness.
Less Test Coverage – Comprehensive test coverage may be difficult or impossible due to
the wide variety of possible configurations and limited access to devices.
Higher Cost – Client-side testing requires purchasing and maintaining a device lab,
consuming upfront and ongoing capital and costly engineering hours.
What is Cloud-Side Testing?
Cloud-side testing is when tests
are executed in the cloud rather than on a local client device or browser. For example, a mobile app developer might
write tests in Appium, upload them to the cloud, and run them in parallel on hundreds of devices simultaneously. As a result,
they can achieve better test coverage in a fraction of the time.
Cloud-side testing conveys several benefits:
Better Coverage – Cloud testing platforms maintain large device libraries, enabling you
to test the same app across a broad range of devices, operating systems, and browsers to ensure compatibility.
Faster Tests – Cloud testing platforms enable you to run hundreds or thousands of tests
in parallel, making your test suite speedy without sacrificing test coverage or quality.
Less Overhead – You don't need to build and maintain a device lab. This saves upfront
capital costs and frees up engineering resources to work on apps rather than test infrastructure.
These benefits may appeal to browser-based gaming or mobile apps with features varying across devices. You can
ensure applications work for everyone by enabling tests across different devices. In addition, cloud-side testing may appeal to
apps with high UX requirements and a large test suite by allowing more comprehensive tests to run faster.
However, cloud-side testing also comes with its own set of challenges:
Complexity – Setting up, configuring, and managing cloud test environments can be complex
and may require specialized knowledge and skills.
Security – Tests may involve sensitive data, and running tests in the cloud requires
extra precautions to ensure data security and comply with data privacy and security regulations.
Compatibility – Cloud testing platforms may provide a broad range of devices, but they
may not be the best fit if you need legacy devices or specific environments.
Cost – Cloud testing platforms often involve subscriptions, and organizations without a
budget may not have access to the level of testing they require to meet their demands.
Networks – Cloud-based tests require good network access, as local machines and the cloud
platforms must transmit test commands and responses to each other.
BitBar Bridges the Gap
Client-side tests are the most common, but cloud-side testing platforms are becoming increasingly popular. Unlike
many cloud-side testing platforms, BitBar offers both client-
and cloud-side execution, enabling you to choose the best option for each project – or a combination of
testing approaches to discover bugs more quickly.
BitBar makes it easy to run your client-side Appium tests in the cloud using a quick cURL or as part of a continuous
integration process. That way, you can run them like you execute tests on your local machine. The only difference is
that the Appium server and the tested device are in the cloud rather than on-premise.
Alternatively, you can upload your existing Appium tests to BitBar Cloud and then, using the API or the cloud UI,
select and start test execution on multiple devices. As a result, you don't need to modify your Appium tests, can
execute tests in parallel on various devices, and
receive an alert when the execution completes.
Several other features address many of the challenges of cloud-side testing:
- Simplicity – Creating automated
tests is extremely simple in BitBar. You can click on "Create Automated Test" through the web interface,
configure a test run, generate boilerplates for test scripts, select desired test devices and environments, run
the tests with a click, and view the results in helpful reports.
Security – The BitBar SecureTunnel enables you to run tests on web, hybrid, and mobile
applications hosted on internal development environments, behind a firewall, or local host. BitBar also offers
both public and private cloud deployment
- Scalability – BitBar enables you to execute tests on hundreds of real devices in
parallel, dramatically reducing the time it takes to conduct tests. You also have access to a massive device lab
with the latest, most popular iOS and Android models without the
cost of building and maintaining an in-house device lab.
The Bottom Line
Client-side tests enable developers to launch tests from their local machines and quickly test new features –
and BitBar makes it easy to run these kinds of tests using cloud devices or browsers. But in many cases, cloud-side
tests are best for developing a robust test suite. You can achieve better test coverage and focus on building new