Pros and Cons of Using XCTest for iOS Testing
There are actually very few test automation frameworks that are tightly coupled with the development tool itself. XCTest framework is one of those frameworks that enables its users to write basic unit, performance and some level of UI tests for iOS apps. And as always, frameworks that couple tightly with their development tool and the environment has some pros and cons that users should be aware of. In addition to the pros and cons of XCTest, we’ll take a look at some basic things about the XCTest framework in this blog.
Download This Free XCTest Guide to Learn Pros and Cons of XCTest in iOS App Testing and More
The thing with frameworks (like XCTest framework) that integrate tightly with the development environment is that those frameworks typically provide easy-to-use and seamless workflow with other components and features of the development environment. Also, there have been lots of changes in the iOS app test automation landscape lately due to Apple’s decision of deprecating UI Automation from Xcode.
Now, if you haven’t used XCTest before and you started out with it, there are a few things worth mentioning before diving in. Let’s start with a brief introduction.
A Brief Introduction to XCTest Framework
Like Android Studio, Xcode provides feature-rich software testing capabilities for developers that can significantly help to enhance the stability of the software. Furthermore, Xcode provides XCTest and XCUITest that are extremely helpful for achieving and building better quality software. Thoroughly tested apps, and regardless if done in unit level or UI level, improve the user experience and accelerate the adoption of those sort of apps.
Frankly, XCTest is not a new framework but it has evolved quite well with Xcode releases. In fact, the XCTest framework was introduced with Xcode 5 couple years ago. What XCTest basically does is to allow its users to do unit testing for Xcode projects (iOS apps among them) as it is currently considered as one of the top options for iOS app testing. And writing any tests with XCTest is a trivial task to iOS developers because XCTest is fully compatible with both Objective-C and Swift. In addition, all test classes developers create in Xcode project are basically subclasses of XCTestCase.
XCTest tests can be executed in simulators or real physical devices. If you use real devices locally, you just need to make sure all provisioning side of things (provisioning profile) is valid and set properly to test target. The test methods used in XCTest are instance methods so no parameters are passed nor they return a value. For this the name also begins with ‘test’ and all added tests are visible in Test Navigator of your Xcode project.
If you are interested to compare the top iOS test automation frameworks for app testing we have provided some comparison of those, with code examples and also some best practices here at Bitbar blog.
Pros and Cons of XCTest Framework
Many of today’s test automation frameworks are based on frameworks and layers that development tools provide with them. In the case of iOS, this is XCTest and with Android, most of the frameworks are based on JUnit and Android instrumentation.
The advantages of XCTest over the other frameworks for iOS app testing are as follows:
Easy to Learn and No Additional Components Required. Xcode provides everything to get started with test automation with XCTest. Xcode’s XCTest is an integral part of the tool so testing is fairly easy to start and convenient to work with.
Native iOS Language Support. Basically writing tests with the same programming language that your application is built with is not a requirement, but it gives some confidence for developers to create tests for their apps. Another way around, some developers may also think this differently and prefer other additional languages, frameworks and tools to be used in test creation. Nevertheless, there is no learning curve or language barriers to get started with XCTest test creation.
Xcode Test Recorder. This is mostly a feature for UI tests but as XCTest is closely related with XCUITest the UI recording is possible with Xcode environment. The UI testing capabilities with Xcode include UI recording, generating code out of those recordings, run tests identically as those were intended while UI test was recorded. Record-and-playback testing tools have plenty of benefits and can get testers to do the job even without understanding the underlying software. In addition, recording tests again is possible or simply editing the generated piece of code with whatever changes have been done for the app itself. In short, record and playback tools provide a fast way to create tests, a decent level of accuracy, and not being too sensitive for user-input errors.
Integrating with Continuous Integration. Integrating XCTest with continuous integration is also easy. Xcode allows XCTest tests to be executed using command-line scripts/shell and seamlessly integrated with Xcode’s continuous integration Bots. However, there is a way to integrated XCTest scripts and the development environment with more widely used CI systems, such as Jenkins, but there are lots of things you need to know and sometimes – despite XCTest and Xcode have provided this capability for some time – tests end up failing just for no reason.
Faster than ‘On-Top-of-It’ Frameworks. Many other frameworks are relying on XCTest. And despite that those frameworks provide more features, higher abstraction (or at least way to think so) level for testing and better capabilities, they still rely on the foundation of XCTest. This naturally makes XCTest faster as you don’t need to rely on abstraction APIs, albeit in many cases lightweight.
XCTest isn’t perfect but it provides some excellent basic functionality and capabilities to exercise tests on your iOS apps. As XCTest has improved by each version of Xcode, there are few cons you should consider:
No Cross-Platform Support. Apply typically builds everything to be available only on their own tools, devices and environments. As this is understandable in case of integral testing API and functionalities, many testers prefer cross-platform test automation frameworks and have that one test script to work on both platforms, Android and iOS. Then a great option – and also XCTest based framework on iOS – is Appium. On Android, Appium utilizes Selendroid and uiautomator as its foundation and converts those commands written in Selenium Webdriver to native commands.
Limited Programming Language Support. As this is clearly one of iOS app development’s advantages, it is also one of its cons. Basically app developers and tests created for those apps are limited to build with Objective-C or Swift. Again, the abstraction framework on top of XCTest will expand the programming language selection and provide testers freedom of choice with their tools and programming languages. Appium is a great example here again as it provides support for all major and several not-so-mainstream programming languages as well.
Flaky. XCTest isn’t that new but it still has some issues with the basic usability and the robustness of creating tests for iOS apps. Doing some the basic unit tests and running on those in simulators may work just fine but when you try to run the same tests on devices, you’ll see that there is some room for stability improvements with XCTest and Xcode in general.
Conclusion
XCTest is a valid choice for basic unit level testing of iOS apps and functionality. However, you need to add lots of those sort of basic tests to thoroughly cover the logic, and that makes functional UI testing more suitable for iOS apps.
What are your experiences with XCTest and do you feel it will be the number one choice for app developers? Weigh in with a comment in the section below!