Understanding Continuous Integration Testing

As software development practices increasingly trend towards Agile development and DevOps, Continuous Integration and Delivery have become the go-to methods for receiving fast feedback, meeting changing requirements, and optimizing quality.

But what exactly is CI/CD and how does testing fit in? As more organization adopt these practices, some will increase testing, while some will let it fall through the cracks. Find out how teams are including continuous integration testing to balance speed and quality more successfully.

What is Continuous Integration?

Continuous Integration is a practice where a team of developers intermittently merges their code changes into a shared repository. Developers are integrating multiple times throughout the day and ideally committing to changes about once a day or a few times a week.

Continuous Integration is preferred by software teams because it allows daily constant feedback in development from all contributors, while it’s easier to catch and resolve bugs early on in the process.

In turn, this ideally leads to Continuous Delivery — since builds are constantly managed and shipped regularly, new features can be delivered to the end user on a daily or weekly basis instead of a quarterly or yearly one.

Additionally, by integrating small, frequent changes, it’s easier to catch issues early in the SDLC. Teams often find that this CI/CD results in higher quality software with fewer bugs, and when bugs are found, they’re usually much easier to fix.

Where CI/CD and Agile Intersect

Continuous Integration piggy-backs onto Agile development since Agile is often achieved through the continuous integration of features, as it was created so developers could solve issues as they come instead of trying to predict and solve every change up front and only test right before the product launch.

While teams following an Agile methodology strive to support more change throughout the development process, Continuous Integration is a way that developers to do this by making changes multiple times throughout the day or week. Additionally, sprint planning allows for the constant feedback from both customers and developers as well as other team members.

However, though Continuous Integration often works best as part of Agile or Extreme Programming, it can also be leveraged in other environments such as Waterfall or RUP.

The Need for Continuous Integration Testing

Another element of CI and CD that is less commonly acknowledged is Continuous Testing, or the practice of testing for every integration. You may even consider continuous testing a prerequisite for Continuous Integration and Delivery.

It makes sense that Continuous Testing goes hand-in-hand with Continuous Integration to make sure that bugs are found early and are easy to fix. While many changes are made on a daily basis, there’s an opportunity each time for those changes to disrupt a previously working part of the code.

While teams implement Continuous Integration and Delivery to gain a competitive edge, it’s important to acknowledge that ensuring quality is equally important in the eyes of the end user — no one wants to use a new feature that has a bug. Continuous Integration testing is a critical step for organizations that want to differentiate from their competition.

By automating tests to match the speed of Continuous Integration, rapid delivery can be more effectively achieved and acceptable standards of quality can be met simultaneously.

Open Source CI Tools

Utilizing a Continuous Integration strategy is often complemented with the use of one or more open-source tools.

• Jenkins – Jenkins is the by far the most popular open source continuous integration tool. It’s a Java-based automation server that has hundreds of CI/CD plugins to support building, deploying, and automation. Basically, Jenkins works to check for changes on the server, trigger a build when it detects one to deploy it for testing, and notify test teams of found results. It has a robust, supportive community, easy installation, and numerous integrations that make it a leading Continuous Integration server. Learn about installing and configuring Jenkins here.
• Buildbot – Buildbot is primarily used to manage and schedule different jobs in the Continuous Integration process. It supports job execution across multiple platforms and monitors source-code repositories while reporting the results. Buildbot can only be configured through a Python script, though it does support all major operating systems.
• TeamCity – TeamCity allows you to develop in Java, .NET, or for mobile platforms, while giving you additional functionalities and features such as gated commits, build grids, and integrations as well as more than 100 plugin options, while the Open API allows you to create your own plugins. Read about installing TeamCity.

For Continuous Integration testing, open source tools like Selenium and Appium are most popular for automating tests. Additionally, tools like CrossBrowserTesting can also be used to execute test automation and create an environment for continuous testing in the cloud.

Additionally, while automation is essential for keeping up with the speed of a well-oiled CI process, it won’t always be the answer to testing. For brand new features, it’s best to resort to exploratory testing. For other instances where teams should trade in automation for live testing, check out our blog about deciding which tests to automate.

Conclusion

As more QA and DevOps professionals begin to implement Continuous Integration and Delivery, Continuous Testing will have to be quick to follow in order to better meet customer demands, keep a competitive edge, and secure a standard of quality.

As technology advances and web testing trends change, being able to adapt to change and follow faster development cycles will become the differentiator for many software teams. Continuous testing is how those teams will meet demands of agile practices, frequent integrations, and rapid delivery cycles.