The Company is one of the largest and longest-standing mutual insurance groups in the Northeast of the United States, covering eight states, primarily on the east coast. To provide Quality Insurance Protection to their customers, they have numerous web applications used by both internal and external customers. They drive quality into each product by failing fast on every change. Their UI functional and end-to-end tests are executed across various supported browsers to ensure customers have a continuous quality experience.
The Company is embracing CI/CD and shift left methodologies. In 2020, they purchased CrossBrowserTesting to accelerate their test automation and use LoadNinja and Zephyr Enterprise for performance testing and test-case management, respectively.
Most recently, they migrated their cross-platform web testing from CrossBrowserTesting to BitBar, the unified mobile device and browser testing cloud from SmartBear.
The DevOps Team at the Company champions automated testing at all levels of the software stack, which is essential as part of a CI/CD roadmap across multiple teams. The Company would like to improve its previously manual processes via automation to advance its rapidly growing IT department and drive quality into every product. The DevOps Team is the thought leader who will make these infrastructure services, tools, and libraries more robust.
As part of their long-term strategy, the plan is to create or update processes, best practices, tools, and libraries that all teams will use as a self-service component. The DevOps Team Scrum Master explains how this idea fits in with the Team’s overall ethos of helping the entire Company. “As we bring these services forward, we’ll teach the rest of our teams how to fish instead of giving them a fish.”
Before 2020, the Company only had three applications with functional and end-to-end automated tests. One of their automation engineers looked into running automation on servers locally. However, when the Company began thinking about the situation a few years later, they knew it wouldn’t be feasible due to maintenance and scalability concerns. They started looking at the cloud. Their fundamental goal was to find a SaaS solution to help all their teams succeed.
That’s why they began using CrossBrowserTesting in 2020. According to DevOps, time was a key factor. “Engineers were wasting time on their PCs while running tests locally. We needed to optimize our time usage and find a solution to run in the cloud. We wanted to shift left, fail fast, and run all our tests within our CI/CD pipelines.”
Big transition – in only one sprint
CrossBrowserTesting enabled them to expand their automation efforts for a couple of years. Still, when the Company was allowed to join the beta program for web app testing on BitBar, they eagerly agreed.
The DevOps Team enjoys the benefit of participating in product testing with SmartBear, “The ability to jump on beta solutions and to see if they fit our needs is great.” Per their Scrum Master, product testing lets them not only learn how to use their tools more efficiently, but it also contributes to making those tools work best for them.
The transition from CrossBrowserTesting to BitBar took only one sprint. One of the Team’s Automation Engineers explains how that happened. “It took two weeks to reconfigure one of our applications with a single person doing it, so it wasn’t hard. There’s been fine-tuning, but the heavy lifting was quick and painless.”
How BitBar helps shift left
Shift left is a significant initiative for this Insurance Carrier. BitBar helps make it a reality. The Team Lead explains, “aside from unit, integration, and other pre-deployment tests, ideally in a CI/CD pipeline, when we deploy to some of our lower environments, we’d like to trigger our automated functional and end-to-end tests and be able to stop the build at that point if there are any issues. Thus, reducing the feedback loop, which in my view, is the whole point of automation.”
For teams that haven’t adopted CI/CD and automated testing that the Team is rolling out, that feedback loop, rather than being relatively small on every commit/deploy, is spread out across months. The gap between development and testing can cause productivity loss, timelines, and scope creep. The Team Lead explains it’s a considerable risk he’d like to eliminate or at least reduce. “Automation has a part to play regarding functional and end-to-end automation test efforts. Having BitBar solves the problem of deciding where to run these tests.”
For the Team to host this kind of environment without BitBar would have serious downsides. For example, currently, they have a project where the automation tests can only run on a single thread on a machine. It’s running on one of their automation engineer’s laptops, which takes four days to run, eating up her bandwidth.
“She didn’t have anywhere else to run it,” says the DevOps Team Lead. “But with the SecureTunnel configuration setup, she can run in BitBar now. Moving to BitBar can help run all the tests concurrently, reducing the time it takes to execute. It’s a huge saving in both time and effort.”
BitBar helps elevate their goals – and confidence
“BitBar expands our automation capabilities,” another senior member says. “Before SmartBear, we used different automation tools to do what we do now, but less effectively and efficiently. CrossBrowserTesting helped us expand with parallel testing in the cloud, and now BitBar does it even better. It’s helped us speed up the process and improve our testing efforts.”
It’s also given different projects the flexibility to work on other tasks while running their testing in the BitBar cloud. But the benefits go beyond the dramatic time savings. The various web applications have new confidence in deployments.
If a bug occurs, they know “it will fail fast and give immediate feedback to their CI/CD integrations.” When any team sees a green build, it gives them “assurance that there are no new issues in terms of functionality and acceptance criteria.”
Another Engineer building out Automation Frameworks and Libraries for the different teams appreciates the extra flexibility BitBar gives her. “While the projects run in the cloud, I can confidently work on other things.”
“Moving to BitBar can help run all the tests concurrently, reducing the time it takes to execute. It’s a huge saving in both time and effort.”
Flexibility in the cloud and other benefits
Using the BitBar cloud has additional benefits. The Director of Information Services says, “This is a great leverage point for us. We’re trying to reduce our installation footprint internally, and with a long-term goal of getting rid of the data center, SmartBear products align with that.”
Other benefits found while using BitBar:
- More efficiency. DevOps knows it can stick to its roadmap because SmartBear products let them fail fast and move on. The overall process speeds up.
- Quick and easy transition. The Company only needed one sprint to migrate to BitBar from CrossBrowserTesting.
- Confidence and flexibility. Engineers can rely on their tools and spend energy rolling out new features and fixing issues quickly when needed.
- Integrated products. When the tools work together in the cloud, the entire Team performs better.
Three SmartBear products work together
In addition to BitBar, the Company uses two other SmartBear products: Zephyr Enterprise and LoadNinja. “The SmartBear products help us deliver quality and make informed go/no-go decisions,” says the Director of Information Services.
“LoadNinja gives us performance metrics and a view of how an application will perform when it goes live. It’s a huge value-add because I know whether our go-live is at risk. It tells me whether we need to concentrate on performance optimization. It makes my decisions easier.”
Getting three products from the same vendor is also a plus. “Our goal is to automate tests as much as possible, so BitBar and LoadNinja are core to that. One being functional tests, the other being performance tests. Even though they’re different products, there’s synergy there. And the Team benefits from that.”