Aras + TestComplete – Owning the Lifecycle

Great Expectations

Enterprises like Volvo, Microsoft, Airbus, Dow, and Honda have high expectations. And Bill Turner, VP of Quality and Support at Aras Corporation, is the man charged with seeing those needs met. To achieve this, Turner heads up three teams at Aras – software testing, product support, and product release. By his own account, he has people “across 11 time zones – all contributing into one value stream.”

From a management perspective alone, Turner has his work cut out for him. But the difficulty of maintaining such a diversified team pales in comparison to the complexity of shipping and maintaining quality software, especially on an enterprise scale, while meeting ever-increasing test loads and ever-shrinking timelines.

Greater Complexity

Adding to all this is that the Aras software is subscription- based open-source. Turner explains, “We have a self-described data model, which means that our clients can change whatever they like. We want you to go into our software, make your own forms or change the data model. It’s one of the great things about Aras, but it makes a lot of challenges for the testers.” The potential for their subscribers to use Aras software in ways the team hasn’t anticipated inevitably raises questions about maintaining quality. Turner summarizes the key issues, asking, “How do you ensure that you’ve got the right testing in place? How do you ensure that the testing is repeatable and that there are no regressions? Finally, how do you make it Agile?”.

Automated Testing Was the Answer

Aras has been using TestComplete, the SmartBear UI test automation tool, for five years, since needing to rebuild their automation after transitioning their solution’s technology.

At that time, according to Turner, their testing matrix “went from double digits, to triple digits, to quadruple digits in a matter of months,” as a result of testing to ensure multiple browser compatibility.

The team realized they needed to build new frameworks for their testing to meet the increased need. Turner recalls, “I asked the guys to go out and evaluate options.” The brief was simple. “We needed to be up and running and working and making tests. Understanding how long it would take to get on-board was a key thing. The team came back and said, universally, they wanted to use TestComplete.”

The Ease of Getting Up to Speed

“The time it took for someone to learn how to write an automated test in TestComplete was much lower than everything else we were looking at. It wasn’t just that it was good. It was that the entry-level for the technical expertise was lower as well.”

Ease of adoption was a major factor in that choice. Turner explains, citing the example of an engineer on his team who was in the process of switching careers from manual to automated testing.

“With other technologies, there is a deeper need for experience, like C#, but TestComplete was a gateway for her. In a very short time, she has shown that she can write the tests that we need to write.” It was a key realization – for Turner, the reduced learning curve allowed him to involve his entire team in the automation process, not just the testing engineers.

Support for the Long Haul

“We were trying to start something we hadn’t done before and we needed somebody to turn to. The SmartBear support organization was responsive, courteous, and helpful. The cost-to-benefit ratio certainly proved out there.”

Beyond cost and ease of adoption, Turner expected support for his tool of choice. He wanted assurances that if they bought into a solution, “It would be there down the road when we hit a rock wall and needed somebody to talk to.” Based in Boston, “a stone’s throw” from Aras headquarters, SmartBear offered him the expertise he required and the confidence to move forward.

Diverting the Waterfall

When the team first brought TestComplete on, Aras was having difficulty with testing an office connector whose functionality had expanded to support multiple client platforms. According to Turner, maintaining quality on what had begun as a small suite had “actually got to be larger than the testing of our core product.” To reduce that load, the team had two alternatives. Either skip something and introduce risk, or start automating. They chose the latter, and set to work.

“We started with a core-level smoke test, eventually marrying into the Agile process on every build every two to three days. To make sure that software was functional, without regressions.”

By adopting the Agile methodology – and testing incrementally with TestComplete through every change to break the “giant waterfall” into manageable streams – Turner’s team reduced a task that had previously taken “five man-months, and turned it into two weeks of running various automation configurations.”

The Real Benefits of Shifting Left (Emotions Included)

By shifting testing left and integrating it into the development process, Turner was able to accelerate the delivery of new features while maintaining a predictable release cycle. He could now “release new software for our add-ons and applications with the core product every five weeks.” Automation had radically reduced the time it took to implement customer requests and add new functionality.

The wins with TestComplete extended beyond the QA team. Turner points out that “those teams that aren’t my automation specialists can actually develop the tests themselves as part of the delivery.” In particular, Turner was able to push the knowledge out and have his SAFe and Agile teams take on automation within the development process. It empowered non-linear, adaptable test practices where they could have the optimum effect.

The final benefit came in human terms. As Turner puts it, “When you’re in QA, a state of boredom sets in because you run the same thing over and over again. You want to disengage from that to keep people motivated.” With the bulk of the repetitive testing eliminated, Turner says his testers are now able to focus on new user stories and test cases, pushing automated testing at Aras even further along.

Above All, Quality Software Means Satisfied Customers

Asked about his final thoughts on the journey Aras has taken with SmartBear and TestComplete, Turner responds:

“At the end of the day, if I’m delivering quality software and the customer sees the effect, and they’re happy, that means they renew their support subscriptions. You’ve got this direct tie between the results of making quality software and increased revenue for the company.”

Benefits and Results

• Increased Quality and Test Efficiency - In the five years since he first introduced TestComplete into the QA process at Aras, Turner says the company has gone from “0% automation to 88% automation.” Describing the change, he asserts “it makes light-years of difference. Overall, we have developed around 70,000 tests.”
• Reduced Timelines - Turner reports a reduction of testing times that occupied resources for a total of 5 man-months every release to a cycle of two weeks. “It took a little time to get there, but every piece of automation we have has sped up our Agile development process. Right now we’re even down to pull requests.”
• Improved Release Quality and Predictable Release Cycles - According to Turner, the pre-testing that the QA team does with automation “finds the problems before the support team gets to hear about them.” He adds, “When you know the product is working, you don’t have to worry that you’ve pushed off your functional testing to the end.”