How the NVIDIA Tegra Team Accelerated Code & Document Reviews with Collaborator

Stringent requirements

Scott Taylor, a career verification engineer, works on the NVIDIA product line that produces chips designed for the autonomous vehicles and self-driving car market. “As more and more people use your products, the documentation and the scrutiny on your design quality and code increases,” he points out.

The autonomous vehicle market has a particularly stringent set of requirements for code and documents. ISO 26262 is a safety-related standard specific to the automotive industry, which focuses on reducing possible risks due to a design failure or any kind of unintended behavior. Things like a problem in the specification, a bug in the implementation, a random event like an alpha particle, or any foreseeable operation errors or misuse.

“You need to have evidence of adhering to all of the necessary processes, and be able to audit them,” Scott explains. “If there’s a problem at any time in the lifetime of that car, we have to be able to prove what we did to mitigate it. This information and all of this documentation – which is stored in many different formats – needs to be archived and accessible for decades. Any tool we use must be able to handle this requirement.”

Handling complexity and scale

When your software directly affects the health and safety of the people who use it, compliance is more than just a business goal. NVIDIA uses Collaborator to maintain compliance at scale.

With autonomous vehicles, one major compliance challenge is maintaining documentation for decades. Scott summarizes the additional issues involved. “We have to provide audit logs of who did what, and when they did it: timestamps, who reviewed it, what the process was, and whether that process was followed. There must be clear identification of who is responsible for approvals at any given time. These results can never be tampered with. For instance, you can’t change or delete comments in a review. That audit trail has to be complete and absolute. These are major challenges for a review tool where you’re balancing the rigors of auditing with ease of use.”

Their tool also needs to handle scalability, where documents may be hundreds of pages long. And in any given review, dozens of versions can be uploaded, with multiple milestones and comments from multiple reviewers. The need for inter-document dependencies means that as one document changes, all the other documents that reference it (or are making assumptions on what that document says) also need to be cross-checked. There’s a tremendous amount of complexity in making all of this work.

Performance is also key. With a finite amount of server capacity, and thousands of users using Collaborator at any given time, a slow response can hamper results. This led to engagement between NVIDIA and SmartBear to make sure the entire system could scale to seamlessly handle this level of complexity.

Incredible support

“It’s not just that we log a bug and wait for something to be fixed,” explains Scott. “In order to support the ISO standards, we were able to explain what new features and bug fixes we needed. SmartBear’s ability to resolve those was almost as fast as if NVIDIA had an internal team. There’s a tremendous back and forth going on.”

As use models evolve, usability challenges change over time. The challenges faced a year ago are different from the challenges today. For engineers – the power users of Collaborator – there are always conversations about finding the right usability balance point. One of the keys to making this work has been for NVIDIA to get involved with SmartBear directly. NVIDIA views the relationship they have with the SmartBear team as an actual partnership.

Scott explains how this works in practice. “A couple of years ago, we needed better internal support and external partnership to make all of this flow. We said to SmartBear here are the key pinch points for us to get our first chip ISO-certified. How can we work together to make this happen in the timeframe we need? SmartBear provided incredible support. There were times when we asked for a new feature, and within a week we had a functional version. The interlock was fantastic during this key timeframe. This is a shining example of interactions between a customer and vendor.”

Seeing the benefits

One of the biggest benefits has been that NVIDIA can now confidently meet their ISO compliance. “The external auditors who look at those work products, they’ve had to ask very few questions about that side of things.”

As a result of using Collaborator, which includes over 150,000 reviews in the last ten years, NVIDIA has improved their products without sacrificing quality. Better reviews mean they’re able to meet their milestones with less rework, thus bringing their products to market more quickly.

Key benefits:

• SmartBear support delivered new features quickly, sometimes within a week
• Time to market, reviews, and quality have all improved, thanks to Collaborator
• Over the last five years, the process for documentation and code reviews has been streamlined
• External auditors who monitor ISO compliance have to ask very few questions because they’re provided with the information they need

“Over the last five years, we’ve streamlined the process for both documenting and code reviews. And with all the features and the processes, we now have a great story on meeting our ISO compliance. We expect this this trend to continue.”

Final impressions

Overall, Scott has praise for the product. “We’ve improved our time to market by using Collaborator. We were able to outsource certain things that we didn’t have to do ourselves and, at the same time, get better reviews and better quality. Meeting our milestones for this, the amount of time it takes to do a review, has decreased. There are some pretty complicated database queries to figure out that, but we spot checked a few and said, this same document a year ago took this long to do. Now we’re getting them done two or three days faster.”