Learn More About Collaborator
Just this past May, Fiat Chrysler had to face the tragic ramifications of an error in their software. In rollovers, the side air bags and seat belt pretensioners in a number of their Ram pickups were malfunctioning. This problem only came to light after a suit was filed at the end of 2016, regarding the failed air bag deployment in the rollover crash of a 2014 Ram 1500.  With the recall already underway, the focus turns to the process at Fiat Chrysler and the other automotive companies involved.
Last September, General Motors also had to issue a recall of nearly 4.3 million vehicles because of software defects with their air bags.  They only discovered the issue after a crash killed one person and injured three. Were these crashes that spurred massive recalls the kind of scenarios that should have reasonably been anticipated and tested? Probably, but even if they absolutely were, a disconnect between their code review and testing processes was evident and dangerous.
Of course, functional safety standards exist to attempt to prevent these kinds of accidents. Automotive Original Equipment Manufacturers (OEMs) in the US have to comply with industry standards like ISO 26262 by incorporating quality management features throughout their product development process. Other companies, depending on their location, may also need to include considerations for the similar but distinct standard, Automotive SPICE®.
These standards have quality and risk management expectations for each step in the development lifecycle, from initial product design through software development and production. The initial version of ISO 26262 was published in 2011; and next year, edition 2 of ISO 26262 will be released.  In order to streamline some compliance considerations, it is being reported that edition 2 will group process-related requirements into one section, require a communication channel between functional safety and related disciplines, and expand its scope to cover motorcycles, trucks & buses, autonomous systems, and semiconductors. Because the standards focus so much on risk assessment and quality management, what the part is doesn’t matter as much as how the part is developed. Still, as new technologies hit the market, the how is changing too.
For example, with the Internet of Cars on the horizon, what additional quality management features will the industry need to incorporate in order to address network security concerns? Today, every car is comprised of hundreds of programmable computing elements and millions of lines of code.  If the existing standards can’t prevent software bugs from creating air bag malfunctions, what chance do they have at mitigating risk when human drivers take a literal backseat? Yes, the standards are changing, but maybe not fast enough.
Now, automotive companies will need to step up more when it comes to managing the quality of their software. Even the best developers will create bugs after coding a certain number of lines. With more source code being created in the automotive industry than ever before, the risk has never been higher. If you start with the reasonable assumption that defects are inevitable, then the question really becomes how you can identify and fix them early enough that they don’t reveal themselves in a tragic accident and expensive recall.
Software product quality assurance in the automotive industry has been intertwined with process assurance for more than 40 years. ISO 26262 provides the guidelines for safety assurance in new product planning, from concept through decommissioning. The core pillars of this standard are Automotive Safety Integrity Levels (ASILs) ratings, verification, and validation.
Specific to the automotive industry, ASIL ratings are a means of prioritizing areas of risk mitigation. The risk levels are ranked highest to lowest as ASIL D, ASIL C, ASIL B, ASIL A, and QM (Quality Managed). While risk level is important for determining the overall ASIL Hazard and process impact to your organization, the same outcome of hazard elimination is expected to be consistent across the scale. Before you can start building the Safety Case for your development process, you need to first determine the dependability requirements for your system, based on the following criteria. 
The three ASIL dimensions:
Moving forward, controllability will be an interesting area to watch as assisted and self-driving capabilities become mass market technologies. With no driver, controllability decreases dramatically and increases ASIL risk levels across the board.
Verification is the demonstration that the designed model or code satisfies specifications and requirement, while also not including any unintentional functionality.
In practice, this process is often a combination of reviews, static analyses, and comprehensive functional testing at the model level.
Software testing is one of several verification activities intended to confirm that the software development output meets its input requirements. As Capers Jones points out, "A synergistic combination of formal inspections, static analysis and formal testing can achieve combined defect removal efficiency levels of 99%."  Where tool assisted peer review stands out is in code and document inspections as well as providing a central location for reviewing test cases, plans and the results of static analysis tools.
While some believe static analysis of the code is best done by automated tools, code reviews are actually more effective at finding errors than automated tools. Most forms of testing average only about 30% to 35% in defect removal efficiency levels and seldom top 50%. Formal design and code inspections, on the other hand, can achieve 95% in defect removal efficiency. 
There are even some verification requirements that can only be satisfied by code review. “While analysis may be used to verify that all requirements are traced, only review can determine the correctness of the trace between requirements because human interpretation is required to understand the implications of any given requirement. The implications must be considered not only for the directly traced requirements but also for the untraced but applicable requirements. Human review techniques are better suited to such qualitative judgments than are analyses.” 
In addition to code reviews, document reviews are needed throughout the software development process to ensure that the Software Development Plan is being followed. When choosing a peer review tool, make sure that the solution you select can handle both code and document reviews.
As systems become more capable, it becomes harder to test all the ways they will be used in advance. Once you test software and fix all the problems found, the software will always work under the conditions for which it was tested. The reason there are not more software tragedies is that testers have been able to exercise these systems in most of the ways they will typically be used. But all it takes is one software failure and a subsequent lawsuit to seriously damage a company’s reputation. Test-and-fix approaches are vital dynamic testing approaches. Whether performed on individual units or the entire system, these dynamic approaches share one common shortcoming: they all rely on test cases.
Test case scenarios are constructed from the same source documents that developers use, such as requirements and specification documents. These documents are much more comprehensive at defining what the finished product should do, rather than what it shouldn’t do. Developers inject about 100 defects into every 1,000 lines of the code they write.  Many of these defects will have no impact on the test case scenarios designed for testing. Yet, they could have devastating, unforeseen effects in the future.
ISO 26262 does not go into detail as to how code reviews and evaluations should be performed. While thousands of organizations have successfully implemented and defended peer code reviews successfully, many have failed. The difference most often comes down to poor implementation strategies that can be readily addressed:
One of the most important contributions a company can make to successful adoption of code reviews are the tools it provides its teams. The right tool set will enable each development team to find its own best way to do code reviews, enabling a bottom-up approach to code review design and ensuring fuller achievement of potential gains and regulatory compliance.
It should be noted that this paper has steered away from discussing any particular software development methodology. A peer code review process can be implemented within waterfall, Agile and other methodologies with equal success. The point to focus on is that not only will implementing peer code reviews make the products your company produces better, it will make the processes and the people that produce them better as well. Code reviews are a powerful tool eliminating defects, but achieving compliance can be burdensome. Even in organizations where code reviews have been “adopted,” they are skipped as much as 30% of the time, primarily because they are inadequately supported. 
Too often, organizations believe they can have ad-hoc development processes, and then use an inspection process at the end to remove all defects. It just will not happen. Industry statistics indicate that for every four errors pulled out, one new error is injected. Therefore, only portions of defects are actually removed if the attempt is applied only to the end of the implementation process. To approach zero defects, inspection must be an iterative process. 
For years, it was believed that the value of inspections is in finding and fixing defects. However, in examining code inspection data, it becomes clear that inspections are beneficial for an additional reason. They make the code easier to understand and change. An analysis of data from a recent code inspection experiment shows that 60% of all issues raised in the code inspections are not problems that could have been uncovered by latter phases of testing or field usage because they have little or nothing to do with the visible execution behavior of the software.
Rather they improve the maintainability of the code by making the code conform to coding standards, minimizing redundancies, improving language proficiency, improving safety and portability, and raising the quality of the documentation — benefits which are not possible from automated testing. 
Peer reviews create an environment of shared understanding and collaboration. As developers review and comment on each other’s code, whether in real-time or asynchronously, they all get better. In the end, the code review provides a platform for continuous process improvement, leading to improved standards, better developers, better efficiency, a higher quality finished product, and the peace of mind that comes from knowing the organization can prove compliance.
Collaborator | Robust Peer Reviews, Fast
Collaborator is the most comprehensive peer review solution on the market, designed with highly-regulated industries in mind. When code and design quality is critical and ISO 26262 compliance is on the line, Collaborator enables custom review workflows so your team can conduct code and document reviews specific to your development processes. With comprehensive integrations to data archiving, Collaborator makes peer reviews seamless and fast. Start a free 30-day trial today.
Table of Contents