I wanted to take a moment to introduce myself as I've just begun blogging here at SmartBear. I am excited about the opportunity to discuss with you and help solve challenges related to the world of software quality, testing, and also share opportunities for technical and business innovation. You may follow and connect with me on Twitter or on Linkedin.
Even after years of increased awareness about the importance of testing early and quality control, many organizations still consider testing to be an expense rather than a vital and critical element of the software development process. Some of the deeply entrenched misconceptions include a view of automated testing as a replacement for human testers. On the other end of the extreme, is a view of testing as a tedious, boring activity of validation that takes place at the very end of the process leading to delays in production.
Testing is a core business activity that starts early, continues throughout the project, generates valuable business metrics and ultimately leads an organization to success. Are you extracting the full potential out of your test process? Are you testing smart?
Although best practices show that automated testing doesn't necessarily find all the bugs (less than 40% to be precise), it saves time and money by finding defects early. To measure and test for quality - what actually matters to the end-user - a combination of methods and activities like nightly builds and code review are critical for success in delivering a high quality product.
Daily/Weekly Builds: Test Early and Often
To get the most benefit from your testing time and investment, it's important to test at regular intervals. What does this look like? You don't have to look far for examples:
- In this TestComplete testimonial Dana Epp from Scorpion Software explains: "We run regression testing daily at noon with TestExecute on a dedicated build and test system. This allows developers to check in work before noon and know before the end of the day if their submissions for fixed defects or new feature have been accepted against the entire product codebase."
- Just recently the Netflix test team blogged about test automation for their mobile applications. Amol Kher from Netflix writes: "We have a continuous build integration system that allows us to run our automated smoke tests on each submit on a bank of devices."
One of the key benefits of automation when integrated in the development process is that it provides feedback early - shortly after a defect is introduced into the code base, or ideally even before that. Additional automated and manual testing is likely to be required later on, but that doesn't take away from the value of early feedback through test automation.
It's important to keep in mind that it is possible to test much earlier than most people think it is. By getting started early, you are well on your way to a smoother running development lifecycle (and a higher quality end product).
What are some of the other best practices that can lead to a significant improvement in quality with minor changes to the existing process? An often overlooked opportunity for improvement is code reviews.
Better Code Review: What and When
Review is not testing; it is a different activity. However it shares the same goal - improvement of quality. Furthermore, just like early testing, code reviews have the potential to reduce the time from the moment when the bug is introduced to its detection. This saves time and money.
Benefits of this practice are clear:
- Code review practices find 99% more bugs when used together with other quality improvement activities like testing and static analysis. Here is an illustration of benefits as well as additional data points listed in Deb Cobb's excellent blog post.
- In his book, "Best Kept Secrets of Peer Code Review" author Jason Cohen cites that it is 2400% cheaper to fix bugs in development than in production.
Often, the main obstacle in the adoption of a more structured approach to code review is concern about unnecessary overhead. As with testing, tool automation can help extract more value from an existing review process by eliminating tedious tasks, allowing you to focus on writing better code and by collecting metrics along the way as an added benefit towards continuous improvement.
- In this testimonial for CodeCollaborator, Chris Holl, Director of Software Best Practices at Intuit, says the following: "CodeCollaborator goes far beyond this, particularly in reading information directly from the configuration management system to show reviewers exactly what changed from version to version." Also: "Entering defects is very easy and doesn't hinder the inspection at all. Other metric data, such as time spent performing the inspection are automatically captured."
Once you are on the path of continuous improvement through automation there are additional perks along the way. For example - code review of your test scripts. Tests can have bugs too.
As with everything in life, balance is key. Quality is the result of good practices throughout the development lifecycle, such as automated testing and code review. Alone, each process only gets you so far; together, they can help ensure fewer bugs, less time and better quality.
What are your best practices for smart testing?