SmartBear
Cart
Login
Welcome back to SmartBear Software
Click the links below to login to your favorite tool or into your client services portal
SmartBear Forum
Register
Log in
Development
DevComplete
ALMComplete
AQtime Pro
Automated Build Studio
Testing & QA
TestComplete
TestExecute
TestRecorder
QAComplete
LoadCOmplete
Web Monitoring
AlertSite
Free Tools
QAPlanner
DevPlanner
Looking for a Free Trial?
Products
Products
For Development
Collaborator
Code Review & Document Review
ALMComplete
Lifecycle Management
AQtime Pro
Performance Profiling
Automated Build Studio
Software Release Management
SoapUI Pro
API Testing - SoapUI Pro
All Development
For Testing & QA
TestComplete
Automated Testing
QAComplete
Test Management
WebLoadUI Pro
Load Testing
AQtime Pro
Performance Profiling
LoadUI Pro
API Load Testing
All Testing & QA
For Web Monitoring
AlertSite
Website Monitoring
AlertSite API Monitoring
Free API Monitoring
AlertSite Insite
Internal Website Monitoring
Free Tools
AlertSite for API Monitoring
CodeReviewer
WebLoadUI
DéjàClick
AlertSite for Magento
Open Source Tools
SoapUI
LoadUI
Get the Free Tools
For Your Needs
Agile Testing
API Testing
Automated Testing
Code Review
Load Testing Tools
Test Management
Web Monitoring
Resources
Resources Overview
eBooks
Templates & Worksheets
Videos
Whitepapers
Support
Support when you need it
SmartBear technical representatives are always ready to help with your product needs.
Browse our support portal
Know what you're looking for?
Downloads
User Guides
How To Tutorials
Technical Articles
Screencasts/Videos
General support
Forums
Troubleshooter
FAQs
Product Versions
Community
About Us
Contact Us
Our Partners
Our Team
Our Customers
Careers
Awards
Our Speakers
Request a Speaker
News & Events
News Releases
Events
In the News
Webinars On-demand
Prior Events
Blog
Share
Home
›
CodeCollaborator Video Case Study-Be a Boy Scout-Be Prepared!
CodeCollaborator Video Case Study-Be a Boy Scout-Be Prepared!
SPEAKER 1: So another thing we found was this. Now, this part is not a proper experiment, so big star, because this was not something we even expected to study. This was something we discovered in the data. And so we tried to analyze it and draw some conclusions, but big asterisk, it's not in any way a controlled study. But I found it very interesting, and I think it's significant. There you go. So what we found was this, in our tool, if the code author wishes to, the code author can go through and annotate the code first. Here's why I did this. I called this method. I put this over here because of whatever-- like explaining or prepping the reviewer. So we called this Author Preparation. And it's not required. This is the problem. So some people decided to do it-- self-selection-- again, not an experiment. Some people decided to do it, and some people didn't, and there was a huge discrepancy in the average defect density. See, I just told you I don't believe in that number. But that's OK. The average defect density with preparation was half of those without preparation. And that's a huge diff-- I mean, and it was statistically significant. But it was a huge difference. So we tried to ask why that is-- again, not an experiment, so this is subjective now. And so what we did is we picked 350 random reviews in which there was Author Prep, and we looked at it to try to see if we could subjectively decide what was going on. And there was at least two theories. I mean, there's more, but we had at least two theories. The optimistic theory was that the reason there were fewer defects found is that just like when you teach someone something, and then you really internalize it at that point by having to describe what you did to someone else, you find a lot of your own mistakes. You've experienced this sometimes, right? As you're writing-- I put this here, and wait, I shouldn't have. Right? Back, back, back. Hold on. So that's the optimistic view, is you find your own errors in having to regurgitate what you just did. The pessimistic view is that the reviewers were primed-- that's the psychological word-- "primed"-- to not find defects because the author says I called this method, and the reviewer goes, yes, you did. Of course, that's not a code review-- [CHUCKLING]-- right? So, wait, should you have, right? That is actually the question. Or the author has all these explanations, and the reviewer's like, oh, wow, sounds like he knows what he's talking about-- cool. In other words, you're sort of not in the mode to find defects because you either now trust the author somewhat, or you're in matching mode instead of thinking mode. And that's why we didn't find defects, because we're not trying as hard. That's the pessimistic view. Of course, there's probably the most likely theory, too, which is that since this was self-selection, the developers who are naturally more conscientious and therefore probably didn't have as many defects or found them were the ones who did it. And again, since it wasn't a controlled experiment, we can't know. But I submit to you that we looked at 350 reviews, and we tried to see which of these theories seemed correct.
