Webinar Q&A with Jeff Sutherland - Secrets of High Quality Software Development
Test and Monitor | Posted March 16, 2012

We thank everyone who joined us for Jeff Sutherland's presentation on the Secrets of High Quality Development which is now available on-demand for those who may have missed it.

Below, Jeff has answered some of your questions that we did not have time
for during the live event. We hope you find it valuable. Please feel
free to ask any additional questions in the comments. 

Q: What size team is too small for scrum?

Zero is the quick answer. We’ve seen Scrums of three of two, and even one person. But for most teams the rule of 7 plus or minus two is the way to go. This usually provides enough team members to allow for cross functionality, but not so many that communication and sharing become difficult. The more common problem I see is that teams are too large. If you have a team with more than 9 people, your team is just too big. A team of 5 will go faster than a team of 7.


Q: How do you deal with product ownership and customers on the team with shrink wrap software where there are a wide range of different types of customers with differing needs?

We coach all of our venture companies on developing “personas,” which are archetypes of users for the product. Product Owners should take our Certified Product Owner course to dive into detail on this.

There needs to be one “vision” for a product. Now, there may be different types of customers (personas) that will be buying your product for different needs, but there needs to be one person who is prioritizing all of the differing customer needs. Is it more important to implement X feature for a certain type of customer, or Y feature for another? These can’t be of equal priority. If the product is large enough that you have multiple teams and multiple product owners, there still needs to be a Chief Product Owner who can settle questions of priority among the product owner teams. I strongly recommend that even in large projects, there be ultimately only one backlog that is organized in priority order. The reason for this is that the whole group should be focused on the top priority items, rather than diffusing their efforts on whatever features they think are important, rather than what the product owner and the customers think is important. Read the Citrix Online case study to see how to prioritize releases of a portfolio of products at the enterprise level (http://jeffsutherland.com/HICSS2010EnterpriseScrum.pdf). This is another topic covered in detail in our Product Owner course.


Q: You mentioned that high performance teams do code reviews and automated testing. How important is the role of tools in implementing scrum?

Just enough and as simple as possible is the rule for tools. Many of our venture companies use open source tools for configuration management and continuous integration. Some use SmartBear tools, particularly for automated code reviews. What is important is the goal. All software should be tested, bug free, and ready to ship at the end of each Sprint. 90% of the time, the bottleneck there is the testing. By automating testing I’ve seen companies drop weeks if not months off their deployment time. As to what tool you use? I’m actually agnostic. It could be off the shelf, could be one you build, could be crazy Jim down in the basement who does testing in the wee hours of the night while living off coke and fritos. The important part is to reach the end of each Sprint with shippable code. We mandate the use of automated testing and continuous builds for our venture companies, as we do not think they can compete effectively in a global market without them.

The implementation we like the best is at Central Desktop. Their testers work with the Product Owner to implement natural language acceptance tests in a tool called Cucumber before the sprint starts. All acceptance tests run during every build. At the end of the sprint, if all tests are green, they can comfortably push a button to move the code to production.


Q: Where do requirements & design documents, test planning and reporting documentations fall within the three artifacts of Scrum?

The key to requirements is that you have just enough to build the product. Huge requirements documents are rarely even read, let alone followed. You don’t want too little, but you don’t want anymore than you absolutely have to have. I build the requirements into the User Story process. In cases, such as life critical applications in Health Care, you may need to use lightweight Use Cases, but think of the minimum you actually need. Then you put that into the backlog as part of the User Story.

For Test Planning, many good teams are doing Test Driven Development, so the first thing they do with any story is to write a failing test. That should just be one of the tasks associated with that story in the Sprint Backlog.

Scrum actually gives you more data than you could possibly want for reporting. You want to know when the product will be released? Look at the release burn-down. You want to know when a certain feature will be complete? Look at the Sprint backlog, or at the estimate in points the team has made of that story and extrapolate from there.


Q: Could you please quote quick examples where Story Points - successful scenarios / failure scenarios. Also, from your observation how effective the Story Points had been?

The venture company I am currently working with has a huge problem in that they only measure stories in hours and not points. I actually have told them I think it is their biggest impediment in moving forward, as they have never been able to generate a hyperproductive team, even though subjectively everyone knows they multiplied performance by a factor of 3. I’ve also seen companies that use points as a thinly veiled expression for hours which will also cripple them.

The reason to use points, which I use in my company, Scrum Inc., and all the best Scrum teams use them, is that you can actually get a real read on your acceleration. You can never get more than forty hours of work in a week out of someone. But if you measure all the stories in hours, you won’t get a sense of how much faster they can go. If I measure someone in points, and after a few Sprints they are doing double or triple the number of story points they did in the first Sprint, I know they are accelerating towards hyperproductivity. But the number of hours a team can do in a Sprint is static. So how will you get good data as they accelerate?


Q: Please give a few examples of reasons for Scrum adoption failures. Already aware of lack of leadership support as major impediment to adopting Scrum.

Well, leadership support is a big one. The Standish Groups surveys indicate lack of management support is the leading cause of project failure. We’ve been working a lot recently with management to get them to see the value they will get from implementing Scrum and show them how to take responsibility for projects (which they should have been doing all along).

The other reasons?

Not removing impediments. If people aren’t focused on exposing and removing impediments, they will never get the full benefits of Scrum. Part of the whole reason Scrum is as effective as it is is that it mercilessly exposes dysfunction and impediments. If these are surfaced but not dealt with, everyone will just give up. Sometimes organizations and people are so wedded to their dysfunction they’d rather be messed up than go fast.

Laying a Scrum framework on a waterfall process…Oh we’ll do 3 Sprints of requirements, then 3 of Analysis…etc. That is just pretending you are agile.

Another is adopting some parts of Scrum, but not others. Not realizing that the Scrum framework is a self-reinforcing one. For example, sometimes teams don’t want to do the Sprint Retrospective, or even worse, they pretend to do it, going through the motions, without actually delving deeply into their practices to figure out how they can measurably improve their process. If they don’t do that, they will never achieve the true benefits of Scrum.

All that said, even badly implemented Scrum is much, much better than waterfall. At Yahoo, after implementing Scrum for hundreds of teams, they got a 37% reduction in software development costs on the average for poorly implemented Scrum.

The Standish Group reports three times the success rate for Agile projects compared to traditional projects during the past decade. They recommend always doing Agile projects. The Dept. of Defense has mandated Agile for all projects. The Gartner group says stop doing waterfall. All projects should be Agile, without exception.


Q: Dr. Jeff, one of the most common issues faced while estimating the effort for a feature being implemented as a part of a sprint is that the product owner always challenges the effort estimate provided by the team member .... how should this be addressed?

The person doing the work always estimates the work. Period. No exceptions. If necessary, refer them to research on the Wide Band Delphi Technique. This has been proven over and over to be the most accurate estimation technique. Product Owner’s challenging estimates will cause an anchoring effect and create bad estimates. Look at the research or do your own research!

If the Product Owner thinks the estimates are too large, he or she can ask why and de-scope the feature if appropriate. Trying to manipulate estimates will always get you in trouble, demotivate the team, and sabotage self-organization. The focus needs to be on process improvement. Acceleration sprint-to-sprint is more important than velocity this sprint.

If a Product Owner doesn’t believe the estimate, they should wait. Let the Sprint proceed, and see if the estimate was right or not. Scrum is all about measurement. Gather the data then have the conversation. It’s also important to remember that an estimate is just that, an estimate. Teams forecast what they can do in a Sprint, and it is crucial for beginning teams to be allowed to find their true velocity. If things are taking a lot longer than the Product Owner thinks is necessary, do a root cause analysis…find out why. Likely there are impediments that are getting in the team’s way that need to be addressed.


Q: Does Scrum encourage prototypes, in order to improve estimates on our backlog with fewer unknowns and with improved dependencies?

If the team thinks they need to prototype a feature, they should put that into the backlog as a story and tell the Product Owner the reasons for it. At the end of the Sprint they should be able to demonstrate that prototype at the Sprint Review (or at least an iteration of it if it is a large prototype).


Q: What can be done if a team refuses to self-organize? Do "alpha-animals" in teams destroy self-organization? How can we deal with them?

I’m going to take these last two together. I recently finished writing something about this for the upcoming edition of the Scrum Handbook. Here’s a free preview:
One complaint I hear a lot is that “My team isn’t self-organizing.” This usually happens when one or more members of the team is in a cynical “seen-it-all-before” mindset, which can drag down the rest of the team. The book “Tribal Leadership,” by Dave Logan, John King, and Halee Fischer-Wright lays out a fascinating taxonomy of levels of development. They separate it out into Levels 1-5. To oversimplify level 1 is the level where “the world sucks,” level 2 is where “my world sucks,” level 3 is “I’m great, but you’re not,” level 4 is “we’re great, but they’re not,” and level 5 is “the world is great.”

I don’t want to re-hash their whole book, but when teams aren’t self-organizing, it usually indicates that one or more team members is stuck at level 2. This is the mindset that nothing can change, management sucks, this process sucks, etc. The behaviors driven by this kind of thinking are passive-aggression, lack of initiative, and usually an attempt to bring others to their level as they are the smart guys who see the world “as it really is” and anyone who thinks differently is just a fool. Think of you’re average DMV employee.
Addressing this problem can be difficult, but it isn’t impossible.

First of all, it is important that the Scrum Master be skilled in interpersonal relationships and the dynamics of motivation. Usually, direct confrontation doesn’t work that well with people in this mindset. Instead, what I’ve found effective is pairing that person with someone at a higher level, so they can learn and change through example.
A good Scrum team lives most of the time at level 4, and that kind of team often builds up an immune response to people at lower levels, either forcing them to rise up to the level of the team, or forcing them out.


Q: In highly regulated environments like Medical Devices, how difficult is it to "marry up" the Waterfall up-front requirements and "only when needed" detailed User Stories in Agile Scrum? Is there a conflict that cannot be resolved?

Today, upfront requirements for medical devices are often not updated sufficiently at the end of the project to have accurate traceability (since 65% of requirements tend to change during development). The Waterfall requirements should be broken down into user stories and implemented as specified to ensure traceability. The user stories should be spawned from an enabling specification that is short, accurate, and up to date, ensuring traceability.


Q: In a heavily regulated industry like heath care and pharmaceuticals, how can we adopt a pure Scrum methodology (instead of waterfall or a Scrum/Waterfall hybrid) while still being able to document the requirements-to-design-to-test/validation traceability required by regulators?

A recent FDA medical device project had 12,000 pages of documentation. Working with regulators, Scrum coaches reduced the documentation to 800 pages, mostly automated, in the context of working with regulators. Ongoing work suggests documentation could be reduced to 300 pages and be compliant with traceability requirements. You need to let the Scrum team find the minimum documentation needed to satisfy traceability requirements in the context of working with regulators.


Q: How does a scrum team deal with upper management groups that aren't fully invested in the scrum process? I.E., the team is given a hard ship date at the start of the project, the feature set includes only large features that cannot be broken down, and the resources are fixed. And then the team is asked to "do Scrum," and the team is told that they have "committed" to the ship date.

The team never commits to a ship date. This is the Product Owners responsibility based on business needs and known velocity of the team. If the management is interfering with the team, this is a major dysfunction and needs to be addressed as an impediment. Coaching of the management is required. Management dysfunction is the primary cause of all project failures and it is independent of Scrum. As least with Scrum, the success rate on the average worldwide is three times better than waterfall, even with the dysfunctional management in the world.


Q: How do you improve and measure code quality during the iterations?

The definition of Done should include no additional technical debt and there should be a plan for technical debt removal. This means no known bugs generated in the sprint and reducing the number of historical bugs as prioritized by the Product Owner.





By submitting this form, you agree to our
Terms of Use and Privacy Policy

Thanks for Subscribing

Keep an eye on your inbox for more great content.

Continue Reading

Add a little SmartBear to your life

Stay on top of your Software game with the latest developer tips, best practices and news, delivered straight to your inbox