You don't have to contribute your code to the world to gain advantages from open source methods. An expert offers suggestions on how to use open source practices to run internal code bases effectively.
Let's say your business doesn't have one bit of interest in sharing its internal code with the outside world. Does that mean that open source methods of programming can't do you any good? Camille Fournier, Director of Engineering for retailer Rent the Runway, Apache ZooKeeper committer, and former Goldman Sachs VP, believes adopting open source software development methods can help even your organization.
At OSCon, the major open source convention in Portland, OR, Fournier opened her session, “Internal Open Source: Running internal projects with open source methods,” by observing that while "open source software is everywhere," not all companies have adopted it.
“Most companies, big and small, have some code that is shared by many but owned by few. It may be in the form of useful libraries for common tasks like logging and configuration. It may be in the form of internal services that cross business functions,” she said. “Managing these libraries and services often falls onto the teams that originally created them, or on a core tech team that owns all shared software. This model has many drawbacks: slow changes, a lack of transparency for users, and a tendency to cause bottlenecks for other developers."
The modern enterprise has fully embraced open source software (OSS), Fournier said, but there is still an opportunity for us to learn from the processes used to create that software and apply it to our internal shared software. “Encouraging shared contribution across teams to these code bases, with a 'committer group' that ensures quality and stability, solves many of the challenges of shared software,” for example. “It’s also a great opportunity to give developers exposure to code they may otherwise not have the time to examine, and increases shared knowledge across the company."
In many organizations, developers face the friction of communications overhead, Fournier said. “It's part of all human endeavors. If your developers are spending more time in meetings than in your code, then you know you're in a high-friction environment."
Start-ups solve this problem by working with five people in a room. But once you grow, Fournier pointed out, you have a great deal of friction if you solve issues by having people talk directly to each other in meetings or by pulling or polling people. “Open source projects have had to handle this from the get-go," since they regularly work with teams with remote workers – what you’d ordinarily call “telecommuters.” So open source projects manage the communication friction problem by making both the code and issue-tracking public.
Plus, Fournier reminded us, "Transparency also produces quality work."
Code improvement comes from community. Fournier observed, "We can always get better if we can learn from each other." This can be done with mailing lists, instant-messaging, and "public" code repositories and bug tracking. Adopting such practices inside a private business can create "a living, breathing community inside your company."
Communities can also give developers recognition not only from their bosses but from their peers. Fournier believes this leads to even better quality code, as "It also increases their commitment to their company." It does this by encouraging programmers to feel more that they have a stake in the company's development efforts.
Therefore, she thinks that perhaps most internal projects should be handled as if they were open source software. In particular, you should look for shared corporate IT needs. “OSS exists and thrives in place of shared need,” Fournier said.
Typically, IT needs are handled in one of three ways:
First, "Every team implements exactly what they need. That's great but it doesn't scale, and it wastes a lot of developers cycles."
Second is collective code ownership. This often shows up in Agile environments. The problems with this approach is that it usually requires a lot of mentoring and "It doesn't tend to scale super-well for everything." That said, she conceded, "It's great for long-lived projects with long-lived developers."
Finally, it's very common to create a “core” team that owns all centralized code. That can work well, but if it's not done in OSS fashion the core team often doesn't use the code it produces. This can lead to a ton of meetings with high friction and it can also lead to feature bloat.
Open source methodology can avoid all the problems to which these other methods are prone, she asserted. Fournier gave examples from her experiences in bringing open source methodology to a Fortune 500 financial services giant and to a start up.
At the gigantic multi-national financial services company with 10,000+ developers, "Each division would have its own development team. There was little centralized development and no one talked with each other." How bad was it? "Even internally you'd get a compiled library from other groups instead of source code."
Each group also had a huge disparity on how they did software development. "Some groups didn't do automated testing, others used waterfall development, Extreme Programming here, Agile there." Having different styles can be okay, but "When you have a super-wide spread of styles, anytime someone moves they have to relearn everything. That's bad for internal mobility and for code sharing,” said Fournier.
To move from this workflow style to an OSS methodology, Fournier first needed to get management buy-in. That actually proved relatively easy, but it was also essential.
Once that was done, she and her colleagues started to work on building shared code repositories, shared ticketing systems, mailing lists, and code review systems. That was not easy because, except for mailing lists, the company didn't have any of these fundamentals in place.
So, after creating those tools, her team pushed for others to adopt this company-wide shared methodology. They did so with birds-of-a-feather meetings and education. The most significant advantage was probably the creation of a shared platform for the company’s software development life cycle (SDLC) which helped to bring in the various federated teams.
It worked. Within months, 40 projects moved to the open platform. When the core teams moved, entire division teams started to move their projects into the shared platforms.
At the small startup, Fournier began by using OSS from day one. Even with time constraints and heavy business pressure, it worked well when the developer team was divided into two teams: front end and infrastructure.
Six months ago, she said, the startup changed its team structure to pods, cross-functional teams aimed for business goals. It worked well. People loved the code visibility, and they loved being able to modify code.
But along the way this new structure lost the open source code review process, and it turned out developers really missed this. "Everyone wanted more formal code reviews,” Fournier said. "Developers were nervous about their code without code reviews." She admitted, "I was undermining myself. We need code ownership." She's since restored this in the development process; the developers are much happier.
Not convinced that OSS is the right way? Fournier believes that if you start small and simple, you may be surprised at how well it can work for you. In addition, she believes that OSS practices can bring a great deal of developer joy. Even symbolic projects can have a halo effect and improve your development teams' morale.
What do you think of Fournier's experiences and thoughts with OSS methodology? Do you agree or disagree? Do you have any personal experience? Let us know what you think.
See also:
