Learning by Shipping

We are trying to change a culture of compartmentalized, start-from-scratch style development here. I’m curious if there are any good examples of Enterprise “Open Source” that we can learn from.

—Question from reader with a strong history in engineering management

When starting a new product line or dealing with multiple existing products, there’s always a question about how to share code. Even the most ardent open source developers know the challenges of sharing code—it is easy to pick up a library of “done” code, not so hard to share something that you can snapshot, but remarkably difficult to share code that is also moving at a high velocity like your work.

Developers love to talk about sharing code probably much more than they love to share code in practice.  Yet, sharing code happens all the time—everyone uses an OS, web server, programming languages, and more that are all shared code.  Where it gets tricky is when the shared code is an integral part of the product you’re developing.  That’s when shared code goes from “fastest way to get moving” to “a potential (difficult) constraint” or to “likely a critical path”.  Ironically, this is usually more true inside of a single company where one team needs to “depend” on another team for shared code than it is on developers sharing code from outside the company.

Organizationally, sharing code takes on varying degrees of difficulty depending on the “org distance” between developers.  For example, two developers working for the same manager don’t even think about “sharing code” as much as they think about “working together”.  At the other end of the spectrum, developers on different products with different code bases (perhaps started at different times with early thoughts that the products were unrelated or maybe one code base was acquired) think naturally about shipping their code base and working on their product first and foremost.

This latter case is often viewed as an organizational silo—a team of engineering, testing, product, operations, design, and perhaps even separate marketing or P&L responsibility.  This might be the preferred org design (focus on business agility) or it might be because of intrinsic org structures (like geography, history, leadership approach).  The larger these types of organizations the more the “needs of the org” tend to trump the “needs of the code”.

Let’s assume everyone is well-meaning and would share code, but it just isn’t happening organically.  What are 5 things the team overall can do?

1. Ship together. The most straight-forward attribute two teams can modify in order to effectively share code is to have a release/ship schedule that is aligned.  Sharing code is the most difficult when one team is locked down and the other team is just getting started.  Things get progressively easier the closer to aligned each team becomes.  Even on very short cycles of 30-60 days, the difference in mindset about what code can change and how can quickly grow to be a share-stopper. Even when creating a new product alongside an existing product, picking a scheduling milestone that is aligned can be remarkably helpful in encouraging sharing rather than a “new product silo” which only digs a future hole that will need to be filled.

2. Organize together to engineer together.  If you’re looking at trying to share code across engineering organizations that have an org distance that involves general management, revenue or P&L, or different products, then there’s an opportunity to use organization approaches to share code.  When one engineering manager can look at a shared code challenge across all of his/her responsibilities there more of a chance that an engineering leader will see this as an opportunity rather than a tax/burden.  The dialog about efficacy or reality of sharing code does not span managers or importantly disciplines, and the resulting accountability rests within straight-forward engineering functions.  This approach has limits (the graph theory of org size as well as the challenges of organizing substantially different products together).

3. Allocate resources for sharing.  A large organization that has enough resources to duplicate code turns out to be the biggest barrier to sharing code.  If there’s a desire to share code, especially if this means re-architecting something that works (to replace it with some shared code, presumably with a mutual benefit) then the larger team has a built-in mechanism to avoid the shared code tax.  As painful as it sounds, the most straight-forward approach to addressing this challenge is to allocate resources such that a team doesn’t really have the option to just duplicate code.  This approach often works best when combined with organizing together, since one engineering manager can simply load balance the projects more effectively.  But even across silos, careful attention (and transparency) to how engineering resources are spent will often make this approach attainable.

4. Establish provider/consumer relationships.  Often shared code can look like a “shared code library” that needs to be developed.  It is quite common and can be quite effective to form a separate team, a provider, that exists entirely to provide code to other parts of the company, a consumer. The consumer team will tend to look at the provider team as an extension to their team and all can work well.  On the other hand, there are almost always multiple consumers (otherwise the code isn’t really shared) and then the challenges of which team to serve and when (and where requirements might come from) all surface.  Groups dedicated to being the producers of shared code can work, but they can quickly take on the characteristics of yet another silo in the company. Resource allocation and schedules are often quite challenging with a priori shared code groups.

5. Avoid the technical buzz-saw. Developers given a goal to share code and a desire to avoid doing so will often resort to a drawn-out analysis phase of the code and/or team.  This will be thoughtful and high-integrity.  But one person’s approach to being thorough can also look to another as a delay or avoidance tactic.  No matter how genuine the analysis might be, the reality is that it can come across as a technical buzz-saw making all but the most idealized code sharing impossible. My own experience has been that simply avoiding this process is best—a bake-off or ongoing suitability-to-task discussion will only drive a wedge between teams. At some level sharing code is a leap of faith that a lot of folks need to take and when it works everyone is happy and if it doesn’t there’s a good chance someone is likely to say “told you so”. Most every bet one makes in engineering has skeptics.  Spending some effort to hear out the skeptics is critical.  A winners/losers process is almost always a negative for all involved.

The common thread about all of these is that they all seem impossible at first.  As with any initiative, there’s a non-zero cost to obtaining goals that require behavior change.  If sharing code is important and not happening, there’s a good chance you’re working against some of the existing constraints in the approach. Smart and empowered teams act with the best intentions to balance a seemingly endless set of inbound issues and constraints, and shared code might just be one of those things that doesn’t make the cut.

Keeping in mind that at any given time an engineering organization is probably overloaded and at capacity just getting stuff done, there’s not a lot of room to just overlay new goals.

Sharing code is like sharing any other aspect of a larger team—from best practices in tools, engineering approaches, team management—things don’t happen organically unless there’s a uniform benefit across teams.  The role of management is to put in place the right constraints that benefit the overall goals without compromising other goals.  This effort requires ongoing monitoring and feedback to make sure the right balance is achieved.

For those interested in some history, this is a Harvard Business School case on the very early Office (paid article) team and the challenges/questions around organizing around a set of related products (hint, this only seems relatively straight-forward in hindsight).

—Steven