Using data to inform strategy
Access to rich usage data is something that is a defining element of modern product development. From cars, to services, to communications the presence of data showing how, why, when products are used is informing how products are built and evolve. To those developing products, data is an essential ingredient to the process. But sometimes, choices made that are informed by data cause a bit of an uproar when there isn’t uniform agreement.
The front page of the Financial Times juxtaposed two data-driven stories that show just how tricky the role of data can be. For Veronica Mars fans (myself included), this past week was an incredible success as a Kickstarter project raised millions to fund a full length movie. Talk about data speaking loud and clear.
This same week Google announced the end of life of Google Reader, and as you can see from the headline there was some controversy (it is noted with irony that the headline points out that the twitterspehere is in a meltdown). For all the 50,000 or more folks happy with the VM movie, it seems at least that many were unhappy about Google reader
Controversy
The role of data in product development is not without controversy. In today’s world with abundant information from product development teams and analysis of that data, there is ample room to debate and dissect choices. A few common arguments around the use of data include:
- Representation. No data can represent all people using (or who will use) a product. So who was represented in the data?
- Forward or backward looking. When looking at product usage, the data looks at how the product was used but not how it will be used down the road (assuming changes). Is the data justifying the choice or informing the choice?
- Contextual. The data depends on context in which it is collected, so if the user interface is sub-optimal or drives a certain pattern the data does not necessarily represent a valid conclusion. Did the data consider that the collection was itself flawed?
- Counter-intuitive. The data is viewed as counter-intuitive and does not follow the conventional wisdom, so something must be wrong. How could the data overlook what is obvious?
- Causation or correlation. With data you can see an end state, but it is not always clear what caused the end-state. If something is used a lot, crashes a lot, or is avoided there might be many reasons, most not readily apparent or at least open to debate, that cause that end-state. Is the end-state coincident with the some variables or do those variables cause the end-state?
When a product makes a seemingly unpopular change, such as Google did with reader, some of more of these or other arguments are brought forward in the discussion of the choice.
In the case of Reader, the official blog stated “usage of Google Reader has declined”. While it does seem obvious that data informed the choice, if one does not agree with the choice there is ample opportunity to dispute the conclusion. Was the usage in absolute or relative decline? Were specific (presumably anonymous) users slowing their usage? What about the usage in a particular customer segment? The counter-intuitive aspect of the data showed, as most dialog pointed out strong first party usage among tech enthusiasts and reporters.
The candid disclosure of the use of data offered some transparency to the choice, but not complete transparency. Could more data be provided? Would that change how the product development choice was received?
Conversely, no one is disputing the success of the VM Kickstarter project (making a movie is similar to product development). The data is clear, there is a high level of demand for the movie. I know that fits my intuition as a fan of the series. The fact that this data came via a popular (and transparent) web service only seems to validate our intuition. In this case, the data is seemingly solid.
While these are just two examples, they happened in the same week and show the challenges of data, transparency, and product development choices. While data can inform choices, no one is saying it is the only way to make a choice or that those making products should only defer to data. Product development is a complex mix of science and intuition. Data represents part of that mix, but not the whole of it.
Data is not strategy
Ultimately, data contributes to product development but does not replace the very unscientific notion of what to build and why. That’s the art of product development and how it intersects with business strategy. This follows from the fact that products are developed today for use in the future. The future is uncertain for your own product’s evolution, but all around you is uncertainty. Data is an input to help you define a strategy or modify it, but cannot replace what is inherently the uncertain side of innovation.
In Eric Ries’ Lean Startup book (or http://en.wikipedia.org/wiki/Lean_Startup), there is an interesting discussion on how the role of data can contribute to an early stage product. While the anecdote and approach are described in the context of a project very close to Eric, I think we can all see parallels to other products as well. My intent is not to restate or recast the book, but to just reflect upon it a bit.
One part of developing a new product, as described, is to develop a minimum viable product (MVP) that does not reflect the final product but is just enough of the product to collect the maximum validated data about potential customers.
An interesting point in the description is how often the people that will use this early version of the product are enthusiasts or those especially motivated and forgiving about a product while under development. The tricky user interface, complex sign-up, or missing error conditions and things like that might not matter to these folks, for example.
Not every product ultimately targets those customers—they are not super large in number relative to the broad consumer world, for example. As you learn and collect validated data about your product strategy you will reach a critical point where you essentially abandon or turn away from the focus on enthusiasts and tilt towards a potentially larger customer base.
This is where your strategy comes into play. You have iterated and validated. Presumably these early users of your product have started to use or like what you have or at least pointed you in a direction. Then you’ll take a significant turn or change—maybe the business model will change, maybe there will be different features, maybe the user interface will be different. This is all part of taking the learning and turning it into a product and business. The data informed these choices, but you did not just follow it blindly. Your product will reflect but not implement your MVP, usually.
But with these choices there will probably not be universal agreement, because even with the best validated data there can be different approaches to implementing the learning.
Data transparency
The use of data is critical to modern product development. Every product of every kind should have mechanisms in place to learn from how the product is used in the real world (note, this is assuming very appropriate policies regarding the collection and usage of this data). This is not just about initial development, but evolution and maturing of the product as well.
If you’re going to use data to design and develop your product, and also talk about how the product was designed and developed, it is worth considering how you bring transparency to the process. Too often, both within an organization and outside, data is used conveniently to support or make a point. Why not consider how you could provide some level of detail that could reduce the obvious ways those that disagree with your approach might better understand things, especially for those that follow along and care deeply. Some ideas:
- Provide absolute numbers for the size of the data set to avoid discussions about sample size.
- Provide a sense of statistical significance across customer types (was the data collected in one country, one type of device, etc.).
- Provide the opportunity for additional follow up discussion or other queries based on dialog.
- Overlay the strategic or social science choices you are making in addition to the data that informed the choices.
Transparency might not remove controversies but might be a useful tool to have an open dialog with those that share your passion for the product.
Using data as part of product development will never be free of debate or even controversy. Products today are used across millions of people worldwide in millions of different ways, yet are often used through one software experience. What works in one context doesn’t always work in another. What works for one type of customer does not always work for another. Yet those that make products need to ship and need to get products to market.
As soft as software is, it is also enormously complex. The design, creation, and maintenance preclude at today’s state of the art an everything for everyone approach. To make those choices a combination of data an intuition make a good approach to the social science of product development.
–Steven
PS: Please click the link on the first use of data for a discussion of data versus datum. :-)
Combining guessing and planning in product development
At the extremes of product development methodology, characterized by waterfall and agile approaches, are different views about planning. Many today would say that planning a product “up front” is nothing more than guessing that locks you into a guess that will be wrong. At the other end, the conventional wisdom is that you should get something to the market soon for testing as a best guess and then iterate and learn to further develop a product. Since no team really practices a method precisely, let’s look at how to get the best out of the planning aspects of your own product development process.
Challenges
Developing a product, new or an update, always faces the same challenge at the start. Using software as an example, folks want to get coding as soon as possible since not coding is clearly wasting time, but the team (and management) want to know that the code will yield the right product (useful, cool, innovative, profitable, whatever). There are usually those on the team that claim instinct tells them what to go build. There are those that are certain they can work out an answer to the right product with enough up front ideation.
To compound these broad challenges, different disciplines have different perspectives. It is likely testers will want to spend more time up front on building a baseline and foundation for the work, but a baseline relative to what? Ops needs to know answers to questions in order to scale and provide the required infrastructure, but those answers require a lot of information you probably don’t know. Designers usually want time to iterate in lower fidelity media in order to hone in on the design language and overall approach. Business folks want to know that their key problems (onboarding, churn, referrals, etc.) will be addressed.
All of these lead to the natural tension between a desire to get started and a desire to pause and make sure the start heads in the right direction.
Waterfall (using this as a positive description) methods argue that an effort should begin with work to identify the problem being solved, available technologies, and proposals for how to go forward. The basic notion is that you should spend the energy considering a wide range of alternatives before you just start down a path that might be fruitless. These days you will be hard-pressed to find proponents of waterfall approaches because of the downsides often associated with waterfall execution.
The downsides of this approach, as critics claim, is that you waste a lot of time building up robust plans that are “going to be wrong (or outdated) anyway”. This criticism essentially states the reality that most plans are still guesses. This is particularly true in a fast-changing, multi-player marketplace where you can wake up one morning to a competitive product or dramatically new approach to solving the problem you identified many moons ago. The ultimate downside of the waterfall approach is that it is viewed as stubborn—so stubborn that even in the face of awareness about a changing market, the team has no choice but to move forward. Proponents will offer tools to mitigate any of these risks and say none of the risks are intrinsic to the methods.
Agile development methods respond to these criticisms by creating an approach where the primary focus is to get enough product work done so you can learn from real customers and then iterate. At the extreme, efforts should actively strip away all non-essential elements of the product development process and focus on the essence of the idea. Agile methods focus on constant iteration, learning, and responding to usage of the product (or lack thereof). Teams focused on iteration work in a tight loop to quickly adapt to the changing landscape and competitive dynamics, in addition to learning.
The downsides of agile methods are actually trickier to pin down right now. These days being critical of agile methods labels one as somewhat of a Luddite in the world of product development. That said, there are understood challenges with agile methods. The pressure to release can often result in a quality bar that is less than customers (or your own testers) would appreciate. A focus on learning might cause you to learn that your service does not scale or scales in a cost-ineffective manner. A large project (people, code, partnerships) is challenging enough to keep coherent and multiple efforts executing in different iterative loops poses a significant architectural and communication challenge. Proponents will offer tools to mitigate any of these risks and say none of the risks are intrinsic to the methods.
It is no surprise that proponents of any method can point to successes, while critics can point to failures of methods as well. In reality, the causal relationship between a project success and the method used is weak at best. Even with examples of success, we need to keep in mind that product development projects are not traditional repeated processes and as such the ability to draw scientific conclusions from them is limited. This should be apparent from the fact that successful products come from both methods, and equally true is that products can be failures from both methods.
It is not unusual for failures in waterfall-styled projects to be ascribed to the methods, but not so with successes. Whereas failures in agile projects usually ascribed to external factors, not the methods. In today’s climate, it is not unusual for agile to be viewed as a causal factor for a successful product.
That’s really why starting the project by declaring the methodology runs the risk of missing the big picture. It runs the risk of spending finite and scarce time on abstract or “meta” concepts and not the work at hand. The best thing is to avoid going down that path in the first place and focus on getting clarity on what work will get done and when that will happen (and why!).
The truth is that starting a product is always a guess. Whether you plan every detail or just start coding, beginning a new product is a leap of faith based on intuition. To those of us that build or have built new products, this leap is the most interesting, terrifying, and rewarding part of product development. It does take a special confidence to make that leap. Planning every last detail can give you a false sense of security. Coding out of the gate can give you a false sense of progress. Guessing is guessing, whether you have 1000 pages of specs and high fidelity model or a whiteboard with a sketch and a functional prototype.
These challenges—choosing between methods and the known challenges of any methods—are real. In product development we are faced with these every time we start a new project. If you have a clear starting point, clear points in time to check on your progress, and a plan so you know what you’re working against then you’ve defined a methodology that works for you.
Where to start?
In a previous post we talked about focusing on the work and not worrying about the label of the methodology. A concrete way to realize this is to take a step back and see how to combine the elements of agile and waterfall in the right amount for your project.
An approach that scales from new projects to next iterations, and small project to big, is to plan to iterate your plans. While this sounds like an obvious cop-out to pick the best of both extremes, it is what reality tends to look like in practice. If you start out knowing you are going to commit time to up front planning, but recognize you will take points in time during development to adjust and learn, then you can mitigate the challenges of both methods.
Of course agile proponents say there are always some plans. And waterfall proponents always say there is room to adjust. Let’s just say those labels and attributes don’t matter and try to arrive at an approach of where to start.
Every project can start with a plan. Legendary products start with a sketch on the back of a placemat at a diner or an all-night coding session. The original plans for some pretty big projects were conceived and documented in pretty short, but articulate, memos or detailed sketches/prototypes. The spark for a plan can come from anywhere and different people have different ways of translating that spark into something more than one person can internalize and visualize.
While a tool like PowerPoint can communicate the gist of the plans, the details will be too open to interpretation. So write down the plan in long form—writing is thinking.
The simple act of writing down a product plan in a couple of dozen pages opens you up to have useful discussions with a broad set of people. If you’ve identified the problem being solved, competitive products/services, technology bets, and overall investments you have the basis of how to talk with marketing, design, development, testing, and product management. Everyone can look at the plan from their perspective and offer insights, advice. You can even package this up as a dialog or exercise with potential customers.
Combining this overview with a functional low-fidelity prototype is a way to visualize the plan for a broader audience. It is usually a good idea for the prototype to support the written description and to lead with the written description. You’ll want to minimize the time you spend on “don’t worry this UX (user experience) isn’t final” or responding to feature suggestions without the context of where you are heading.
This is all a plan needs to be. It is a guess. You can’t prove it is right. You can’t prove it is a good business, great product, or the right thing to do. You can criticize it. You can add more to it. You can find problems. That will be true of any plan (or frankly any product). But you now have a foundation to move forward. To build something that is a “target” that is shared by a group—a vision.
With this plan comes the first chance to iterate. What’s amazing about just writing this down is how much you’ll find you’re iterating on your own thinking. You can think of this somewhat as agile planning. This shouldn’t be new though – anyone who has “told a story” of a product or an idea knows that the story improves quite a bit as you tell it more. This is basically the same thing. Any good product plan is a story—the problem you are solving is the challenge to overcome, the competitors are the antagonists, the technology bets and investments are the plot devices.
Depending on the size of the team/project, the next steps are about the specifics of what to do when. The amount of up front work and the ordering of the tasks is really a choice the team needs to make. Being economical about what you do is of course a key part of ordering. Agile methods often say to do the least possible work to express the unique value of the product. Waterfall methods are about landing all the details early. Different projects will simply have different ideas of what to do at this point and your own intuition as to what makes a good investment of time, relative to quality and time to market, will dominate.
Iteration: Local or globally optimize?
Regardless of the specifics of your development schedule, you are going to iterate. You can choose to iterate after code is in the hands of customers (in a broad or limited way) or just self-hosting until a broader release. The key though is to iterate.
Iteration is as much of an art form as deciding how much of what investments to do up front. It is super easy to fall into a trap of iterating but not making forward progress. You can find yourself rewriting the same code, circling back to previously discarded alternatives, or just changing things but not making them better. As necessary as iteration is, simply iterating does not mean you and the project are moving forward.
The lack of iteration or iteration at only the most fine-grained levels is potentially a sign of a project that won’t learn as it goes. Consider a standard kitchen remodel, something that has been done millions of times. Architects draw up plans and pass them off to contractors. Then you run into existing conditions. You find you’re missing an electrical run or there’s a support wall where one wasn’t expected. It is time to iterate on the plans. You can hack through a solution with your contractor on site. Or you can take a step back and work with the architect to reconsider the design. Either can work depending on your constraints. When you consider the time value of choices, it becomes more interesting to think about taking a step back.
In the heat of the moment with the need to get in market or respond to significant challenges with early customers, a redesign or revisiting decisions seems risky. Maybe the data is poor. Maybe the fear of discovering the need for big changes concerns you. Perhaps you just want to keep moving forward. All too often when problems arise in a project the need to iterate quickly trumps taking a step back. In today’s environment it is often viewed as a positive to iterate and try something different. Activity is not always progress. Change is not always improvement.
Of course the data you use to determine what to try and how to value the feedback is important. It is just as easy to get led astray by the wrong measures of success/failure. Regardless of the quality of data, you’re going to reach a point where you are faced with the need to change something. The question is whether the changes are the right ones.
The point of a change will be to optimize your product. You’re going to have to pick the dimension for which you want to optimize—is it for immediate mitigation or longer term success. It is easy to see mitigating the immediate challenges with some changes. Longer term might feel like another guess. On other hand immediate changes have an obvious fragility relative to broader goals and there’s clearly an appeal to being thoughtful about what to change.
Having a sense of a plan helps you to weigh the alternatives. Are you dealing with a bug or minor design nit or is there a fundamental flaw in the value proposition? There’s no mistaking the reality that you might hit a major reset, especially on a brand new product. There’s also a reality that you will have to revisit a pretty broad set of small design choices—that is steps in a flow or portions of a design, rather than the entire flow or design language.
Defining a time up front when the product is in a state to evaluate all the feedback and make choices about how to optimize is an important part of the process. This checkpoint stage can be a first self-host, private beta, partner beta, public beta or anything in between. Any product today is going to have telemetry and an understanding of how it is used and what you are measuring. This helps you to inform both what is going on and even what you failed to measure correctly.
Armed with a set of potential problems to address—optimizations yet to be done—there’s a simple question to ask of the team, which is “do we need to change/fix this or not, and if we need to take a step back and re-evaluate?”
A way to look at what to change/fix or not is to think of changes relative to the longer term goal, to go beyond the immediate. There’s no doubt the feedback about something is real. The question is really whether the cost to change (hours, risk, churn) gets you closer to the broader goals of the plan or is more reflective of iterative activity.
A checkpoint discussion where members of the team are aware of all the changes going on and what is being prioritized is a way to level-set. Some teams might have bigger challenges or more changes and other teams might be making more local changes. Calibrating these across the team is akin to making sure the whole project is thinking and acting globally, not locally. The plan that was put in place serves as a reminder of what the team was hoping to accomplish. Accountability, aka decision making, can be clear because the roles and responsibilities are clear and communication has been clear. A discussion to inform doesn’t have to be an invitation for everyone else to join in revisiting the choices made by the team.
Is the new data driving you to revisit the plan completely (whether immensely detailed or not)? That could be. For a brand new business and/or a brand new product where the effort is to grow an entirely new customer base, you could be going in a wrong direction. For a product update, there’s always going to be pressure from existing customers to innovate in a more incremental manner versus taking the product in a new direction. The presence of a plan allows you to have an informed dialog as to what went wrong. When you make choices to change things you have a shared foundation upon which to agree about what went wrong.
Is the data driving you to tweak something? That certainly is the case with some changes. The presence of the plan allows you to decide how critical it is to make changes. Too often changes to the code are made because of the presence of feedback even if the change doesn’t really alter the overall outcome. When you choose to keep things a bit more stable it doesn’t have to be viewed as blindly sticking to a plan when it can just be prudent engineering of cost-benefit. The capability to change is not the same as the value of change. Something that might be 10% better might introduce a high risk of change management or might just be 100% different–ask if something really is twice as good (or 10x better) after the change.
There’s a simple view of optimization that one can use in having discussions about changes—whether at the feature level of the whole plan. The idea is to discuss whether the change is a global optimization or a local optimization. When resources are right and time to market critical, optimizing locally is wasteful. When the plan is generally right but has some holes you discover, then making sure you optimize globally leads to an agile view of planning. The following just sketches this conceptual view–believe it or not this can often be a useful visual aid in the discussions around whether a change is needed.
Whether you label the axes performance, suitability to task, conversation rate, success rate, or transactions per second is not really as important as taking a step back and asking the question about whether the change gets you close to where you need to be over time. It is far too easy to get caught optimizing your plan relative to the nearest peak, not the best peak.
Whenever you have more than a few folks working together, having a set of tools to help you make a consistent set of choices across the team is critical. The more there is a shared view of the goals and the way to make choices the easier this becomes—a plan is a way of encapsulating the broader goals and giving you a place to both measure relative success and to decide the target was wrong. The presence of a plan does not enforce rigidity any more than the use of agility guarantees you will iterate to success.
Product development is a lot of guesswork. Planning, checkpointing, and deliberate decision making are tools to help you make the most informed guesses you can make.
This Week’s Poll
This week kicks off a new feature of Learning by Shipping — Three Quick Questions. This is a snap poll to share aggregate (non-scientific) reactions to the topic of this post, which will be reported in the next post. Take the poll – Three Quick Questions. Cameron Turner, an expert in big data and measuring how products are used in the real world is helping with these polls. No identifying information is collected or maintained.
–Steven
###
Benefiting from skip-level one-on-ones
Staying connected to your skip level manger and she staying connected to you are valuable for the project, the team, and each of your ongoing development. Rigorously and consistently making the most of skip-levels, whether as the manager or individual employee, is an important skill to master. This post looks at one-on-ones from the perspective of the manager with some tips for the employee/individual.
Time pivot, becoming a manager
One day in your career you might come to work and it will be your first day as a new manager. Many things will seem different. All of a sudden you not only feel responsible for code and features, but you’ve taken on a new responsibility for people. For those that you now manage, a new set of demands are placed on your time. One-on-ones are a key tool for you as a new manager.
One-on-ones are the most precious time you can spend with members of the team—your new direct reports. Much has been written about how to “have a good one-on-one” or techniques for making the most of the time (there is a post in our One Strategy book or check out Ben Horowitz’s nice post). I’m a firm believer that 1:1s are the most important of all scale management tools.
The easiest thing to remember about a 1:1 is that as a manager the meeting is not for you but for the employee. Your opportunity to learn is based on the topics raised and questions asked by the member of the team, only asking questions to draw out the issues if necessary. The first sign of a struggling management chain is when employees on the team start to see one-on-ones as being called to the carpet on a regular basis or when managers view a one-on-one as their time to manage the project or to be the keeper of the agenda.
As a manager, the notion that your time is no longer your own, but is there to serve the folks on your team is significant pivot, and somewhat counter-intuitive. You might have a manager you feel “takes up too much of your time” or you might be thinking “I have real work to do instead of sitting here”. You might be worried about all you need to get done as a team and react by asking more (or too much) of your new direct reports. In other words you’re in for a bit of learning about how you now manage your time, in addition to how you manage your own work and the role of management. So you aspire to do better when you’re on the other side of the table, and effectively using 1:1s is a key step.
As a first step of this new journey, consider how you can live up to the mantra that your role is to take up as little time as possible from your direct reports. This only benefits the organization as a whole because there will be more time for work, fewer meetings, and overall more time building products. The 1:1 is your request for time, but how you turn that into a benefit for the member of the team makes it valuable and not a manager tax.
A rule of thumb you might follow is don’t ask anyone to do things for you, but ask folks what they could do that helps them get the work done they believe they are supposed to do. In other words, status reports, project reviews, and statistics may all be valuable, but only if the folks on the team decide that on their own.
At the same time, 1:1s are a key tool to get a pulse on the team and on the work of folks on the team. Through an unstructured two-way dialog you can likely learn more about what is going on than you can from status reports or other outputs open to obfuscation, unintentional or not. So spend the energy to make your way over to your direct report’s desk, meet there, and let he/she set the agenda.
When your work is operating from a shared view of the goals and open and honest communication is encouraged, there’s a very good chance challenges and issues will find you rather than you needing to ferret them out. If you’re finding you are surprised by issues, then dig into the root cause before falling back on asking folks to spend more time telling you what might be happening.
When you take on the role of managing managers, presumably you have found this balance for one level of management. As a new manager of managers, your ability to stay connected to the work is critical but the challenge is much more difficult. You might react by falling into the traps you worked hard to avoid as a new line manager. You might feel that your need to stay connected or to drive strategy trumps the need for a skip level manager to be heard, to be listened to, and to be treated just like you would like to be treated.
Skip-level 1:1s
One-on-ones with your direct reports (fellow managers) will not likely feel like enough to get a sense of the project if you’re in the position of managing other managers. A wonderful tool to learn even more about the team is to just continue having 1:1s on the team, but with the direct reports of your direct reports. Personally, I have always found this part of management to be the very best way to learn more about your own team and an incredibly wise investment to make for a number of reasons.
A skip-level 1:1 is exactly like a 1:1 with a direct report in terms of approach. There is no preparation required. The member of the team is not being called to the carpet. The focus is whatever the member of the team wants it to be. Your role as a manager is to listen, perhaps ask a few guiding questions, and to learn by listening.
In fact, the most important part of a skip-level 1:1 is to avoid “making news” or “solving problems”. If something comes up in the meeting that is a surprise, use this as a chance to ask questions and to make sure the member of the team is engaged in addressing the issue through the management in place. A skip-level is not an opportunity for escalation, no more than it is an opportunity to search for decisions you can make or problems you can solve. If someone leaves a 1:1 thinking you decided something or changed course, then that is a good chance to ask yourself if you were stepping on the toes of your direct report.
You might find it helpful to be systematic with skip level 1:1s and work to meet each of your skip level folks once-per month/quarter/year depending on the size of your organization. If you manage a team of 100, it should be possible to have a skip level with every member of the whole team yearly. More than that and you will probably want to structure your skip-levels to include only your directs of directs, perhaps every six months. It can help to have skip-level meetings take place during specific times in the project when they might make most sense (just before or after a milestone for example). As a suggestion, don’t define “skip level 1:1 days (weeks)” as the assembly line can be a bit off-putting to some, especially if scheduled back to back. You might consider a “head’s up” mail with a skip level invitation just so folks don’t panic :-)
It is important to be consistent in the implementation. In other words, it is important to meet with the same frequency and duration with each of the folks. Don’t short change employees that might be further away, in slightly different projects, or just on areas you might find less in need. The more senior you are the more important this consistency becomes as the team looks for signals of your priorities based on who you spend time with.
My own view is that skip levels are so important that I would routinely spend 15% of scheduled meetings in a year in skip level 1:1s. Some find this a surprising use of time given how much might be going on. The reality is that a consistent approach to meeting with people across a team you manage can offer a unique lens on what is going on. It also affords an opportunity for you to reinforce the work you might have been doing with respect to accountability, decision making, and rhythm of the team.
Some topics
For a skip level 1:1, just as with a regular 1:1, the topics are driven by the attendee and not you. Just as with direct reports, some folks will show up with a long list of topics (or questions). Others will show up assuming you have an agenda. And probably everything in between is possible. Regardless, your role is to facilitate the member of team opening up, to reduce the potential stress of the situation, and to reassure the member of the team that this meeting is not a career moment. This starts with you showing up in the office of the employee, not summoning, the employee to your office.
This latter point is critical. A skip level 1:1 is not a meeting to pass judgment or to evaluate performance, just as it is not a time for “new business”. Folks should know this. While one of you might believe that the meeting should be “confidential”, you should be cautious with that sort of dynamic, not just for skip-levels but in general. Confidentiality is important for matters of personnel but is usually counter-productive when managing a project. If someone requests confidentiality for a personnel issue, it should be handled in the appropriate manner.
If a person shows up with a long list, sometimes it is fine to allow them to work through it. It might also be a good idea to “pace” the dialog and start off by asking “what’s on your mind?” or “how are things going?” In a first 1:1, if you don’t have a history with the member of the team, why not use the time to learn more about each other’s background (as appropriate) and to reciprocate?
There’s likely going to be some interest in hearing answers “from your perspective” – “how are things going”, “did you hear about…”, “what do you think…”? That is interest in topics that are perceived to be talked about at some higher level on the team. It is great to spend some time on those, though it is worth considering how you can put those topics in the context of the project rather than just gossip.
For a larger organization, there’s a benefit to spending time in skip-level dialogs on the efficacy of the work environment. Asking questions about the velocity of code, collaboration, getting things done, and so on. In any organization of size, a manager of managers is where action can (and should) be taken to avoid the perils of a stagnating organization.
Similarly, topics that will surely come up will be related to processes that are corporate wide (compensation, recruiting, and so on). It is probably a good idea to be extra careful about “making news” in how you discuss these—that is different than being guarded and evasive—by focusing on the information previously discussed broadly with the team. You might learn something wasn’t clear, in which case there’s a chance to clear things up for the whole set of folks in a consistent manner.
During different phases of the project, it is great to enable a discussion about that—feedback on pre-release, design challenges, ramping up, and so on.
If you’re the individual
A skip-level 1:1 is a great time to offer your perspectives on what is going well and not. There’s a fine line between offering up all the potentially bad news and sounding like you’re setting expectations, and polishing up all the potentially good news and sounding like you’re showing off. You have to be the judge. A few other potential tips/topics:
- Use right level of detail. Speak the truth and what you know, but match the level of detail that your skip level manager will find valuable. Keep in mind no matter how hands on the manager is, you have a lot more detail in your head :-)
- Ask clarifying questions. Managers often have a tough time with “what is that” or “no I don’t understand” so don’t be afraid to ask questions clarifying if you were clear. Feel free to volunteer to expand acronyms or code names, rather than assume a deep knowledge (without sounding too pedantic).
- Discuss cross-group, collaboration, partnerships. There’s a good chance your skip level manager is pretty tuned in and curious as to how things you are working on contribute to cross-team initiatives. Consider focusing on those topics and if you do, just speak the truth as if your partner is sitting right there with you.
- Limit strategy. There’s a time and place for strategy. While you might be tempted to use the time for big strategy topics, this might not further your goals much and might not be the best use of time. Spend the time thinking about how you could help your contribution to the project with insights and information.
- Don’t make news. It is probably not a good idea to “make news” in this discussion–news about the project, you, or feedback about team/people. You might want to resist the urge to share something for the very first time in this forum and certainly not something you would not share with your manager as if she was sitting right there. If you have candid feedback on your manager, then be clear about what you’re doing and don’t conflate that with the rest of the meeting topics.
Above all, make the most of the time to make sure your skip-level manager is familiar with you and your work in a neutral and constructive way.
When used consistently and effectively, skip-level 1:1s are a great, two-way tool for both of you and the team.
–Steven
###
Dealing with doubt and over-confidence in building something new
When you’re creating a new product or service, whether as a startup or within a big company, you’re going to be faced with doubters from every direction. People on the team, your boss, your peers, your investors, friends, family. Even when the first outsiders see the product they will probably be more doubtful than supportive. The most important thing is to avoid doubting yourself.
If you thought up the idea, got funding or approval to go forward then persevering is a key part of getting the work done. The doubting you’ll most certainly get can feel almost crippling. In the extreme it turns into those increasing moments of self-doubt and ultimately a loss of confidence. That self-doubt can prevent new ideas, new products, from growing to success.
The converse of this behavior is to dig your heels in and to just stubbornly move ahead as though no one has expressed any doubt. You’re getting stuff done. As things progress there’s a good chance you’re increasing the distance between you and your early supporters-but-doubters. That over-confidence can prevent new ideas, new products, from making small but necessary changes that can substantially increase their chances for success.
Finding the right balance between caving in and stubborn is something everyone can work on. There’s no right answer, but you can look for signs or signals that your balance needs some tuning.
5 ways to doubt
Doubts are going to be everywhere in a new venture. In new products there are a few common doubts that you should be mentally prepared to hear. If you know these are coming you can use that as a chance to consider how you are going to engage in a discussion about that sort of doubt—not how you dispense with or handle the doubt, but how you can talk about why you might have a different point of view.
- Been done before. Very few new products are “new to the world inventions”. Even things that are new to the world often solve pretty well-known problems. In reality most all products are incremental when you step back and consider the full context and landscape. From Velcro® to the Swiffer ® to Facebook and Instagram, these products were incredibly innovative but by and large the innovation amounted to new combinations of some new technologies aimed at solving somewhat known problems. You can get yourself in quite a spiral if you think your product needs to be an invention versus an innovation. Thinking about your innovation and value delivered can help you through this.
- Just a feature. In new services in the tech industry we constantly see people saying that a new product is “just a feature”. There’s always some truth to that, but it is because it has to do—as a consumer you don’t want every service that comes along reinventing everything around the social fiber for example and as a company you don’t want to spend resources on work outside your value proposition. Finding the balance between your unique perspective and value and simply adding all the stuff around your value is something to work through and be clear about.
- No one wants that. The focus group of one is both your biggest asset and biggest liability in building a product. If you let one person, from your best friend to your spouse to your boss, convince you that no one wants a new product then too many ideas will fail to make it to fruition. As the person taking the risk to seek funding or get approval for an idea, you owe it to yourself to keep pushing. When the focus group of one is yourself and you’re taking the risk that is the very definition of entrepreneurial thinking. You saw a problem, an opportunity, or a solution. There’s always a time to take a step back but at the early stages a focus group of one that is yourself is pretty important.
- Priced wrong. All new technology products are going to be either too cheap or too expensive. If you’re building a new device, it will always be too expensive in the early stages because the industry is, as we all know, based on economies of scale. A new service or app is always going to struggle to simply charge people or find space for advertising from the start. Too cheap/too expensive is going to happen. Rather than just punt or just restate the known answers (from it will scale to freemium) perhaps you can differentiate your answer to these concerns with some novel or detailed thinking.
- Doesn’t fit with strategy. In a large organization you are, with 100% certainty, going to run up against “strategy” as you propose your new idea. This can be a frustrating experience to a champion of a new idea (or new way to solve a problem). You can throw up your hands in a huff. You can claim “innovator’s dilemma”. You can talk about stifling bureaucracy. The important thing to do during this doubting moment is to be informed about these strategic issues. These are real to a large company because a strategy is a unique part of what a large organization delivers to customers—it is more than a collection of products, but the relationship and between them and reasons they are offered.
5 ways to be over-confident
While many from the outside will be doubting you, the most important thing to do is overcome your natural reaction to dig your heels in and be stubborn. When doubt is expressed it is your chance to engage in a dialog and to calmly evangelize your idea while hearing the doubt as feedback. We all know that when you’re pushing a new idea there are things to do better. This is especially true when you’re in the early stages and developing your “story” as to why the idea should get turned into a product. What are some things you might be over-confident about that the doubters might actually be saying?
- Wasn’t clear. In talking about a potential new product the most common challenge is a problem between your brain and your mouth or keyboard :-) In other words, as crystal clear as the ideas are in your head, when you say them or write them down it just seems like other people are not “getting it”. Your own excitement and enthusiasm, or your own “ah ha” moment, isn’t getting translate into a pitch or description that others can grok. After the third or fourth person saying they are confused or your conversations are “all over the map” then maybe you should take a step back and work on the description and story behind the idea?
- Didn’t study the competition. When folks say an idea isn’t new or has been done before, then it could be that you are not expressing the unique attributes of your idea in a compelling way or perhaps your unique attributes are not unique enough (or valued). It could also be that you’re not expert enough on the competition. Maybe in your excitement you missed a competitor or you dismissed a competitor too quickly? Keep in mind the competition isn’t standing still so maybe things have changed from when you looked?
- Design is weak. Software products often get pitched before the design and flow of usage are understood. For a product that is solving a known problem in a new way, the design is a critical element of what you’re offering. In that case it really isn’t enough to pitch the idea as “don’t worry we’ll do a better design later” because your design is integral to the offering. For any product that is entering a commodity space with a new twist on uniqueness (branding, distribution, pricing, bundling, etc.) the design of those elements (yes pricing can be designed) need to be more than sketches since those aspects are key to what you’re doing. Without that level of detail you might be missing the crux of the doubt.
- Trying to do too much. “Boiling the ocean” is a common refrain when experienced people see an idea for a product that involves touching many known areas of existing products. If you’re service starts off with the need to build out a whole infrastructure before you can even start to show your unique value or if you have a feature list a mile long, then there’s a good chance the doubt is not focused on your idea, but on the scope of what you’re trying to do. Everyone loves big ideas, but rebuilding the world as the pre-requisite is a sign that you can do a better job scoping the first milestone or so of work.
- Clinging to the wrong elements. Many times in talking about an idea and in the early stages, every single choice you make is critical. As the one originating the idea you tend to think of your product as a finely balanced set of decisions, each carefully interrelated. As things progress, you owe it to yourself and doubters to make sure you are revisiting some choices. Do you really need that architecture? Is that UI widget really that important? Is it critical that you have that feature? In most every new product you can see something that you know was an early choice and doesn’t quite fit anymore. Be the person leading the charge to back out of choices that are no longer key to delivering the value proposition. You’re in a unique position to decide what can really go.
Keep in mind
From the moment you think up an idea until the first working models/prototypes are used by potential customers you’re going to run into doubts from all corners. It is easy to lose confidence. It is easy to become over-confident. Balancing these two extremes is an important part of being brave enough to keep pushing forward. New ideas can’t get turned into products without the skills to navigate this complex and emotional stage in product development.
Two things are always part of these early stages and important worth keeping in mind.
First, validation is hard to come by. You will get tons of support and even encouragement from those around you. But validation won’t come for a while. Hang tight.
And second, product development is hard. No one said building a new product or getting a big company to break into a new business (or redo an old business) is easy. There are no right answers. There’s no certainty. Doubts come from shaking things up.
–Steven
###
Balancing a flexible work environment
In what could be considered an extremely bold and thoughtful move, according to reports Yahoo recently announced that employees will be required to work from a Yahoo facility rather than “remote”. As one who has spent time on these challenges, the commentary that followed was arguably predictable. With reactions ranging from tone deaf and archaic to downright anti-motherhood, there seems to be a great deal of pushback or at least feedback. Like so many things in managing a large organization there is no clear cut way to manage through this structural and organizational challenge.
What are some of the considerations in attempting to structure a modern product development team?
Challenge
The key challenge in implementing any policy in a corporation is to balance the needs of the individual, the needs of the team, and the needs of the company and shareholders. As one might expect these needs are not always in alignment at the granular level. Even at the macro level it is not always clear, for example, that everyone comes to work to maximize shareholder value on any given day or that choices each party might make to accomplish that would be aligned.
In balancing these needs, a company also has the obligation to be consistent, and have a view as to why an approach is fair for a set of parties involved. This is about balancing fair across many dimensions—all parts of a company should follow the same basic set of rules/guidelines, rules/guidelines should be the same regardless of your position in the organization or type of role, geography should be implemented consistently (while adhering to local laws as well), and so on. Nothing eats away at an organization as a whole more than the feeling that one part of a company gets a better deal than another part. On the other hand when taken down to an individual level what is consistent is not always viewed as fair by some.
One of the main ways companies tend to deal with controversial or cultural issues is to empower managers throughout an organization to “do the right thing” or as it is called in the military “commander’s intent”. Netflix’s famous expense policy is a supremely good example of this approach (imho). The basic idea with this approach is that deciding at a top-most level is not optimal and so this approach allows for optimal or situational decision making where the only management communication is based on an end-state not the details (“ship great products and have a strong organization”). It also tends to optimize for the least amount of consistency across a large organization and thus potentially causes some amount of underground friction. What is laudable about the Yahoo position is that it is a clear choice from the top of the company, whether you agree with the policy or not you cannot argue that it expresses a clear point of view which is worth noting.
Flexibility
It is extraordinarily difficult to argue against having a flexible workstyle/workforce/team, where flexibility is defined by a whole host of dimensions. A modern product is used by a whole world of people (hundreds of millions) and there is every reason to consider that a product used so broadly should be developed by a team that is representative of the breadth of usage. Flexibility in work location is one dimension and the focus of this post and the Yahoo policy (and this post).
People ask for flexibility for a whole variety of reasons: work better from home, required to live far from an office (across the bay to across the country to across the ocean), special needs more easily serviced at a different location, worked on a project full time and needed to move, spouse/family needs (permanent or part time), or even just a feeling that there’s no need to go into an office. Those are just a few. In fact the list of motivations for working from home/remote is probably at least as long as the number of people on a team who appreciate this arrangement.
Particularly for software projects, and extra-particularly for modern cloud-based products, it seems almost absurd on the face of it not to build the products from a flexible team. In fact most of the products even target the very notion of flexible work environments—so the irony of not following that doctrine to build the product is not lost.
So why all the complexity and challenges?
Ultimately there are a bunch of considerations to take into account, none of which is easily reconciled. The flipside is that all of them can be reconciled in the specific. Therein rests the core challenge faced by a company—what if everyone says they will do the right thing, yet the net result is not the right thing for the company as a whole? What if there are a plethora of examples and counter-examples for a given policy? Everyone who works or supports someone remotely has the very best of intentions. That’s a given. Yet there must be more to this given the changes reported at Yahoo (or policies at other companies) and the dialog.
In an effort to spark some dialog, it seems reasonable to offer some of the challenges that a large organization faces with respect to flexible work. That’s what this post is about, a dialog, and not advocating one point of view or another. In fact I am writing this post living this very challenge personally right now with geographically dispersed commitments and folks willing to support me in those, and I see some of the issues discussed.
I’ve worked on teams which have been geographically split, where people have worked from sites all around the world, and where individuals have had a wide variety of special arrangements. It has never been as easy as “just being modern” and there were always extra work required. And there was always extra benefit as well. Talented people making world class contributions have worked in flexible arrangements on those projects. Some worked temporarily. Some worked permanently. Some set out to work for a short time or permanently and changed paths.
In almost all cases one or more of the following challenges were or became part of the dialog.
Collaboration. Software is a highly collaborative process. To develop software requires collaboration across multiple dimensions. Programmers need to collaborate up and down the stack, across API boundaries, and more. These boundaries evolve rapidly while a system is being developed and the evolution does not often take place through code checkin or over email. Collaboration takes place across disciplines and more often than not those are meetings that take place in person and just as often they are not scheduled. Developers, testers, program managers, operations, designers, and more need to talk frequently in an ad hoc manner. Do you tune the amount of flexible work a team supports based on some measure of how much collaboration the product/project requires? Is it reasonable to assume that the same amount of collaboration can happen between co-located teammates as remote teammates? How much of collaboration is intrinsically based on proximity?
Disciplines. One of the challenges is that different disciplines require different levels of collaboration in a typical project or workday. While every discipline is inherently collaborative, one could (and many do) argue that there are more solitary hours in coding or writing than there might be in design or testing, for example. There are certainly very few solitary hours in being a manager or being a product/program manager. Do you have an approach where some disciplines can have more flexible workstyles than others, for example? How does that feel to the rest of the team when motivations for flexible workstyles arise independent of job function?
Integration / consistency. Customers and reviewers consistently ask for more product integration and better product synergy. By almost all accounts, the “farther apart” members of a project are the more difficult this integration and consistency becomes. You can see this even when it comes to internet discussions about org charts or management structures. The root of this is because consistency and integration come from building highly collaborative give-and-take relationships and those relationships get built and maintained through a great deal of personal contact. Do you tune the amount of flexible workstyles to support based on the measure of integration you want across products or assign people differently (perhaps differently than their skillsets) based on the needs for integration across products?
Ship the org chart. A common phrase used in building software from a large organization is to take care to avoid “shipping the org chart”, which basically means to do what you can to structure a team such that the org chart does not show through in developing the product (a subject of a future post, I promise). Because product development has the potential to evolve differently when people are not in physical proximity there is the potential to mimic what would happen if there was an org design, regardless of the actual line manager. Despite a long history (and requirement) of remote work, even Boeing is reportedly struggling with this aspect of 787 production at the organizational level. When organizing a project, do those members of the team working remotely need to be assigned to projects differently to mitigate the org-distance challenges of remote work?
Turnaround to answers. A big part of a collaborative project is the timeline of asked and answered. Even in the same hallway, you can’t always get an answer from someone (despite the perception, people might not be literally chained to a desk). So the question becomes what is the turnaround time for an answer. If a person is working flexibly 40 hours a week during core working hours, then the expected turnaround should be quick (another source of flexibility is what hours, most every company has some notion of core hours, though the days of at your station by 8:00AM like Intel might be history). On the other hand, some flexible arrangements are 4 days at 10 hours each or even 3 12 hour days (putting aside part time which is another flexible workstyle). The question then becomes one of whether the team is blocked because it is a flex day? Of course this is no different than a person being out sick or on vacation, and some say that it is easier to work around a regular schedule. This challenge extends to teammates in different time zones as well—the further away the less core hours overlap. Should a team be structured with expectations of turnaround, even if it interferes with the stated goals of flexible workstyles?
Shared mindset / point of view. So much of building a software project is about the emotional connection shared by members of a team. The feeling of community, shared goals, and culture are all part of a team. Remote members of a team, by definition will always miss out on elements of this—not just the hallways and meetings, but lunch, voluntary social time, and more. These are just the nature of human beings and how we build community and evolve. As much as we talk about video conferencing, air travel, or just “days on site”, we all know the challenges of just picking up where you left off the last time you saw folks. How does a team continue to develop and evolve a shared point of view with some members of the team not physically present?
Career velocity. If your organization supports flexible work, then it goes without saying that performance appraisal, compensation, promotion, etc. should all be exactly the same whether you are working remotely or not. This is obvious, but not without challenges. For the disciplines that are highly collaborative or projects that are less about individual effort and more about team effort how do you account for the different number of hours spent doing the in-person work these require? Do you structure remote work so it requires less of this type of collaboration? Then do you choose people or projects suite to that? How would you explain the policy regarding approval for remote work?
Peer evaluation. Most employees participate in a peer review process, both offering and receiving feedback. In some sense full-time remote employees can benefit from the most pure form of evaluation in that literally only the work is evaluated. On the other hand, when a large portion of the work involves the process by which choices are made and the way the end-point is reached, the need to participate on equal footing with the team is important to peer feedback. How does peer evaluation work for the “process” or the “how” of the work not just the output?
Management. Managing remote teammates or being managed remotely are both challenging. The more senior you are the less you need (or want!) to interact with your manager, but that is not always a two-way street. Some managers like a high touch team, especially in some disciplines or some phases of a project. Some employees do better work when they can talk with their manager for a few minutes in an ad hoc manner. Newer employees require nearly daily contact/coaching from their manager. What if the manager is not on site or if the employee is not on site? Sometimes “new” does not mean new to the workplace, but just new to the project (or a new project) or just new to the domain. How do you factor in management and being managed into a remote workstyle?
Individual skills. Most any discussion of remote or flexible work hinges on the individual skills of the person working in this style relative to the needs of the team. This makes a ton of sense—any team should be staffed and organized taking into account the capabilities of the individuals on the team. The challenge becomes implementing this in a fair and consistent manner when it comes to remote work. How do you articulate the criteria for being permitted to work remotely? How do you avoid any sort of criteria from being gamed or politicized?
Accountability. Ultimately, any team functions well when accountability is clear and everyone on the team is signed up for their part of the project in terms of delivery, timeliness, and quality. Remote or not, this should not change. But projects do not always go well. When things don’t go well, the team as a whole looks for reasons. One of the darker aspects of remote work is that it becomes a variable that itself gets evaluated in context of work that was not what everyone (including the member of the team) hoped for. How do you account for the variable of remote work when things don’t go well? What happens if you identify working remote as a causal factor? Is it really a causal factor or just a difference? Is the accountability of the manager or the remote employee or the team for not working in a way consistent with the remote employee?
Life changes. Many of the motivations for remote work stem from life changes. Our society is highly mobile these days and so people choose to move to other cities/countries for a whole variety of reasons. Housing prices are crushing in much of the world and so commutes can be awful and wasteful. Spouses also work in most American families and that creates a variety of possible motivations for flexible work. Families grow and the need for one parent to be at home beyond the statutory parental leave is a strong motivator for flexible work. Life happens and perhaps someone needing care moves in with you or you yourself require major life adjustments and flexible work becomes necessary. How do you craft a policy around flexible work that accounts for these life changes that may be temporary or permanent? Given all of the above potential challenges, what is the right way to address the specifics of life? Do these life changes call for a separate policy or approach?
These are some examples. There are many more because there are so many individual circumstances. All of them can be painted in stark terms with obvious answers if that’s your goal. The truth is that at a policy level they can all be dealt with. But at an individual level this always becomes a special case. That’s just the complexity of a large organization. It why there is no easy answer and certainly no right answer. Companies ultimately can choose to differentiate themselves on policies in this regard, as is the case with any workstyle, work culture attribute (dress code, parking, office style), or even compensation or perq.
Proof flexibility works
In any discussion of flexible work, the proof point of large successful open source projects is raised. In some ways this discussion can bubble up to the difference between developing a product in a company and developing a product that by definition and from the start is distributed in nature. It goes without saying that arguing against an existing proven success only adds to the difficulty. Are the challenges of flexible work, some outlined above, due to the very nature of corporate organizations? Or are the successes of the flexible work in open source projects rooted in the very nature of those projects?
There are also whole companies like 37signals or WordPress that are for-profit and foundationally about remote work. What practices do they have that make their experiences work for the team, product, or corporate interests? Are there attributes of the product that make flexible work easier or more supportable? Is that a design or code architecture difference?
Approach
Almost all of challenges described add up to expressing that teams build software. The larger the software the larger the team. The need to have a high performance team to build high performance products is obvious. Building and maintaining a high performance team is an incredibly difficult challenge. On the one hand the team wants the very best talent from wherever and however it can find it. On the other hand keeping the team operating in a holistic manner over time is hard enough as it goes through ups and downs of product development.
Whether you argue that a flexible work environment solves the talent side of the challenge or not is just part of the equation. How to manage the challenge of maintaining the team and product over time is the other side. As with so many product development and management challenges, knowing the challenge you face is a huge part of the work. Committing to address the challenges is critical. Just as critical is deciding where energy on the team is best spent—not every challenge is one the team should take on when faced with finite resources, time, and seemingly infinite business needs.
Product development, including team management, is a social science. That means there are no right answers but just approaches or choices in context.
–Steven
###
Rightsizing your product plan, but not your vision
Creating a product, whether totally new or an update, means deciding what’s in and what’s out. The main execution constraint you have is the time you are willing to spend developing your product (or the number of developers, roughly the same thing). In your planning you need to decide the right amount of work to do to create, or justify, the product—rightsizing your product plan. Executing a rightsized plan without compromising your vision is a core product development skill.
What is rightsizing?
While most all product development debates take place at a fairly granular level—having a specific feature, architecting an investment, choosing how to communicate the work—there are some broad topics that can have a profound impact on how the product evolves. The most critical first step of a project is to decide the “scope”. Deciding the scope of a project is an active conversation across all the stakeholders involved.
For software and service projects (note, project=product=service) the scope determines a whole host of choices, and even how you articulate the scope can open up or foreclose options. You sort of need to start by checking in with the realities of your foundation:
Entirely new product. An entirely new product is the opportunity to scope a product to a minimal set of features or needs. In other words you can literally pick the smallest set of features/investments to express your scenario or goals. It has become common to refer to this as a minimum viable product or MVP. Another aspect of “new” is whether the project is new to your company/organization or new to the industry as a whole. There’s a tendency to view scoping differently if something is entirely new to the world versus new to your organization. An MVP can take on one meaning for a startup where there are no expectations for “minimal”. For an existing company, this becomes increasingly challenging—things like globalization, accessibility, security, integration with existing account infrastructure, and more can set a significantly higher bar for “minimal”.
Evolving an existing product. Most all software is about evolving an existing product. In scoping the work to improve an existing product the main dimensions that determine the scope will be compatibility with the current product—in user experience (keystroke, flow), data (file formats, previously saved work or settings), features (what a product does), or platform (APIs, add-ins). In scoping a product plan for an existing product, deciding up front to maintain 100% of everything itself has a cost, which to the outside world or management, might be counter-intuitive. Regression testing, design constraints, and even what you choose to do differently with existing features determine the scope of the new work for the release. Sometimes a product can be new for the company even if it evolves an existing product, but these same constraints might apply from a competitive perspective.
Disrupting an existing product. Any project can evolve for some period of time and eventually requires a significant overhaul—scenarios change, scale limits reached, user experience ages, and so on. A project that begins knowing you will disrupt an existing product poses a different set of scoping challenges. First and foremost you need to be clear on what part of the project you are disrupting. For example, are you considering a full re-implementation of an existing product or are you re-architecting a portion of an existing product (again, say the UI, API, or features)? Sometimes a product can be new to your organization but disrupt a competitive product, which brings with it a potentially different view of constraints.
Side-by-side product. One type of project scoping is to decide up front that your project will coexist with a product that solves a similar problem from your customers/company/organization. This approach is quite typical for internal IT systems where a new system is brought up and run in parallel with the old system during a switchover period. For a consumer product, side-by-side can be a shorthand for “keep doing it the way you’re doing it but try out our system” and that can apply to a specific set of customers you are targeting early in development.
Each of these is more granular and real-world in an attempt to cover more of the software projects so many of us work on. Typically we look at projects as “new product” or “update” but tends to over-simplify the project’s scope.
Many projects get off to a rocky start if the team is not clear on this one question of scope. Scoping a product is an engineering choice, not simply a way to position the product as it is introduced. For example, you might develop the product as an evolution of a current product but fail to get some of the baseline work done. Attempting to position the product as “totally new” or “just run it side by side” will probably backfire as many of the choices in the code do not reflect that notion–the seams will be readily apparent to customers (and reviewers). As with many challenges in product development, one can look back at the history of projects, both successful and not, and see some patterns to common challenges.
Pitfalls in scoping
Deciding and agreeing up front to the scope of your product is a critical first step. It is also easy to see how contentious this can be and can often generate the visceral and stereotypical reactions from different parts of a collective team.
If you develop an enterprise product and propose something that breaks compatibility you can expect the scoping efforts to be met with an immediate “requirement” that compatibility be added back to the plan from your enterprise sales force, for example.
A consumer product in a space such as note taking or writing, as an example, can certainly be immediately overloaded with the basics of text processing and image handling. Or one can expect reviewers to immediately compare it to the current most popular and mature product. We’re all familiar with the first release of products that received critical reviews for missing some “must have core features” (like copy/paste) even though they had a broadly disruptive first release.
The needs for a product to be global, accessible, or to plug into existing authentication mechanisms are examples that take a great deal of up front work to consider and clarify with the team (and management).
In fact the first pitfall of most scoping efforts might be the idea that disagreements up front, or just different points of view that have been “shelved”, will be easily resolved later on. One coping mechanism is for folks to think that the brilliance of the product’s innovation will be apparent to all parties and thus all the things “out of scope” will go from a disagreement to shared understanding once people see the product. My experience is that this isn’t always how it works out :-)
The most difficult challenge when you’re scoping the project is that you actually considered all of these “obvious” things, yet as people see the product (or plans) for the first time these come across to them like obvious misses or oversights. You probably know that you could add features that exist in the marketplace, that you’re breaking compatibility, that you’re going to need to run side-by-side, that you’re not ready for complex character sets, and so on. Yet as the product is revealed to peers, management, reviewers, or even as the team sees the whole thing coming together there’s always a chance of a bit of panic will set in. If you’ve gone through an effort to plan the scope, then none of this will be news and you will have also prepared yourself to continue forward progress and the discussion for how and why choices were made.
Even with that preparation, there are a few common pitfalls to project rightsizing that one needs to consider as a project goes from planning through execution. These pitfalls can surface as the product comes together and the first customers see it or these can be the reason the product isn’t getting to a stage where others can see it:
- Backing into a different scope. The most critical failure of any project is to think you can revisit the scope of the project in the middle, and still stay on time. If you decide to break compatibility with the existing product and build out new features assuming this approach then you’re faced with rearchitecting the new features, cutting them, or some decidedly tricky middle ground. Taking a step back, what you’re really doing is revisiting the very approach of the whole product. While this is possible, it is not possible to do so on the schedule and with the resources you have.
- Too much. Most all of us have scoped a project to do more than we can get done in the time and with the resources we have. A robust product plan provides a great deal of flexibility to solve this if you were clear on the scope—in other words a feature list that is too long is easy to trim. This is decidedly different than trying to change scope (change from disrupting the product to evolving the product for example). If all you have is too many features, but the intent of the release is consistent with that long list—I promise there are features to cut.
- Too little. In the current climate where MVP is a perfectly good way to develop innovative new products, you can still scope the product to do too little. In the new product space, this could be a sign that you have not yet zeroed in on the innovation or value-add of your product. Similarly, any project that involves a data center deployment (or resources) and a commitment from partners can also be scoped such that the collective investment is more than the collective return on that investment. In the evolution of existing products, such a release might be characterized as simply too conservative. It could also lack focus and just be a little bit of stuff everywhere.
- Wrong stuff. Often overlooked as a potential pitfall of product scoping is a choice to solve the wrong problems. In other words the plan might be solid and achievable, but the up-front efforts scoped the project on the wrong work. This is simply picking wrong. The trap you can fall into is how you cope with this—by simply adding more work or on-the-fly rescoping the product to do more or change scopes. Wrong stuff is a common pitfall for evolving existing products—it is when the scoping efforts lacked a coherent view of priorities.
- Local or global optimization. Scoping a product is essentially an optimization function. For an existing product that is evolving, there is a deliberate choice to pick an area and re-optimize it for a new generation. For a new product, the MVP is a way of choosing a place in the value chain to optimize. This scoping can be “off” and then the question is really whether the adapting that goes on during the project is optimizing the right plan or the wrong plan. This optimization challenge is essentially a downstream reaction to having picked the wrong stuff. You can A/B test or “re-position” the product, but that won’t help if you’re stuck on a part of the value curve that just isn’t all that valuable. Is your optimizing truly about an optimal product or are caught in a trough optimizing something local that is not enough to change the product landscape?
Of course projects go wrong in so many ways, some major and some minor. In fact, part of product development is just dealing with the inevitable problems. Nothing goes smoothly. And just like Apollo 13, when that first glitch happens you can think to yourself “gentlemen, looks like we had our glitch” or you can stay alert knowing that more are on the way. That’s the nature of complex product development.
Approach to rightsizing
Rightsizing your project up front is a way to build in both constraints and flexibility.
Rightsizing is clarifying up front the bounding box of the project. If you’re clear about the mechanical and strategic constraints of a project then you’ve taken the first step to keep things on track and to make sure your commitment to your team, customers, and management to develop a product can be met. One way to think of these constraints is as the key variables for project scoping—you rightsize a project by choosing values for these variables up front.
Mechanical constraints are the pillars of a project from a project management perspective. You can think of these as the budget or the foundation, the starting point:
- People. How many people are going to work on the project? This is the simplest question and the easiest one to fix up front. A good rule of thumb is that a project plan should be made based on the number of people you have from day one. While many projects will add people over time, counting on them to do critical work (especially if the project is not one that lasts years) is almost certain to disappoint. Plus most every project will have some changes in staffing due to natural people transitions, so the best case is to assume new people can fill in for departing folks.
- Time. The next easiest scoping variable is how long your project will last. Whether it is weeks, months, or years you have to pick up front. Proponents of continuously shipping still need to pick how long from the time code is planned and written until that particular code is released to customers in some way—and of course that can’t be done in isolation if multiple groups have code to release. As with people, you can add more time but you don’t get proportionally more work. And as we all know, once the project starts just making things shorter usually fails to meet expectations :-) Many stakeholders will have a point of view on how long the project should last, but this cannot be viewed in isolation relative to what you can get done.
- Code and tools. For any project that is starting from an existing product, one should be deliberate about what code moves forward and what code will be replaced/re-architectected. Starting from an existing product also determines a number of mechanical elements such as tools, languages, and cloud infrastructure. For a new product, picking these up front is an important rightsizing effort and not the sort of choices you can revisit on the fly as often these impact not just the schedule but the expression of features (for example, native v. HTML5 app, or what infrastructure you connect to for authentication). Choosing the code up front will bring in many stakeholders as it impacts the scope of the project relative to compatibility, for example.
While each of these mechanical attributes are relevant to the product strategy they don’t necessarily define the product strategy. Commonly, products talk about release cadence as a strategy but in actuality that is an expression of the mechanical aspects of the project. Strategic constraints are the walls of your project that build on the foundation of your mechanical constraints. Your strategy is how you make choices about what to do or not do for the product. There are a couple of key strategic constraints to address up front:
- Big bets. Every project makes a very small, or even just one, big bet. For an existing product this might be a bet on a new user interface or new business model. For a new product this might be the key IP or brand new scenario. The big bet is the rallying cry for everyone on the team—it is the part everyone is going to sacrifice to make work.
- Customer. Every project needs to start off knowing who will be using the product. Of course that sounds easy, but in a world of scoping a project it means that every potential customer cannot be served by a project to 100% of their needs or wants. Knowing how you are delivering value to the relevant customers is a key rightsizing effort. If you’re building on an existing product and breaking with the past or building a new product, then by definition some folks will not see the product as one that meets their needs. It does not mean you will never meet their needs nor does it mean every customer like that will see things the same way.
- Long term. When rightsizing a project you want to know where you are heading. There are many ways to do this—some very heavy and some very lightweight. The context of your business determines how much effort to put into this work. If you know where you are heading over time, not just one release, then you can connect the dots from what is missing today to where you will be after one or more turns of the crank. A long term discussion is not the same as long term planning. Long term planning is a heavyweight way of making commitments you probably can’t deliver on—we all know how much changes in every aspect of the team, market, business, etc. But long term discussion allows everyone to get comfortable that “thinking” is happening. One way to think of this tool is to make sure the dialog is what the team is thinking about, not what the team is doing, so that the long term dialog does not morph into long term commitments.
The first step in building a product plan is to scope the product—rightsizing. It is common to fall into extremes of this step—being extremely minimal or being too broad. In practice, the context of your business contributes to determining what viable alternatives to rightsizing are. There are tools you can use to actively rightsize the project rather than to let the size of the project just sort of happen. Rightsizing the current project with a longer term view as to where you are heading allows projects to be scoped without compromising your vision. As with any aspect of product development, being prepared and knowing that these challenges are in front of you is the best way to manage them.
–Steven
###
Debating product, feature, or company
A cool new app or service releases and immediately the debates begin over the magnitude of the offering. Is it a product, feature, or company? Within a short time, pundits declare something to be too small, too easy to copy, or too inconsequential to merit the startup label. It will be swallowed by one of the bigger platform players. This strikes me as missing some salient elements of product design, even if it is true sometimes. Context matters.
A new mail client for iOS released this week. The first wave of buzz was pretty high and reviews very solid in our circles. Then a writer declares it to be over for them when “Google writes the few lines of code necessary to provide that functionality to my existing proprietary Gmail app”.
The debate over whether something is an app, a feature, or even a whole company goes as far back as most can remember. I remember having to buy a spelling checker separate from the word processor I used about the same time I had to buy graphing software separate from a spreadsheet. There were many examples before that and certain many after that.
The richer our software becomes the higher the expectations are for the rich set of features a product must have when it is first released, and the depth of offerings a company must have for its first product. These expectations are based on some pretty solid consumer notions of switching costs, barrier to entry, and so on. In other words, a new product should probably do more than an old product if it is to replace what is currently in use. Similarly, a new business should have to put in some effort to replace a product if what is being replaced is worthwhile at all.
New products can be incremental improvements over existing products. When released, such a product battles it out within the context of the existing market (one the company is already competing within or maybe a new company with a clear idea of how to improve). A new product can also take an entirely new, revolutionary, view of a problem space and change the evolutionary path we all thought we were on. Most new products get started because folks believe that there is a new approach. New products can also be new companies or come from innovations within existing companies.
A new approach has many potential elements – new user experience, new factoring of the code (app v. browser), new view of scenarios. New products can also be additive—they can do things that existing products do, but add a new twist. This new twist can be an intellectually deep approach to solving a problem or it can be a clever solution. In all cases, a new product is almost always a combination of many previously existing technologies and approaches, brought together in a unique way.
Before writing off a product as a feature or non-company, we should consider the reality that all new products are mostly new combination of old features and that most companies release a first product that has familiarity with something already in the market. The current market climate, changing dynamics of consumers (or businesses), and every-so-slightly-new approaches can make a huge difference.
What are some things that are worth considering before being so flip as to write something off the week it is released:
Forward-looking / disruptive. Many times a new product is ahead of its time. When it is released it is missing features relative to entrenched products, and so to fully “see” the product you need to allow some time for the product to mature. Our landscape is littered with products that came out deficient in remarkable ways, but the feature combination plus the trajectory of the product more than made up for those deficiencies. This is the very definition of disruptive innovation so many people like to talk about so quickly.
Focus. Often new products (and companies) arise because the entrenched products are not as interested in or focused on a scenario, even if there is some work. Quite often existing businesses have an agenda where a given product is not the true focus of the business, but there are some minimal needs that drive offering something. Many people are quick to say that photo editing that removes red-eye, crops, and auto-adjusts is all most people need and so it is natural for the device to offer this “90% case” of features. Yet app stores are filled with photo editing products. Reviewers debate the merits of filters, sharing, and advanced editing features in each. These whole companies are by definition building features that can be easily subsumed yet they continue to thrive.
Price, place, promotion. It is natural for product reviews to ignore the reality of running a business and to focus simply on the features. In practice, starting a company is also about the full spectrum or offering a product—yes it is the product, but it is also the price, place, and promotion of the traditional marketing mix. Innovation can take place in any of these. This innovation could prove to be highly valued by consumers or create the basis for a partnership between a new company and an entrenched/larger company. How many companies distribute news, weather, sports information but do so uniquely because of business, partnerships or other non-product features? Often incremental products innovate in these elements just as much as they do in product features.
Moat. Warren Buffet has famously said “I look for economic castles protected by unbreachable ‘moats’.” The importance of moats is not changed by app stores, mobile, or social, but it does offer a potential to create a new kind of moat that goes beyond the relatively simple view of product features or large companies with the ability to add more code to their existing offerings. A new product might look a whole lot like an existing product but it might create a vibrant new community of consumers or attract the interest of developers to extend the product because of a creative platform, just to name two possible moats.
All of these and more can change the context in which a new product is released. What’s super interesting is how the context–consumer needs/desires, platforms, and so on–can change the view of the importance of any of these attributes over time. What seems important today might not be tomorrow, and vice versa. The elements that create moats or determine the viability of a product change over time, even if the features look pretty close to the same.
There is no doubt that the app that prompted this post has caught the interest of a lot of folks as hundreds of thousands are using it and at least that many are waiting. There’s also no doubt that building a new mail program is a challenge.
At the same time, the very existence of this effort – whether it is labeled a company, product, or feature – shows someone has the passion and desire to improve the state of the art. Someone (and their backers) saw the opportunity to take on the establishment and challenge the notion that the envelope was being pushed or the entrenched folks were simply not making the most of the opportunity they have.
It is more difficult than ever to release a new product to market. Expectations are high. Word spreads fast. Large players are numerous. Let’s all keep in mind that every product we currently use was once a new product that probably failed more than a few first looks.
–Steven
###
Focusing on the work, not the methodology
Want to get developers fired up? Kick off a debate about development methodologies – waterfall, agile, lean, extreme, spiral, unified, etc. At any given time it seems one method is the right one to use and the other methods, regardless of previous experience, are wrong. Some talk about having a toolbox of methods to draw on. Others say everyone must adapt to a new state of the art at each generation. Is there a practical way to build good software without first having this debate?
There’s a short answer. Do what feels right until it stops working for the team as a whole, and to do so without debating the issue to death, then iterate on your process in your context. The clock is running all the time and so debating a meta-topic that lacks a right answer isn’t the best use of time. There’s no right methodology any more than there is a right coding convention, right programming language, or right user interface design. Context matters.
We’re all quick to talk about experimentation with code (or on customers) but really the best thing to do is make some quick choices about how to build the software you need and run with it. Tune the methods when you have the project results in the context of your team and business to guide the changes. Don’t spend a ton of time up front on the meta-debate about how to do the work you need to do, since all you’re doing is burning clock time.
Methods
Building a software product with more than a couple of people over anything longer than a few months requires some notion of how to coordinate, communicate, and decide. When you’re working by yourself or on a small enough codebase you can just start typing and what happens is pretty much what you would expect to happen.
Add more people or more time (essentially the same thing) or both and a project quickly changes character. The left hand does not know what the right hand is doing or more importantly when. The front and back end are not lining up. The user experience starts to lack a consistency and coherency. Fundamentals such as performance, scale, security, privacy are implemented in an uneven fashion. These are just a natural outcome of projects scaling.
Going as far back as the earliest software projects, practitioners of the art created more formal methods to specify and implement software and become more engineers. Drawing from more mature engineering processes, these methods were greatly influenced by the physical nature of large scale engineering projects like building planes, bridges or the computers themselves.
As software evolved it became clear to many that the soft part of software engineering could potentially support a much looser approach to engineering. This coincided with a desire for more software sooner. Many are familiar with the thesis put forth by Marc Andreessen Why Software is Eating the World (if not, worth a read). With such an incredible demand for software it is no surprise that many would like projects to get more done in less time and look to a methodological approach to doing so. The opportunity for reward for building great software as part of a great business is better than ever. The flexibility of software is a gift engineers in other disciplines would love to have, so there’s every reason to pivot software development around flexibility rather than rigidity.
Definitions
For this post let’s just narrow the focus to the ends of the spectrum we see in this dialog, though not necessarily in practice. We’ll call the ends of the spectrum agile and waterfall.
In today’s context, waterfall methods are almost always frowned upon. Waterfall implies slow, bureaucratic, micro-managed, incremental, removed from customers and markets, and more. In essence, it feels like if you want to insult a product development effort then just label it with a waterfall approach.
Conversely, agile methods are almost always the positive way to start a project. Agile implies fast, creative, energetic, disruptive, in-touch with customers and markets, and more. In essence, if you want to praise a product development effort then just label it with an agile approach.
These are the stereotypical ends of a spectrum. Anything structured and slow tilts towards waterfall and anything creative and fast tilts towards agile. At each end there are those that espouse the specifics of how to implement a method. These implementations include everything from the templates for documents, meeting agendas, roles and responsibilities, and often include specific software tools to support the workflow.
It is worth for a moment looking in a mirror and stereotyping the negative view of agile and the positive view of waterfall, just for completeness. A waterfall project is thoughtful, architectural, planful, and proceeds from planning to execution to completion like a ballet. On the other hand, agile projects can substitute chaos and activity for any form of progress—“we ship every week” even if it doesn’t move customers forward and might move them backwards.
Of course these extremes are not universally or necessarily true. Even defining methodologies can turn into a time sink, so it is better to focus on reality of getting code written, tested, deployed/shipped and not debugging the process for doing so…prematurely.
Reality
In practice, no team maintains the character of the ends of this spectrum for very long, if ever. There is no such thing as a pure waterfall project any more than there is a pure agile project. In reality, projects have characteristics of both of these endpoints. The larger or longer a project is the more it becomes a hybrid of both. Embracing this reality is far better than focusing finite work energy on trying to be a pure expression of a methodology.
In discussions with a bunch of different folks recently I’ve been struck by the zeal at which people describe their projects and offer a contrast (often pointing to me to talk about waterfall projects!). One person’s A/B test is another person’s unfinished code in hands of customers. One person’s architecture for the future is another’s slogging plan. The challenge with the dialog is how it positions something everyone needs to do as a negative relative to the approach being taken. Does anyone think you would release code without testing it with a sample of real people? Does anyone really think that doing something quickly means you always have a poor architecture (or conversely that taking a long time ensures a good architecture)?
We all know the reality is much more subtle. In fact, like so many product development challenges context is everything. In the case of development methodologies the context has a number of important variables, for example some include:
- Skills of team. Your team might be all seasoned and experienced people working in familiar territory. You might be doing your n-th project together or this might be the n-th time building a similar project or update. You know how hand-offs between team members work. You know how to write code that doesn’t break the project. Fundamentals such as scale, perf, quality are second nature.
- Number of engineers / size of team. The more people on a project the more deliberate you will likely need to be. With a larger team you are going to spend time up front—measure twice, cut once. The most expensive and time consuming way to build something in software is to keep rebuilding something—lots of motion, no progress. The size of the team is likely to influence the way the development process evolves. On a larger team, more “tradition” or “convention” will likely be in place to begin with. Questioning that is good, but also keeping in mind the context of the team is important. It might feel like a lot of constraints are in place to a newcomer, especially relative to a smaller previous team. The perspective of top, middle, bottom plays into this–it is likely tops and middles think there is never “enough” and bottoms think “too much”. On a large team there is likely much less uniformity than any member of the team might think, and this diversity is an important part of large teams.
- Certainty of project. You might be working on a project that breaks new ground but in a way that is comfortable for everyone. This type of product is new to the organization but not new to the industry (for example, a new online store for a company). Projects like this have a higher degree of certainty about them than when building something new to the industry.
- Size of code base. Even with a small number of people, if you happen to have a large code base then you probably need to spend time understanding the implications of changes. In an existing body of code, research has consistently shown about a 10% regression rate every time you make changes. And keep in mind that “existing body of code” doesn’t have to mean decade’s old legacy code. It could just mean the brand new code the 5 of you wrote over the past six months but still need to work around.
- Familiarity of domain. The team might have a very high degree of familiarity with a domain. In Dreaming in Code the team was very agile but struggling until they brought in some expertise to assist with the database portion of the project. This person didn’t require the up-front planning one might think, because of the domain expertise they just dove right in building the require database. But note, the project had a ton of challenges and so it isn’t clear this was the best plan as discussed in the book.
- Scale services. Scaling out an existing service or bringing up a new service for the first time is a big deal. Whether you’re hosting it yourself or using a service platform, the methods you use to get to that first real world use might be different than those you would use for an app delivered through a store. The interactions between users, security and privacy, reliability, and so on all have much different profiles when centralized. Since most new offerings are a combination of apps and services, it is likely the methods will have some differences as well.
- Realities of business. Ultimately there is a business goal to a project. It is easy to say “time to market is everything” and many projects often do. But there is a balance to competitive dynamics (needing features) and quality goals (no one needs a buggy product) that really dictate what tolerances the marketplace will have for varying levels of quality and features one delivers. Even the most basic assumption of “continuous updating” needs to be reconciled with business goals (something as straight forward as how to charge for updates to a paid app can have significant business implications for a new company).
The balance with all of these attributes is that they can all be used to justify any methodology. If the team is junior then maybe more planning is in order. If the project feels breakthrough then more agility is in order. Yet as is common with social science, one can easily confuse correlation with causality. The iPad is well-known to be almost a generation in the making (having started years before the iPhone then shelved). Facebook is well-known to favor short cycles to get features into testing. In neither case is it totally clear that one can claim the success of the overall effort is due to the methodology (causal v. correlation). Rather what makes more sense is to say that within the context of those efforts the methodology is what works for the organization.
We don’t often read about the methodology for projects that do well but not spectacular. We do read about projects that don’t do well and often we draw a causal relationship between that lack of success and the development methodology. To me that isn’t quite right—products succeed or fail for a host of reasons. When a product doesn’t do what the marketplace deems successful, the sequence of steps to build the product are probably pretty low down on the list of causal factors—quality, features, positioning, pricing, and more seem more important.
Many assert that you simply get more done using agile methods (or said another way, waterfall methods get less done) in a period of time, and given the social science nature of our work it is challenging to turn this assertion into a testable hypothesis. We don’t build the same product twice (say the same way that carpenters know how to frame a house on schedule). We can’t really measure “amount of work” especially when there can be so much under the hood in software that might not be visible or material for another year or two.
The reality of any project that goes to customers is that a certain amount of work needs to get done. The methodology dictates the ordering of the work, but doesn’t change the amount of work. The code needs to be written and tested. The product needs to work at scale, reliable, secure, accessible, localizable, and more. The user experience needs to meet the design goals for the product. Over time the product needs to maintain compatibility relative to expectations—add-ins and customizations need to be maintained for example.
All of this needs to happen. All of this requires elements of planning, iteration, prototyping, even trial and error. It also requires a lot of engineering effort in terms of architecture and design. On any project of scale, some elements of the project are managed with the work detailed up front. Some elements are best iterated on during the course of the project. And some can wait until the end where rapid change is not a costly endeavor.
There’s no magic to be had, unfortunately. You can always do less work, but it doesn’t always take less time once the project starts. You can’t do more work in less time, even if you add more resources. You can sacrifice quality, polish, details, and more and maybe save some time. You can plan for a long time but it doesn’t change the amount of time to engineer or the value of real people using the product in real situations. You can count of fixing things after the product is available to customers, but that doesn’t change the reality of never getting a second chance to make a first impression. All of this is why product development is so fun—there simply aren’t magic answers.
There are ways to be deliberate in how you approach the challenge.
Checkpoints drive changes
Rather than advocate for a conversion of the whole team to a methodology or debate the best way to do the work, the best bet seems to always be much more organic. On a small project, ask what would work for each of the team members and just do that. On a large project, put some bounding principles in place and some supporting tools but manage by results not the process as best you can.
In all cases, the team should establish a rhythm of checkpoints that let everyone know where the project stands relative to goals. Define a date along with criteria for the project at that date and then do an assessment. This assessment is honest and transparent. If things are working then great, just keep going. If things are not working then that’s a good time to change. The only failure point you can’t deal with on a project is a failure to be honest as a contributor and honest as a team. If the dynamic on the team is such that it is better to gloss over or even hide the truth then no methodology will ever yield the results the team needs.
In projects I’ve worked on that have spanned from 3 months to 3 years, the only constants relative to methods are:
- Establish the goals of the project, the plan, in such a way that everyone knows the target—execute knowing where you want to end up and adapt when you’re not getting there or you want to adjust the target.
- Create a project schedule with milestones that have measureable criteria and honestly assess things relative to the criteria at each of those milestones.
The bigger the project the more structure you put on what is going on when, but ultimately on even the biggest projects one will find an incredible diversity of “methods”. At least two patterns emerge consistently: (a) that scale services and core “kernel” efforts require much more up front planning relative to other parts of a project and failure to take that into account is extraordinarily difficult to fix later and (b) when done right, user experience implementation benefits enormously from late stage iteration.
Approach
Given all that is going on a project what’s the right way to approach tuning a methodology?
Very few software projects are on schedule from start to finish, and even defining “on schedule” can be a challenge. Is a 10% error rate acceptable and does that count as on time? If the product is great but was 25% late, time will probably forget how late it was. If the product was not great but on time, few will remember being on time.
So more important than being on time is being under control. What that means is not as much about hitting 4pm on June 17th, but knowing at any given time that you’re going to be on a predictable path to project completion. Regardless of methodology, projects do reach a completion date—there’s an announcement and people start using what you built. It might be that you’re ready to start fixing things right away but from a customer perspective that first new experience counts as “completing the project”. Some say in the agile era products are never done or always changing. I might suggest that whenever new codepaths are executed by customers the project as experienced some form of completion milestone (remember the engineers releasing the code are acting like they “finished” something).
The approach of defining a milestone and checkpoints against that milestone is the time to look at the methodology. Every project, from the most extreme waterfall to the highest velocity agile project will need to adjust. Methodologies should not constrain whether to adjust or not as all require some information upon which to base changes to the plan, changes to how people work, or changes to the team.
Checkpoints should be a lightweight self-assessment relative to the plan. When the team talks about where they are the real question is not how are things going relative to a methodology but how things are going relative to the goal of the project. The focus on metrics is about the code, not about the path to the code. Rather than a dialog about the process, the dialog is about how much code got done and how well is it working and connecting to the rest of the project.
That said there are a few tell-tale signs that the process is not working and could be the source of challenges:
- Unpredictable. Some efforts become unpredictable. A team says they are going to be done on Friday and miss the date or the team says a feature is working but it isn’t. Unpredictability is often a sign that the work is not fully understood—that the upfront planning was not adequate to begin the task. What contributes to unpredictability is a two-steps forward, one-step back rhythm. Almost always the answer to unpredictability is the need to slow down before speeding up.
- Lots of foundation, not a lot of feature. There’s an old adage that great programmers spend 90% of the time on 10% of the problem (I once interviewed a student who wrote a compiler in a semester project but spent 11 of 12 weeks on the lexical phase). You can overplan the foundation of a project and fail to leave time for the whole point of the foundation. The checkpoint is a great time to take a break and make sure that there is breadth not just depth progress. The luxury of time can often yield more foundation than the project needs.
- Partnerships not coming together. In a project where two different teams are converging on a single goal, the checkpoint is the right time to sanity check their convergence. Since everyone is over-booked you want to make sure that the changes happening on both sides of a partnership are being properly thought through and communicated. It is easy for teams that are heads down to optimize locally and leave another team in a tough spot.
- Unable to make changes. In any project changes need to be made. Surprisingly both ends of the methodology spectrum can make changes difficult. Teams moving at a high velocity have a lot of balls in the air so every new change gets tougher to juggle. Teams that have done a lot of up front work have challenges making changes without going through that process. In any case, if changes need to be made the rigidity of a methodology should not be the obstacle.
- Challenging user experience. User interface is what most people see and judge a product by. It is sometimes very difficult to separate out whether the UI is just not well done from a UI that does not fit well together.
- Throwing out code. If you find you’re throwing out a lot of code you probably want to step back—it might be good or it might be a sign that some better alignment is needed. We’re all aware that the neat part of software is the rapid pace at which you can start, start over, iterate, and so on. At some point this “activity” fails to yield “progress”. If you find all or parts of your project are throwing out more code, particularly in the same part/area of a project then it is a good time to check the methodology. Are the goals clear? Is there enough knowledge of the outcome or constraints?
- Missing the market. The biggest criticism of any “long” project schedule is the potential to miss the market. You might be heads down executing when all of a sudden things change relative to the competition or a new product entry. You can also be caught iterating rapidly in one direction and find competition in another. The methodology used doesn’t prevent either case but a checkpoint offers you a chance to course correct.
It is common and maybe too easy to get into a debate about methodology at the start of a project or when challenges arise. One approach to shy away from debates about characterizing how the work should be done is to just get some work done and iterate on the process based on observations about what is going wrong. In other words, treat the development process much like the goal of learning how to improve the product in the market place through feedback. A time to do this is at deliberate checkpoints in the evolution of the process.
So much of how a product evolves in development is based on the context—the people, the goals, the technologies—using that to your advantage by knowing there is not a high degree of precision can really help you to stay sane in the debates over methods.
This post was challenging to write. It is such a common discussion that comes up and people seem to be certain of both what needs to be done and what doesn’t work. What are some experiences folks have had in implementing a methodology? What are the signs you’ve experienced when regardless of the methodology the project is not going in the right direction? Any ideas for how to turn checkpoints into checking up on the project, not revisiting all the choices? (ok that last one is a favorite of mine and a topic for a future post).
–Steven
###
Embracing ideas is how new things happen
Ideas are everywhere. Embrace them. Criticizing an idea is a sport on the web—snarking. Sometimes the very web that craves innovation and newness steps on the dialog with unrelenting and utterly irrelevant snark. In practice, turning ideas into products is the amazing, miraculous, thing that is product development.
Products get developed pulling from the same ideas, tools, and even skills available to a broad set of people. Mere inches separate just products from breakthrough products. Part of a breakthrough product is about embracing new paradigms and those new paradigms should be debated and discussed, but most of all embraced. It takes a brave person to question the existing and put forth new paradigms.
Paradigm to dialog
In a recent op-ed piece a concept was put forth, a potential new metaphor for computing by the highly regarded and widely published David Gelertner. It was a thought-provoking essay. I think that is what the author intended.
In fact the essay argued for a new metaphor by establishing the importance of metaphors in the evolution of computing. Metaphors are how we interact with computing—from two digit numeric displays, to line printers, to command lines, to GUI, to the web, and now social and mobile. Beyond that, the details of these metaphors matter greatly and are worthy of much dialog and debate.
Regretfully as we have all come to expect, the comments on the post quickly devolved into meaningless snark and commentary unworthy of the packets and storage they consumed. There were even tweets about the commentary. What was an essay that could provide the foundation for an interesting dialog among those that build products became, in part, a snarkfest.
Some claimed the idea was not new. Google reader is in fact the worldstream. Some claimed that we live in a stream every day with Facebook and twitter. Some said the idea won’t work.
My read is that this was an essay about metaphor, not a spec and certainly not an app you can try out and comment on. The very “desktop” I am writing this on was envisioned in an essay in 1945, As We May Think, by then head of Office of Scientific Research. Let’s not forget that the web itself once started off as an essay on a metaphor of hyperlinking by Ted Nelson in 1965 which was not implemented until 1967 or until 1993 depending on how you count.
Would these essays have been the subject of such snark? Maybe they were and history has rightfully forgotten those expressions. We should all be clear that essays like these are what change the face of computing.
Read them. Join the dialog. Ignore them if you want. They are not all great or even good. But to criticize them in a content-free manner or to debate them as though they are product specs misses the point completely.
Dialog to product
Taking the dialog around a paradigm shift and turning into a product is magical. Few people can or do make this happen. It is very hard.
A product of a new paradigm is by definition different, but not every aspect is different in every way. In fact the most amazing part about a paradigm shifting product is how much of a derivative work it turns out to be.
Every new product and every new paradigm product are the result of taking the ingredients of the world and pouring those into the primordial soup of product development. Out comes a new product. It is never exactly the same as what came before.
What separates one plain old new product from one that changes our view of technology is a set of choices separated my mere inches. What’s the difference between Facebook and Myspace or Friendster (or a dozen other services)? Were there touch phones before the iPhone? MS-DOS was one of many command line program loaders.
I stumbled across this view of turning an idea into a feature in an app. I love how it characterizes the amount of work and number of choices it takes to do so. How Software is Built Today.
Taking an idea and turning it into a product is incredibly fun and challenging.
Picking the ideas to push to a product takes a lot of guts and sometimes a leap of faith.
Essays that challenge the status quo and push us to think about our world differently are the very source of breakthroughs that we all want.
Embracing that dialog is the start of paradigm shifting products.
Super Sunday
Today is a big football game. As Al Paccino’s character, Tony D’Amato said in Any Given Sunday one of the all-time great locker room speeches in film:
You find out life’s this game of inches, so is football. Because in either game – life or football – the margin for error is so small. I mean, one half a step too late or too early and you don’t quite make it. One half second too slow, too fast and you don’t quite catch it. The inches we need are everywhere around us. They’re in every break of the game, every minute, every second. On this team we fight for that inch. On this team we tear ourselves and everyone else around us to pieces for that inch. We claw with our fingernails for that inch. Because we know when add up all those inches, that’s gonna make the f***ing difference between winning and losing! Between living and dying!
Ideas are everywhere. Ingredients for new products are everywhere. It is mere inches that separate run of the mill from great to paradigm shifting. The one certain thing is that if you build products you should embrace ideas wherever they come from.
–Steven
PS: This blog is also available on http://snarkfree.com
Balancing tradeoffs across different customers
Balancing the needs of different types of customers within a single product is an incredibly difficult challenge in product design. Most every product faces this in the course of choosing features or implementation. Designing products in a changing world with changing definitions of success can be a real challenge, but there are a number of creative approaches that can be used.
Tradeoffs across different customers
I was lucky enough this recently to spend some time with the CEO of a growing company (> 150 developers). The company faces a constant struggle in their product line over how to balance the feature demands of end users “versus” IT professionals. This is an especially acute challenge in a growing company where resources are limited and winning those early paying customers is critical.
The use of “versus” is intentional. Most of the time we view these trade-offs as binary, either/or. That’s the nature of the engineering view of a challenge like this. In stark contrast, the sales/marketing view is often an “and” where the most desired end-state is to meet the needs of every type of customer. In practice, the reality of what needs to be done and what can be done is much more subtle.
Because IT pro and end-users are often viewed as working against each other, this is natural (By the way, I’ve never been a fan of the term user or end-user – a wise program manager I had the pleasure of working with once pointed out “only one other industry refers to customers as users, let’s not follow their lead.”) IT Pros think end-users seem to exist to cause information leaks and network slow-downs. End-users, let’s just call them people or humans, think IT is there to prevent any work or progress from happening. Again, reality is less extreme.
But we face many tradeoffs in developing feature lists, and thus product plans, all the time. In fact for just about any software product/service these days you can easily list a variety of customer types:
- Humans. These are the broad set of people who will use your product. Generally you don’t assume any extreme level product usage skills. These are typical customers. Of course all other customers are human too :-) The challenge each faces is representing their constituency and role beyond that of typical customer.
- IT Pros. For most tech products, IT Pros are the folks that deploy, purchase, or manage the product. They might also provide infrastructure and hardware required to use a product or service. IT Pros also champion the people in the organizations they support.
- Developers. Developers contribute add-ins or consume APIs to develop customized solutions. For many products, developers form a critical part of the ecosystem and often create the stickiness associated with a successful product.
- Power users/enthusiasts. These folks know the ins and outs of a product. Often they teach others to use a product, staff the newsgroups or self-help forums, write blogs and articles about your product. These are your fans. Power users also have feature requests (demands!) for more control, more customization and so on. Notice right away how this could work against IT Pros who want less of those or Humans who might be perplexed by such features.
- Channel partners. Many products are sold direct to humans or IT. On the other hand there are a large number of products where there are intermediate partners who are required to sell, service, or otherwise transact with paying customers. In an ad-supported product these are your advertisers. This provides another tension point our industry commonly talks about—the aspects of ads in apps and web sites which are important to the channel but might not be valued as much by Humans, as an example.
- Markets. Many products aim to meet the needs of a global customer base. Even in a global economy, there are major differences in features and scenarios around the world.
Not every product has every potential customer type and above is not a complete list. Often there is overlap; for example, most IT Pros are often enthusiasts and/or power users. The term “persona” often gets used to represent a customer type. That is a good tool, but rather than focus on the details of the person, just defining a broad category is enough when planning and scoping the product. The persona comes later in the process.
In industries defined by a combination of physical goods and physical distribution channels, products are segmented and offered with different attributes at different prices for different customers. Software and hardware products that intermix, or don’t distinguish between, both work and personal life, and often switch between those many times throughout a day, pose a special challenge to product designers (and marketers). Working within this consumerization trend motivated this post.
Relative to designing for such a scenario, a product plan might find itself in a tough spot because of challenges in the plan or approach:
- Focus nearly exclusively on one target customer type. Sometimes the approach is to just draw a line in the sand and say “we’re all about end-users”. Often this is the default most products take. In some products you can see a clear view of who the target is and a clear strategy. There are then “some features” aimed at appeasing other potential customer types. You might rationalize this by combining customer types, by saying something like “Developers are just humans who can code” for example.
- Do a little bit everywhere. There might be a case where a product is not quite deliberate and the organization or up front resource allocations end up dictating how much each type of customer gets. For example if you allocate a few developers to each segment then the let folks plan independently, the chances of features holding together well are reduced. More likely, features might end up competing or conflicting as they are developed.
- Have a plan (and some execution) and then realize late in the process you’re missing a customer. We have talked about the need to bridge the engineering and marketing efforts. In some plans there are engineering plans that don’t get buy off and as the product starts to come together the inevitable panic of “there are no features for X” where X is a customer type not receiving enough love. Meetings. Panic. Last minute changes. Doesn’t work.
The key to resolving tradeoffs is to know you’re making them up front. Product development is inherently about tradeoffs in many dimensions—in fact product development can be viewed as a series of tradeoffs and the choices made relative to those tradeoffs.
Planning to make choices
A recurring theme with this blog will be to surface issues while you are planning and to acknowledge up front that the plan is going to make choices about what to do. Rather than think of this as a micro-management waterfall approach, the team needs to arrive at principles that guide decision making every day for the team. Principles and plans work better than budgets, organizations, or requirements—principles are what smart and creative people can use effectively as a tool to address the tradeoffs inherent in product development. Tools like budgets and requirements are more like weapons people can use against other parts of the team to prevent work from happening. Principles tell the team the starting point and the end point and offer guidelines for how to make choices as the path to the end point is developed.
As an example, consider a budget that you establish that says how many resources will go to Humans and how many will go to IT Pros. Sounds great on paper. It seems like the easy way up front to decide how to address the conflict or tension.
This can backfire because it is not a holistic plan for what the product will be. In fact is almost prevents the holistic plan from happening because the first choice you make is to partition resources. The leaders of those resources are then incented to just make a plan for their efforts rather than think about the whole of the project. That isn’t evil or malicious, just a natural outcome of resource allocation and accountability.
This same dynamic might occur if you partition the team into front end/back end, or UI/service for example. Such a structure is fine, but should be done in the context of a holistic plan. Putting in place such a structure before there is a plan and hoping the plan resolves difficult problems can be difficult.
Conflict and tension are actually created by a resource budgeting approach as people naturally defer choices and decision to the management and resource allocation tools, not a collaborative product plan.
Approach
Building on the previous post about developing a framework, one can use these same tools as a way of cross-checking the plan against customer types.
We talked about a tool where you identify feature ideas and assign costs as a way to arrive at a holistic view of the product. This tool can be used with different folks in the org or even customers/partners.
Once you have arrived at a list you can take this a step further. A good idea is to refine the list of proposed ideas with your own knowledge of feature descriptions and granularity. What is helpful at this point is to develop a catalog of features that you feel could be effectively communicated to customers broadly.
For example for a very large product these would be the sessions you might give at a customer workshop describing the product. For a first generation product these features would be the product information page on a web site or even the table of contents of the product overview document you might give to a member of the press.
The way to gain an understanding of the tradeoffs you are making relative to customer types is to take the features and align them with the different customer types you have identified. This can easily be done by a spreadsheet or even a list on a whiteboard.
When you’re done every feature is listed once – you don’t give yourself credit in more than one place for a feature. This forces you to really decide who values a feature the most. Features will naturally fall into place. For example, management features will fall to IT Pros or ease of use to end-users.
Most products will show the inherent “tilt” towards a customer type during this phase. You then step back as a team and ask yourself if you’ve made the right tradeoffs or not.
Then iterate and make sure you are really delivering the holistic plan. Once you get closer to a holistic plan you can allocate resources. Iteration doesn’t stop there of course. You can move forward with a more refined view of the plan and the resources. Implementation then progresses based on the resources at hand, which is better than ideation and planning based on resources. Unlike a characteristic waterfall approach, a good planning process is a process of iteration, convergence, and parallel efforts across disciplines.
Design tradeoffs
The above all sounds good, probably. If you do the above you can solve the resource allocation and overall scope of the product relative to different types of customers. It doesn’t, however, get to the heart of one really hard problem. What to do when customer needs conflict?
One thing to do is bury your head in the sand and just say there is no conflict. That is saying, for example, that end-users will value features missing that IT removed or that IT will just get over themselves and not mind arbitrary extensions being loaded on the device. That doesn’t work of course.
Because product development does not end—a release is just a point in time, which is even more so on today’s continuous product cycles—you do need to get comfortable not doing everything for everyone every release. There is always a next release. So resolving design tradeoffs needs to be about having a set of principles and a product architecture that you can build on.
This is where understanding where your own product is going is crucial, a longer term strategic view. Most products when they are new receive a combination of praise and criticism from power users, as one example. If the scenario or problem solved is compelling, power users will praise the product. They are, generally, quick to offer up suggestions and feedback for how to add flexibility or more features. That’s exciting.
In the process of designing the product, a key responsibility for the designer is to know where a design is heading. How will you know how to add more power and control? If you don’t have ideas while designing the product in the first place you might be designing yourself into a hole. Famously, copy/paste were missing from smartphones when first released. Even with a new touch design language, designers clearly understood where this would go. That was important and made it easy to introduce without a major shift in the overall ease of use that was the hallmark of the design. This could have easily been a conflict between power and ease of use.
Code architecture or architectural approaches play a key part in how you think about where you are today relative to where you are going. Many of the architectural differences between the different OS platforms we see today compared to how an OS looked 10 years ago result from making tradeoffs in the architecture. App stores, sandboxing, APIs with brokers are all about tilting the architecture towards security, battery life, and end-user safety.
We look at these changes in architecture today and compare them to where we came from and can easily see the difference. But think about the debates and planning that took place—this was a big change in approach. It was not easy for those making platforms to make these architectural tradeoffs in such a new way. Creatively addressing new requirements is a key part of understanding your product evolution over time.
An important tool is having a set of product principles–design language, architectural framework, and customer value proposition. These principles not only guide the development team but make it easy to articulate the tradeoffs made to customers when you’re done. That doesn’t make the dialog easy or even get folks to agree with the choices you make. But you’re having a conversation informed by what your product is trying to do.
An amazing change in happening in our industry today with “BYOD” or bring your own devices to work. This is a whole new level of design tradeoff the industry is facing. Since the late 1990’s the focus on administration has been to “lock down” or “control” computing devices. That worked well given the choices and challenges faced.
Consumers now can bring their own devices to work, work from home, or find ways of doing work outside the scope of the corporate network/software/device. This doesn’t change the security, IP, and safety needs of a corporation or government agency. It does change the decision framework of IT. Their internal clients have choices and how those choices overlay with the design tradeoffs in products is very interesting.
Just as APIs and OS capabilities are changing, and perhaps resetting expectations of some customer types, the way devices and software are managed are changing architecturally. As you’re planning the product, developing an architectural approach that plays out in a forward looking way is going to be a key part of resolving the design tradeoffs.
This is the engineer part of resolving tradeoffs—sometimes it is not just coming up with features, but new architectural approaches can put you on a new trajectory. That new trajectory defines new ways to design products for many types of customers.
The only thing you know for sure in addressing customer tradeoffs is that there is no right answer that always pleases all customers. That’s the nature of appealing to multiple customer types. That’s why product development is also a social science.
An Example: the enterprise challenge
These days, one of the most challenging product design debates centers on how one balances the needs of enterprise IT and the community of people, humans, using products within an enterprise. There is a major shift going on in our industry with the wave of consumer products able to fulfill scenarios outside of the control IT.
The dramatic change is decidedly not in the enterprise need to secure the digital assets of an organization or to maintain the integrity of corporate networks or to even manage the overall usage of corporate resources (balancing work and non-work). These requirements are not only still there, but in a world where a single leak of customer or financial data can make international news or the interest of regulatory bodies these requirements are more intense than ever.
The dramatic change, however, is in the ability for people within an enterprise to easily acquire tools to accomplish the work they need. In another era, obtaining servers, licensing software, getting it on site and running were all tasks that required IT sponsorship and often resources. Today tools such as cloud storage, peer to peer communications, CRM, or even commerce are just a few clicks away and require nothing more than a corporate credit card, if that.
The only policing mechanism that can be deployed “against” these tools is a policy approach. While one can block TCP/IP ports or suspicious network traffic (and certainly block downloaded client code on managed PCs), even this is challenging. Network access itself is easily obtained via a WiFi/4G access point or phone as hotspot.
The traditional approaches of locking down a device simply don’t work. In a world of mobile devices and perhaps even multiple operating systems, there isn’t a clear focal point for these lock down efforts and certainly not even a single implementation that is possible.
Where some might see this as freedom, others might see this as chaos. Smart product design efforts see an opportunity. There’s an opportunity to design products that embrace this challenge and opt to provide an architecture for IT to get done what is required of them and at the same time making it easy for people to discover and spread tools to other coworkers that solve the business challenges they face.
The design challenge is about defining a new architecture that takes into account the reality that you can design yourself into a corner if you go too far in either direction. You can focus too much on empowering people and face either a policy choice from IT or worse an active campaign against your product from the enterprise analyst community. You can focus too much on IT and your product might enable IT to turn a cool product into a locked down or customized experience that drives end-users to your competition, which is only a click away.
This is where design principles can come into play. What is the current implementation of management—what has been implemented by IT organizations that has might have lowered the satisfaction of corporate computing relative to home computing or BYOD, for example? Efforts like excessive logon scripts or complex network access requirements that keep people from using the network and favoring the path of least resistance, perhaps? Or perhaps a customized user experience for a product that is also used on home PCs and thus “different” at home and work, driving more use of the home PC for work?
Given an environment like that what is an architecture that takes into account the downsides of existing approaches to enterprise management and creates a more favorable experience while maintaining security, knowing that options abound?
Can a device be managed without changing the user experience? Can a software product be locked down relative to critical functions to reduce support calls without IT getting between the work that needs to be done and the employee?
These are the sorts of questions one needs to raise in reconciling the apparently contradictory needs of IT and people using modern products and services.
Really diving into these as you design your product has the potential to develop a real competitive advantage for your product and service.
–Steven
###
Learning by Shipping 