Learning by Shipping

I’ve been surprised at the “feedback” I receive when I talk about products that compete with those made by Microsoft.  While I spent a lot of time there, one thing I learned was just how important it is to immerse yourself in competitive products to gain their perspective.  It helps in so many ways (see https://blog.learningbyshipping.com/2013/01/14/learning-from-competition/).

Dave Winer (@davewiner) wrote a thoughtful post on How the Times reviews tech today. As I reflected on the post, it seemed worth considering why this challenge might be unique to tech and how it relates to the use of competitive products.

When considering creative works, it takes ~two hours to see a film or slightly more for other productions. Even a day or two for a book. After which you can collect your thoughts and analysis and offer a review. Your collected experience in the art form is relatively easily recalled and put to good use in a thoughtful review.

When talking about technology products, the same approach might hold for casually used services or content consumption services.  In considering tools for “intellectual work” as Winer described (loved that phrase), things start to look significantly different.Software tools (for “intellectual work”) are complex because they do complex things. In order to accomplish something you need to first have something to accomplish and then accomplish it. It is akin to reviewing the latest cameras for making films or the latest cookware for making food. While you can shoot a few frames or make a single meal, tools like these require many hours and different tasks. You shouldn’t “try” them as much as “use” them for something that really matters.  Only then can you collect your thoughts and analysis.Because tools of depth offer many paths and ways to use them there is an implicit “model” to how they are used. Models take a time to adapt to. A cinematographer that uses film shouldn’t judge a digital camera after a few test frames and maybe not even after the first completed work.

The tools for writing, thinking, creating that exist today present models for usage.  Whether it is a smartphone, a tablet, a “word processor”, or a photo editor these devices and accompanying software define models for usage that are sophisticated in how they are approached, the flow of control, and points of entry.  They are hard to use because they do hard things.

The fact that many of those that write reviews rely on an existing set of tools, software, devices to for their intellectual pursuits implies that conceptual models they know and love are baked into their perspective.  It means tools that come along and present a new way of working or seeing the technology space must first find a way to get a clean perspective.

This of course is not possible.  One can’t unlearn something.  We all know that reviewers are professionals and just as we expect a journalist covering national policy debates must not let their bias show, tech reviewers must do the same.  This implicit “model bias” is much more difficult to overcome because it simply takes longer to see and use a product than it does to learn about and understand (but not necessarily practice) a point of view in a policy debate.  The tell-tale sign of “this review composed on the new…” is great, but we also know right after the review the writer has the option of returning to their favorite way of working.

As an example, I recall the tremendous difficulty in the early days of graphical user interface word processors.  The incumbent WordPerfect was a character based word processor that was the very definition of a word processor.  The one feature that we heard relentlessly was called reveal codes which was a way of essentially seeing the formatting of the document as codes surrounding text (well today we think of that as HTML).  Word for Windows was a WYSIWYG word processor in Windows and so you just formatted things directly.  If it was bold on screen then it was implicitly surrounded by <B> and </B> (not literally but conceptually those codes).

Reviewers (and customers) time and time again felt Word needed reveal codes.  That was the model for usage of a “word processor”.  It was an uphill battle to move the overall usage of the product to a new level of abstraction.  There were things that were more difficult in Word and many things much easier, but reveal codes was simply a model and not the answer to the challenges.  The tech  world is seeing this again with the rise of new productivity tools such as Quip, Box Notes, Evernote, and more.  They don’t do the same things and they do many things differently.  They have different models for usage.

At the business level this is the chasm challenge for new products.  But at the reviewer level this is a challenge because it simply takes time to either understand or appreciate a new product.  Not every new product, or even most, changes the rules of the predecessor successfully.  But some do.  The initial reaction to the iPhone’s lack of keyboard or even de-emphasizing voice calls shows how quickly everyone jumped to the then current definition of smartphone as the evaluation criteria.Unfortunately all of this is incompatible with the news cycle for the onslaught of new products or the desire to have a collective judgement by the time the event is over (or even before it starts).This is a difficult proposition. It starts to sound like blaming politicians for not discussing the issues. Or blaming the networks for airing too much reality tv. Isn’t is just as much what peole will click through as it is what reviewers would write about. Would anyone be interested in reading a Samsung review or pulling another ios 7 review after the 8 weeks of usage that the product deserves?

The focus on youth and new users as the baseline for review is simply because they do not have the “baggage” or “legacy” when it comes to appreciating a new product.  The disconnect we see in excitement and usage is because new to the category users do not need to spend time mapping their model and just dive in and start to use something for what it was supposed to do.  Youth just represents a target audience for early adopters and the fastest path to crossing the chasm.

Here are a few things on my to-do list for how to evaluate a new product. The reason I use things for a long time is because I think in our world with so many different models

1. Use defaults. Quite a few times when you first approach a product you want to immediately customize it to make it seem like what you’re familiar with.  While many products have customization, stick with the defaults as long as possible.  Don’t like where the browser launching button is, leave there anyway.  There’s almost always a reason.  I find the changes in the default layout of iOS 6 v. 7 interesting enough to see what the shift in priorities means for how you use the product.
2. Don’t fight the system.  When using a new product, if something seems hard that used to seem easy then take a deep breath and decide it probably isn’t the way the product was meant to do that thing.  It might even mean that the thing you’re trying to do isn’t necessarily something you need to do with the new product.  In DOS WordPerfect people would use tables to create columns of text.  But in Word there was a columns feature and using a table for a newsletter layout was not the best way to do that.  Sure there needed to be “Help” to do this, but then again someone had to figure that out in WordPerfect too.
3. Don’t jump to doing the complex task you already figured out in the old tool.  Often as a torture test, upon first look at a product you might try to do the thing you know is very difficult–that side by side chart, reducing overexposed highlights, or some complex formatting.  Your natural tendency will be to use the same model and steps to figure this out.  I got used to one complicated way of using levels to reduce underexposed faces in photos and completely missed out on the “fill flash” command in a photo editor.
4. Don’t do things the way you are used to.  Related to this is tendency to use one device the way you were used to.  For example, you might be used to going to the camera app and taking a picture then choosing email.  But the new phone “prefers” to be in email and insert an image (new or just taken) into a message.  It might seem inconvenient (or even wrong) at first, but over time this difference will go away.  This is just like learning gear shift patterns or even the layout of a new grocery store perhaps.
5. Don’t assume the designers were dumb and missed the obvious. Often connected to trying to do something the way you are used to is the reality that something might just seem impossible and thus the designers obviously missed something or worse.  There is always a (good) chance something is poorly done or missing, but that shouldn’t be the first conclusion.

But most of all, give it time.  It often takes 4-8 weeks to really adjust to a new system and the more expert you are the more time it takes.  I’ve been using Macs on and off since before the product was released to the public, but even today it has taken me the better part of six months to feel “native”.  It took me about 3 months of Android usage before I stopped thinking like an iPhone user.  You might say I am wired too much or you might conclude it really does take a long time to appreciate a design for what it is supposed to do.  I chuckle at the things that used to frustrate me and think about how silly my concerns were at day 0, day 7, and even day 30–where the volume button was, the charger orientation, the way the PIN worked, going backwards, and more.

–Steven Sinofsky