How Do I Know When I’m Done with Customer Discovery?

We’ve planned and executed our customer discovery process like George Clooney pulling off a casino heist. We’ve perfected every detail through three rounds of interviews, and now we have a dozen interesting people telling us a dozen interesting stories that have given us the insights into our product that we were looking for.

We’re feeling pretty good about ourselves. We’re going to celebrate at Cracker Barrel.

But a question keeps nagging at us. Are we actually done with the customer discovery process?

Short answer, yes with an if. Long answer, no with a but.

We are done with customer discovery IF we have a concrete hypothesis about customer pains and a value proposition we are ready to test for purchase intent.

That is to say, additional interviews are showing diminishing returns. We are no longer learning at the same pace. We have a decent sample size to synthesize the data, and that data synthesis gives us the focused customer persona we require to move on to an actionable agenda. Goodbye stranger, it’s been nice.

Persona wireframe showing modification

At the same time, we’re NOT done with customer discovery, because we are never done. Potential customers will be continuously discovering our product, so we need to continuously discover how our customers are changing, as well as explore new segments and opportunities. BUT we can’t steer our boat if we never leave the dock. (Unless the boat is towing the dock for some reason, but let’s save that discussion for another time.)

At some point, we need to build a product and see if anyone will buy it — or rather, see if anyone will buy it and then build the product. So we can’t just talk to people forever.

What we really need here is a more practical question: What are the elements of a complete customer discovery process?

HOW MANY CUSTOMERS SHOULD WE INTERVIEW?

This is simply a question of too few versus too many. If our initial customer persona is “independent store owners,” but we interview just three or four retailers, then we are getting a very limited view of small-business ownership. If even one of these people is an outlier — say a niche salesperson among generalists — then our conclusions are going to be skewed away from early adopters, and the next thing we know we’re trying to sell water balloons to porcupines.

A porcupine with two water ballons

On the other hand, if we are interviewing 50 people a week, we are going to get lost in the data.

We are not Google.

We don’t have the resources to blow on an experiment to determine which of these 41 shades of blue will earn us an extra $200 million. Time is precious, so let’s focus on quality instead of quantity.

It’s also important to conduct these interviews (and perform our synthesis) over a relatively short period. If we interview 10 store owners in 10 weeks before we sit down to debrief, it’s going to be hard to remember the context of the early interviews; we’re more likely to unnecessarily fixate on the last thing we heard. This is a classic example of recall bias. We need a small batch of data for our small human brains to deal with.

Every company is different, and every product is different, but a practical rule of thumb would be to interview 5-20 potential customers in a single cohort over a one-week period. Ideally, each team (one interviewer and one notetaker) should be involved in at least five interviews. So a team of four people should run at least ten interviews in a week.

In Lean Startup, this is the Build, Measure, Learn loop backwards. We are trying to Learn who our customer is and what their pain points are. Then we figure out what data we need to Measure — in this case, qualitative data from interviews that will help us figure out our ideal customer persona. Then we “Build” our interview guide. At this point, we can go through the loop forward: conduct the research, evaluate the data, and hopefully learn something useful.

Build Measure Learn Loop

HOW MANY ROUNDS OF INTERVIEWS SHOULD WE DO?

Again: too few or too many?

The purpose of conducting multiple rounds of interviews is to refine our generative research (Who is our customer?) to maximize the value of our evaluative research (How much will they pay for our product)? If we are correctly processing the information we get from our store owners, we will notice our customer persona is no longer radically changing week to week. As Steve Blank tells us, we want lots of data, but we want lots of data so we can extract insights.

With each round of interviews, we should see a reduction in outliers and an increase in actionable insight that will get us closer to our MVP. It helps to start with an open-ended script in the early rounds, noting the questions that provoke responses that help us better understand our customer. We can then use these points to refine our customer persona and add new, more tightly focused questions with each round.

Keep in mind that other experiments — such as surveys and ethnographic observation — will augment our discovery data, but once we’ve refined our customer persona enough to shed our preconceived biases, we are ready for solution testing, and finally an actual build. Hooray!

Two people having high five

HOW DO WE PROCESS THE INFORMATION?

What are we actually looking for in our data that will allow us to move from one round to the next, and ultimately to a product launch?

We want to see actionable insights, such as patterns and themes. If we cross-reference our interviews in a way that helps us update our assumptions and tighten our customer persona, then we are — slowly but surely — on our way to building a business by clearly understanding who our customers are and what we will need to build to make them happy.

To be clear, we are not looking for a certain percentage of customers to agree with us or hand us money. We are looking for a customer segment that our product can effectively target and build a business model on.

So instead of searching for a group of interviewees to validate our preconceived biases, we need to look for something members of our desired customer segment have in common. If only 3 out of 20 store owners are interested in our product, that’s ok. But the fact that those 3 people own single stores while the other 17 own chains reveals a pattern that tells us what to look for as we set up our next round of interviews.

WHEN DOES IT END?

Never.

Two people talkingThe industry is constantly changing. The market is constantly changing. People are constantly changing. Some of the best companies out there do discovery interviews every week.

We need to see the customer discovery process as not only a path to building a product, but a circuit course that helps us exercise and grow a viable business model. So if we want to be disruptors, we need to get comfortable talking to strangers.

Discussion (0 comments)

  1. Pingback: Food for Agile Thought #183: Leadership Health Check, InfoQ’s State of Practices, Feedback Fails Us, Courageous Leader Myth – Matteo Borghesi

Got something to say?

This site uses Akismet to reduce spam. Learn how your comment data is processed.