What’s the first thing we should do?
My default answer is to push teams towards starting with customer interviews and general observations and research.
Why? Because most teams don’t naturally seek out customer understanding. And once your team starts the activities they prefer (coding, spec’ing, designing), it’s especially hard to build the habit of customer development. Customer development is also a lower-risk activity, which is reassuring to big enterprise orgs.
That said, the point of Lean is to avoid waste and learn as quickly as possible, and sometimes starting with customer development is not the quickest path to learning.
When do we build first and use quantitative data to guide us?
Quantitative data — what you get from telemetry or A/B testing — is the only way to prove that what you built had the desired impact, moved the desired metrics.
But it won’t tell you why a solution worked (or didn’t work). And it doesn’t guide you towards what to try next. “Throw spaghetti against the wall and see what sticks” is not the way to learn quickly and avoid wasting perfectly good pasta.
Starting the loop with “build” — BUILD-MEASURE-LEARN — is great when you have a really strong idea of what to build to solve the problem.
It doesn’t make sense if you’re not really sure what the problem is, or you don’t have a clear idea of how to solve it.
When do we start by learning, and wait a bit for that learning to guide what we build?
Qualitative data — what you get from customer interviewing or ethnography — is the tool to use when you don’t fully understand the customer’s problem, the environment they operate in, or how they explain their processes and limitations.
(note that “our backlog is full of customer requests/feedback” is not the same as understanding customers’ underlying problems / behaviors!)
When you’re still asking “what?” and “why?”, you’ll waste less time by starting the loop with “learn” — LEARN-BUILD-MEASURE.
But that’s just the starting point…
There’s a reason it’s a loop. Regardless of what technique you used to start solving a problem, the way you continue to evolve it to build a better solution is to combine qualitative and quantitative data. Customer development and metrics; research and telemetry; interviewing and testing.