You’ve probably heard the “medicine vs. vitamin” analogy of products — that is, to maximize your chances at success, it’s easier to sell medicine (that fix a specific, painful ill) than vitamins (that offer a vague promise of ‘feeling better’).

But I’ve realized there’s another category of product – the “gym membership” product:

• Your customers have accepted that they have a problem they really want to deal with (even if it’s not ‘life-or-death’).
• Of course there’s a free alternative, and some people get along just fine using it, but most people need the accountability, the support, and the motivation of a paid solution.
• Some people will use it religiously every day, some will use it once a week, and some will use it gung-ho for a week and then peter out for a few months.
• Some people will actually never use it, even though they keep paying you.  They’ll kind of resent you for it, though.
• There are a lot of different ways to use it to meet your goals. Some people walk in full of purpose, and know exactly how they need to use it.
• Most people need suggestions on how to use it effectively.  Otherwise they use it poorly or just wander around aimlessly.  They’ll be unsatisfied and rate you poorly.
• But if you force people through a long “initiation” session, they’ll find that obnoxious.
• Once people feel comfortable using it, they’ll find it gives them a lot more energy.  They’ll wonder how they ever got along without it!

You could also call these aspirational products.  We sign up because we want to be that kind of person.

Unlike most products — where the first and biggest challenge is getting people to give a damn enough to give you five seconds of their attention  — it isn’t that big a challenge to get people to sign up.   They see the equivalent of “better body in 90 days” and that’s enough to get them to click.

The challenge is getting people to get started and come back.   To do that, you have to anticipate and answer customer questions.  And the more subtly you do this, the more your app will feel intuitive/”just works”/delightful.

These are the first two questions that I’m trying to answer right now for both KISSmetrics and KISSinsights.

Question 1: What do I do first?

You probably have a pretty good sense for, in an ideal world, what you wish your customers would do first in order to get the most value out of your product.  However, you may be wrong.  This is where, now that you’re in later-stage customer development, it can be tremendously useful to walk someone through using your product the way you think it should be used.

One of two things is likely to happen: they’ll say, “oh, that makes sense – why doesn’t the app tell me I should start by doing that?” or they’ll say “that doesn’t make sense – why can’t I do X first?”

For KISSmetrics, we heard a lot more of the former, so we built a first-user experience designed to tell you how you should start.  Of course, we’re continuing to realize that there’s a lot more the app could be “telling” people, so we continue to tweak.  For KISSinsights, I suspect the latter is more accurate, which means we’ll need to make bigger changes to the first-user experience.

Question 2: How are other people using it?

I think this question really encompasses three things: “I want proof that other people are actually getting value from this”, “I want to see how much trouble they had to go through to get results”, and just plain curiosity.

I posted some examples for KISSinsights, which seems to have been a big help already (judging by the number of emails I’ve gotten since then).

One thing I’m still pondering, though — how do you identify the “goes to the gym every day” customer vs. the “tries to make it once or twice a week” customer, when they’ve just started to use your product?

Dealing with these questions has meant a very different kind of customer interview.  I wrote before that I was wary of telling the customer “here’s our product and here’s how you should use it” – now I think the way to think about it might be:

“Traditional” product development: “Let me tell you about us”

Early-stage customer development: “Tell me about you”

Later-stage customer development: “Let me tell you about other people like you“

But what happens is that we’re eating dogfood with a side order of the Curse of Knowledge.  We know how our products ought to behave and we know what they’re capable of, and that affects our impression of how easy they are to use.

So what we really need to do is eat someone else’s dogfood – that is, experience our products as though we were someone else, a someone who lacks all the inside knowledge that we have.

Here’s what I did last week:

• I offered to set up customers’ KISSmetrics funnels for them (this offer still stands, if you’re interested)
• Customers who agreed, I walked through their sites and identified the key workflows that I believed they ought to be tracking
• I created their funnel events and reports, and checked back to troubleshoot them
• I followed up (via email or phone) to explain what I’d done with their accounts and why I recommended using the product in this way

And you know what?

1. It was a LOT of work.
2. It was totally and completely worthwhile.

Being completely unfamiliar with their sites and goals put me in a similarly handicapped position as the customer who is unfamiliar with my product.  It became painfully obvious that certain copy was being ignored and that some features were grouped poorly.  There were several configurations where I thought, this would be much easier if we’d provided a visual example instead of trying to write inline help.

Equally valuable was the followup.    I would say “I’d recommend you use the product in this way–”, and someone would respond with “–but we can’t, because of this detail of how our application works”.   A step-by-step walkthrough primes the customer for a lot of little “but…”s and “why?”s that might never make it into a bug report or feedback email.

–

As I’ve written this, I think “but doesn’t this sound like the ‘traditional’ product approach?  Telling the customer, “here’s our product and here’s how you should use it” instead of understanding their needs?

And I think the difference is that these conversations were preceded by months of customer development, and conducted (as best I could) as a conversation.   This is where tone makes a big difference.  “Here’s how you should use our product” is very different from “Based on [assumptions], I recommend you use our product in this way – let me know what you think as I explain the details.”

There has been a lot written about early stage customer development – finding your market, identifying pain points and constraints, creating a vision that reflects customer needs without being ‘what customers ask for’.  But I’m finding that later stage customer development is a bit of a different beast.

It’s the time to move beyond abstractions and into specifics.  You’ve proven your case in theory; that’s why customers are using your product.  But there are definitely gaps in practice between how you envision customers using the product and how they actually are.  I’ll continue to write more as I learn more.

“The plural of anecdote is not data.” — Frank Kotsonis

That’s a good way to shut down someone who is trying to use sparse “I know someone who…” stories to second-guess a decision.  But in general, it’s not true.  The plural of anecdotes IS data – as long as you’ve collected them properly and challenged them rigorously.

A couple weeks ago I wrote:

It’s tempting to try and turn as many questions as possible into quantitative ones, because those are easier to measure through unobtrusive means – web analytics, looking for data patterns, surveys – and give clear numerical answers.

The lack of numbers is hard for a lot of companies to deal with.  How exactly do you turn qualitative data into something meaningful?  How do you avoid squeezing the data to fit what you wanted it to say?

“Everyone is concerned about security”

At my Lean Startups presentation at Web 2.0 Expo last week, I said “The customer is the expert; you’re just a notetaker.”  Which, naturally, was taken out of context by quite a few people.

What I mean, though, is that when you conduct customer interviews, you need to sit back and not say things that will influence the interviewee’s responses.  If you agree too enthusiastically, or go from sounding interested to not interested, people will pick up on those cues and (consciously or not), tailor what they say next to make you happy.

I was a psychology major in college, and we had to practice before we could run experiments with human subjects – using a neutral tone of voice, not varying our script, giving very non-committal responses.  This isn’t much different.

When I worked at Yodlee, many of our potential customers felt that their end-users would be reluctant to share their passwords with us.  One large financial institution cited this as proof: in a recent user survey, over 90% of customers had said they were “concerned” or “very concerned” about data security.

Well, of course.  Everyone is concerned about security.  There’s no cost to saying you’re concerned about security.

But if you asked those same people if they’d be willing to pay $ for more security, or give up features in exchange for security, suddenly they’re not so concerned.  In our case, we showed them the product and explained the features and asked “is there anything that would prevent you from wanting to use this?”  No.  “What about security concerns?”  Well… no, we trust you.

It’s all about how you frame the question.  Be as neutral as possible.  Ask a friend outside the company to help you spot potential biases in your wording.

“A friend-of-a-friend said…”

The reason anecdotes get a bad rap is because most of them are full of holes.  When you have an interesting story, make sure you get the background:

• Who said it?
• What is their role?
• Is there anything about the person or their company that is outside your expected target market? (i.e. they’re a nonprofit and you usually sell to commercial companies)
• Might they be a nutcase?  (i.e. that guy who says he never accepts cookies, belongs to no social networks, and only browses the web through an anonymizer… you can probably safely ignore his concerns about security.)

This is critical because otherwise it’s going to seriously frustrate you when you see seemingly contradictory statements in your notes — oh, then you notice all the marketing people have one set of concerns, the engineers have a different set.

“Let me be clear: are you saying…?”

It is really exciting when a customer says something that validates your hypothesis.  Unfortunately, sometimes you want that so much that you conveniently mishear part of what they’re saying.

When someone starts to agree with you, first, write down everything they say.  Don’t try to pick out the important parts during the interview.

Next, repeat back to them what you think they were expressing: “Let me be clear: are you saying that your deployment takes a week and that is too long?”

Don’t stop there – proceed with a modified 5 Whys approach.  (Modified because you don’t want to sound too annoying.)  Even once you know the “pain”, you need to know WHY it’s a pain.

For example:

“Surveying customers is a pain.”

“Why?”

“It takes so long to write a survey… it’s not worth the time.”  (Problem: too time-consuming?)

“Why isn’t it worth the time?”

“Because so few people respond, we don’t really get useful data.”  (Aha! TOTALLY different problem: not effective!)

Commit yourself to revisiting the data

Once you start getting a fair amount of customer data, you’ll have a picture in your head of what the consensus from these customers is.  That picture in your head is highly subject to wishful thinking.

Commit to re-read your raw, unedited notes, after every few interviews to make sure you and reality stay in touch.

“What if we don’t build the right (beautiful/efficient/fully-featured/scalable) thing,

and because of that, something horrible happens?”

We try pretty hard to be lean at KISSmetrics, but we still have a lot of internal debate sometimes around whether and when we should build things.  These are 4 arguments we’ve had over the last 6 or 7 months, and in all cases, Lean was right.

We need a First User Experience as part of the initial KISSmetrics beta.

The argument: I’ve written multiple times about the importance of a good guiding first user experience and I looked at what we were starting out with and said, “everyone is going to show up, get confused, and never come back.”  I mean, I totally thought I was right on this one.

What happened: Our initial beta customers were classic early adopter types who didn’t mind experimenting with our API and writing some Javascript, and were able to use the product and provide tons of valuable feedback to us, all without requiring a wizard to guide them through the process.

Now, 6+ months later, we’re starting to get beyond that initial “early adopter” population, and we’re definitely seeing that beta customers would benefit from a guided setup.  So we’re investing the time in building out a proper first user experience.  But did we need it back in October ‘09?  No.

Lean 1, Fat 0.

We need more granular controls around how much data we show in the initial KISSmetrics beta.

The argument: In the interest of releasing quickly, the initial version only showed your funnel for your last 10,000 events.  No date filtering, and no historical data.  For high-traffic sites, this might be less than a day’s worth of activity!  Come on, we really should let people show the last week or the last month, or give them date-pickers – right?

What happened: About half of our earliest beta customers were smaller startups, so the low amount of data wasn’t an issue.  The other half weren’t crazy about it, but they gave us useful feedback on how they wanted to dice up their data.

Now, 6+ months later, we just released custom date pickers this morning.  Yeah, people have been complaining about this for awhile – but they’ve stuck around and continued using our product in the meantime.

Lean 2, Fat 0.

We need a super cool relative date picker in Sharefeed.

The argument: Shouldn’t a product that’s built around scheduling have a really nice, dead-simple way of indicating dates?  Like, if I want a tweet to go out tomorrow, why can’t I just type in “tomorrow”?  (The classic “wouldn’t it be cool if…” problem.)

What happened: Other priorities intervened and no developers got past the initial experimenting state.  We’ve been using Sharefeed internally for over 4 months now, and the lack of a cool date picker is not a pain point.  Not a single person has emailed us complaining about our date picker.

Lean 3, Fat 0.

“KISSinsights – there’s not much there, is there?”

The argument: OK, I don’t think those were Hiten’s exact words, but they weren’t far off.   The very first version was missing most of the survey configuration options, an “about” page, password reset, and the admin pages (which only we could see) had typos and stray list element bullets.  It was not pretty.

What happened: No one (except me) noticed the lack of an “about” page.  One person emailed me so we could manually reset his password.   By getting that product out in the hands of real people quickly, we knew exactly what features to build/refine next.

2 months later, we have a still-minimal but definitely-viable product, and 700+ beta users.

Lean 4, Fat 0.

I’m speaking at Startup Lessons Learned tomorrow – hope to see you there!

Customer development is the opposite of product development, or “if we build it, they will come” thinking.   We all know that’s not the case – often you build it, and no one cares.

The idea is to validate that you have a market, with problems, that they’re willing to pay to solve — and then build something that solves those problems.   The emphasis is on learning and discovery before you write a 50-page spec or spend months writing code. Read Eric Ries’ post  What Is Customer Development for a lot more detail.

“This doesn’t sound like anything new,” you may be thinking, and you’re pretty much right.   There are plenty of examples of successful companies who used interviews and cheap prototypes to validate ideas before committing to building them.

What has changed is the availability of free/cheap tools and direct access to customers through social media, making it more practical for an individual product manager to do this on their own, on a skunkworks basis even.

But asking the customer what they want never works –

I know.  Customers tend to ask for “me-too” features, or they ask for something and then after you build it, say “I know we asked for this, but it isn’t what we really wanted.”

Customer development isn’t asking customers what they want – it’s seeking to understand what they need, how they work, where their pain points and highest priorities are.  Customers may not be able to articulate what they want, but they can’t hide what they need.

What will I learn?

The short answer: how people are really getting a task done, who is doing what, and why it sucks.

The more detailed answer: What you should be learning from customer development interviews

How is this different from usability testing?

Usability testing teaches you whether or not people are capable of using your product; it doesn’t tell you how likely they are to actually buy it.

In usability testing, you learn about the product you’ve already built.   If your product hypothesis was wrong, it’s kind of too late to do much about it.  In customer development, you learn about your customer before you build the product.  That way, if your product hypothesis is proven wrong, you can change course quickly without having wasted resources and time.

How long will it take?

I don’t know.

Sorry, but that’s the truth.  I can’t guarantee that this methodology will get you useful answers in two days or two weeks.  I can guarantee that you’ll start learning useful new tidbits from your first customer interview, but it may not be something actionable.

At KISSmetrics, I quickly learned that the reason so many companies loved our Survey.io customer development survey is that “I don’t have to write anything – you tell what I should be asking and how to ask it”.

People don’t like to write – in fact, they’ll put off a task if it requires them to think and write something.  That was a good insight, but not enough (on its own) to drive any new product decisions.  We needed more of those insights before they added up to something we could base a product on.

For this reason, customer development needs to be a process — not a project.  Always be listening, always be asking why, and always be testing hypotheses, and it will pay off.  You just never know when.

How do I get started?

If you already have existing customers, reach out to one today and schedule some time to talk with them about how they get their jobs done.  Ask about how they use your products, but also take a step back and ask about the general task that your product is supposed to help them with (i.e. if your product is a bug tracker, ask them about QA testing in general).

You don’t have a hypothesis to test (yet), so this is just an open-ended conversation.  Let the customer talk.   The Evolution of the Customer Development Interview has good tips.

You may be at a company who has existing products and customers, but you don’t have access to them.  As insane as this is, it’s not uncommon to try and lock the product managers up in a little room to wrangle up requirements somehow.   The fastest way to get access is probably to bargain directly with sales.

Ask to join them on a sales visit or conference call.  Offer to help with demos.  If you have to promise to be a “fly on the wall” in order to make them feel comfortable, do it.    In my experience, customers love having a product manager in the meeting – they feel like you’re an ally who isn’t just trying to sell them — and as a result, they’re often MORE likely to buy.  (Tell your salesfolk that.)

How can I find customers before I’ve even built a product?

How were you planning on finding them after you’ve built a product?

Seriously, this is the million-dollar question, and the answer is not “we’ll build it and they’ll come” or even “we’ll advertise a LOT” — unless you’re Apple, Microsoft, Google, or Facebook, you don’t have the dollars or distribution to make those work.

So you have to get creative.   I’ve previously written about how to find prospective customers using methods that take less than $20 and a couple days.

How do I convince my boss that this is a good use of my time?

Unless your boss is already a devotee of Toyota Lean Manufacturing processes or Four Steps to the Epiphany, start out skunkworks style. Remember, bosses like solutions, not problems.  Don’t say “we need to be doing X”, figure out how it can be done and prove the benefits.

Carve out some time, do some customer interviews.  Work with a designer to throw together a fake splash marketing page for a new product and get feedback.  Once you’ve learned something useful, use that to get buy-in for making this a permanent part of your process.

How does this change when I have an established brand and customers?

You may need to create somewhat more polished splash pages/demos in order to not hurt your credibility.

You may also feel more comfortable testing concepts without connection to your company name.  Using another domain name is a good option and worth the time to set up if you’re going to be running experiments frequently.  But you can go even simpler and use tools like shared Google docs (create a new Google account that isn’t linked to your name) or Skitch to post screenshots privately.

Where can I learn how to do this?

Books:

• The Four Steps to the Epiphany
• Tuned In: Uncover the Extraordinary Opportunities That Lead to Business Breakthroughs
• Inspired: How To Create Products Customers Love

Blog posts:

• Why Your Customer Feedback is Useless by Laura Klein
• 7 More Ways People Suck at Customer Development Interviews by Laura Klein
• The Evolution of the Customer Development Interview by Sachin Agarwal
• Customer Development Interviews How-to: Finding People by Cindy Alvarez
• Customer Development Interviews How-to: What You Should be Learning by Cindy Alvarez
• How I Learned to Grok Customer Development by Ash Maurya
• How I Built My Minimum Viable Product by Ash Maurya
• Customer Development Checklist for my Web Startup by Ash Maurya
• What Is Customer Development? by Eric Ries
• When NOT to listen to your customers; when NOT to rely on split-tests by Eric Ries
• How to Develop Your Customers Like You Develop Your Product by Nivi at VentureHacks
• Customer Development is not a Focus Group by Steve Blank
• Customer Development – Past, Present, and Future by Steve Blank
• Customer Development Update (and why I’m sticking with 1-on-1 talks) by Giff Constable
• Seller Beware: Customers Have Their Own Agenda by Brant Cooper
• How to Find Early Adopters by Brant Cooper

People to follow on Twitter:

• @ashmaurya
• @brantcooper
• @cindyalvarez
• @ericries
• @giffconstable
• @hnshah
• @lauraklein
• @richcollins
• @sgblank
• the #leanstartup and #customerdev hashtags

I’ve run enough user testing sessions on a shoestring budget to know a lot of the gory little details, and yesterday I shared a bunch of them with the startups over at the fbFund incubator.

Why Not Online Testing?

Online user testing tools are cheap, and it’s tempting to think that you can get insight without having to figure out what to say to people face-to-face.  But the numbers can hide a lot – if a user successfully completes your face, but they have their face scrunched up in tension the whole time, or they’re cursing you under their breath – that’s critical detail that you’re missing.

The Crazy Guy

You usually get one “crazy person” per 7-8 person group.  You probably want to write that in your notes – “crazy guy” – so you can keep that in mind when reviewing his comments.  On the other hand, if three other people agree with the crazy guy, they probably all have a valid point.

“I Just Don’t Know What to Do Next”

If a user “freezes” and says they don’t know what to do next, resist the temptation to jump in and help.  Remember, you’re playing the neutral moderator, so you don’t take it personally.  Instead, ask what they would expect to do next.  It’s surprising how often you’ll hear something like “Well, I was expecting to see a button that said Pay Now, but all there is here is a link that says Enter Credit Card Information.  I don’t want to click it because I’m not sure…”

1. Crossing the Chasm: Marketing and Selling Technology Products to Mainstream Consumers by Geoffrey Moore

   
   Realization: You can get early customers to love your product, but you can’t use those same methods to convert enough more people into customers to build a business.

   Things to remember: As a startup, you can’t do multiple things well. As a business, you need to focus on the most desperate needs of your customer (which means you must choose your the customers who have desperate needs.)

2. Blue Ocean Strategy: How to Create Uncontested Market Space and Make Competition Irrelevant by W. Chan Kim and Renée Mauborgne

   Realization: You don’t need to compete on the same features and benefits that your competitors value.

   Things to remember: Don’t take the vectors you compete on for granted. Map yourself against the competition. Explore what would happen if you changed things.

3. Four Steps to the Epiphany by Stephen G. Blank

   
   Realization: This is your job: First, reduce risk. Next, prove value. Continuously validate. Don’t try to skip any of these steps.

   Things to use: Clear checklists for how to create a market, and a product and get customers. If any of your plans require outlaying tons of money and resources up-front, you’re probably wrong.

4. The Inmates are Running the Asylum: Why High-Tech Products Drive Us Crazy and How to Restore the Sanity by Alan Cooper

   Realization: Technology products need someone advocating for the user, and that someone is not going to be the engineers who build them.

   Things to use: If you’re in an organization where user experience design is nonexistent or under-empowered, this book lays out examples to help the product manager to step in as user experience advocate until the permanent resource is in place.(Can be heavy-handed and most of the examples are outdated, but still a great read.)

5. The Innovator’s Dilemma: The Revolutionary Book that Will Change the Way You Do Business by Clayton Christensen

   Realization: Listening to your customers works great for releasing new product versions… until it doesn’t. You can’t predict where markets will emerge.

   Things to remember:The key to your success will, in the short term, be incredibly destructive. In short: stay on your toes and be ready to experiment on barely-formed ideas rather than waiting for them to fully form.

6. Getting to Yes: Negotiating Agreement Without Giving In by Bruce M. Patton, William L. Ury, and Roger Fisher

   Realization: Negotiating instead of giving in doesn’t make you mean, it makes you effective.

   Things to remember: People want to feel heard and understood more than they need to ‘win’ in most situations.

7. Freakonomics: A Rogue Economist Explores the Hidden Side of Everything by Stephen J. Dubner OR Predictably Irrational: The Hidden Forces that Shape our Decisions by Dan Ariely

   
   Realization: People generally don’t understand cause-and-effect, don’t behave rationally, and worry about the wrong things.

   Things to remember: You (and your coworkers fall prey to the same competitive traps that your users do, but knowing this can help you avoid them.(Both books are good, but you probably don’t need to read both unless you’re a psych/socio hobbyist.)

8. Why Decisions Fail by Paul Nutt

   Realization: Really smart companies and people make decisions that seem ridiculous in hindsight.

   Things to remember: Recognizing potential problems can help you avoid them. If it’s too late for that, a more informed postmortem leads to better learning for next time.

9. Super Crunchers: Why Thinking by Numbers is the New Way to Be Smart by Ian Ayres

   Realization: Nothing measures user behavior better than … measuring user behavior. Online, everything is measurable.

   Things to remember: Make sure what you’re measuring is as close as possible to the results you want. As I’ve said before, make sure you I your KP. Remember that your competitors probably have more data than you do and be appropriately concerned about that.

10. The Creative Habit: Learn It and Use It for Life by Twyla Tharp

    
    Realization: Creativity doesn’t just happen, it requires hard work and tons of practice.

    Things to remember: Staying out of bad habits and ruts is a constant process.

Resources mentioned during the sessions I was in today:

• The Creative Habit: Learn It and Use It for Life by Twyla Tharp
• Primal Leadership: Learning to Lead with Emotional Intelligence by Daniel Goleman, Richard E. Boyatzis, and Annie McKee
• The Art of the Start: The Time-Tested, Battle-Hardened Guide for Anyone Starting Anything by Guy Kawasaki
• The Art of Possibility: Transforming Professional and Personal Life by Rosamund Stone Zander and Benjamin Zander
• Drawing on the Back of the Napkin: Solving Problems and Selling Ideas with Pictures by Dan Roam
• Implementing Lean Software Development: From Concept to Cash by Mary and Tom Poppendieck
• Predictably Irrational: The Hidden Forces that Shape our Decisions by Dan Ariely
• The Heart of Change: Real-Life Stories of How People Change their Organizations by John P. Kotter and Dan S. Cohen
• Venture Hacks website
• Publisher’s Lunch website

So step 1 is underway: I’ve created an online book club for product managers.   Please join! http://www.booksprouts.com/club/show/426?show_all=false

Currently reading: Predictably Irrational: The Hidden Forces That Shape Our Decisions

Online discussion begins: January 26, 2009