Better Product Managers, and Product Management

Last week, I wrote a blog post in which I said,

The answer to any question that starts with “do you want” or “are you concerned about” will always be “yes”.

I wanted to expand on that a little bit more, because most product managers will need to understand whether their customers really do value a certain feature or value, and they’ll need to understand the obstacles that potentially prevent someone from becoming a customer.

The reason for this phenomenon is that “wanting” is free.  It doesn’t cost your prospective customer anything to “want” feature X, customization Y, added security, privacy, or fiber.

To change this dynamic, you need to make “wanting” no longer free. (A more sophisticated version of this is conjoint analysis; but I’m just going to talk about the simplest level.)

How do you put a cost on customers “wanting”  things?

• Stop asking yes/no questions.   It’s way too easy to say “yes”.
• Insist upon details. This doesn’t mean asking for customers to define the features they want (99% of them won’t be able to anyways), but ask how they would use it, in what situations it would be beneficial, how their life is worse without it.*
• “Think out loud”a cost. “So, if this would regularly save you 2 hours a week, and your time is worth at least $20 an hour, then you’d be gaining $40…?”  (Note: this is not asking people what they’re willing to pay, which they won’t answer anyways, but it gives you some insight into order-of-magnitude — if someone goggles at this question, it probably means they were thinking they’d pay $4, max.)

* This caught me out just the other day: someone had me on the phone for a customer development interview and I said I was interested in some feature.  Well-trained, the interviewer asked me to tell him more about how it would make my life better… and I totally drew a blank.  You caught me, I said, theoretically it sounds interesting but I can’t tell you why, which means I probably wouldn’t pay money or change my behavior to get it.

One of my favorite anecdotes around “wanting” dates back several years, to when I was working with a client on usability tests.  My team had produced the prototypes and handled all of the administration around getting and compensating participants; the client’s user researcher was on hand to ask additional questions (and spy on us because she was clearly suspicious that we’d managed to set up in 1 week the type of testing sessions that typically took her team months.)

The app we were testing was in personal finance, so the user researcher insisted that we ask about privacy.

Despite my objections that constructing a yes/no question would get biased data, the user researcher asked each participant, “Are you concerned about privacy when it comes to your financial information and identity?” Of course, they all said “yes!”

As the final person said yes, I jumped in: “Thanks for participating in our test.  We have your $50 check ready for you – here an evil little glint appears in my eyes — you just need to write down your social security number and mother’s maiden name for our records.”

I waved the check; the guy said “OK,” and reached for the sheet of paper and pen.

(I didn’t actually let him write it down! But here he was, “very concerned about privacy…” — but not concerned enough to give up $50 for it!)

Popularity: 8% [?]

When you see a dropoff like this in one of your product workflows, it’s time to spring into action.

Here are the first 5 questions you should ask yourself:

1) Where is the dropoff happening?

Where the dropoff happens usually provides some clues as to why it’s happening:

• After the homepage: your homepage is crappy OR your potential customers are unqualified
• After being asked to pay: no one likes to pay, but more likely you haven’t communicated WHY they should well enough
• After being prompted to install or configure something: no one likes to spend time, but more likely you haven’t communicated WHY they should / reassured them that it’s safe and easy
• After step 4 or 5: seriously, how many more steps are you going to make them go through?

2) Can I spot the problem?

I shouldn’t even have to mention this, but I know I do.  Go through your own product slowly and look at the place where you’re losing people.  Is there a glaring browser bug?  Is the call-to-action button missing?  Is there some text missing that makes the flow really confusing?

If you know your product “too well”, it can be easy to miss even the giant gorilla running through the basketball game. A good way to overcome this is to sit down with one of your coworkers and “demo” the product to them, moving slowly and narrating your actions and reading the on-screen text out loud.

I just did a recent walkthrough of my own product and — embarrassingly — unearthed some awful copy, a call-to-action that was tiny and below-the-fold, and a payment flow that dead-ended.   That led to some very quick changes that we could roll out even faster than proceeding to user feedback.

3) Is this a “usability” problem or a “decision” problem?

A usability problem means that customers can’t find the next step, can’t read the text because it’s too low-contrast, don’t know which widget to pick, can’t find the information on the page.  You can usually learn about usability problems by putting any random “new” person in front of your software and asking them to complete a task while narrating it out loud.  Throw usertesting.com at the problem or offer to buy someone their coffee at Starbucks if they’ll sit down with your laptop for 5 minutes.

A decision problem is where your customers are making a decision — to install your software, to sign up for your trial, to pay you money, to entrust you with their data, to contribute content to your site.   The decision they make is highly dependent on their situation, how they perceive you, and what expected value they think they will get from you.   You’re going to have to ask real customers, preferably while they’re on this page.   Ask a KISSinsights question like “Is there anything stopping you from completing this purchase?” or “What additional information do you need in order to continue?”

4) Can I make the text shorter and clearer?

Writing good copy is hard, but it’s usually the fastest change you can make to your site.  Based on the insights you got from your user test or survey question, what can you change about the existing text to make the purpose of this page clearer and easier to understand?  And shorter — trust me, almost always, is better.

5) Can I add a screenshot or a video?

Most of the dropoff problems I’ve seen could’ve been improved with some more visuals.  Even if the “real” solution is to reduce steps in the workflow or totally change how the customer interacts with that section, a simple picture or video can be a good stopgap measure.

Popularity: 10% [?]

Customer development, by its’ very nature, is not a scalable process.  This means that those of us who practice it have an excuse to use somewhat slipshod means of scheduling and conducting interviews and analyzing results.  (Yes, I’m looking at myself.)

But “not scalable” doesn’t mean “not a repeatable process”.

Here’s what I did to bring a bit more process to our process and get more of our team doing customer interviews.

Document the questions.

If you’ve been doing interviews for awhile, you should already have a pretty good idea which questions are working in terms of effectively eliciting useful feedback.  Write them down.

Explain what you’re hoping to learn from each question.

Literally, walk through each question with your team and say “I ask [question] because I’m hoping to learn things like X and Y”.

As you go, you’ll probably remember the additional prompts you use: “Sometimes, if the interviewee isn’t sure, I ask [question] as a follow-up or give [example].”   Don’t skip this step.  These prompts are as important as your question list.

Make it easy.

I put all of our questions into a Google Form and added an extra “other” question at the end.  This way, the interviewer can call up the form and take notes directly into it.  Having the question blanks ready makes it easy to take notes if the interview jumps around instead of addressing the questions in order (which often happens).  It also serves as a reminder that the interviewer skipped a question and needs to return to it.

(Note: we do most of our interviews by phone, and our customers are other web companies, so even when we do face-to-face interviews, typing notes into a laptop is pretty much expected.  If you are doing face-to-face interviews with non-techies, I’d recommend making physical paper templates instead, printing them out, and handwriting notes.  Yes, you’ll have to transcribe them again later, which is a pain, but if your interviewee perceives you as rude for having your face in a computer, you won’t get useful feedback from them anyways.)

Create an interview request template and process.

I prefer to compose email requests individually, so they sound more personal.  So this is more of a checklist of concepts you want to be sure to include:

• Introduce yourself
• Talk about why you want to talk to them
• Emphasize how helpful it will be to you
• Put limits on what you’re asking for (“this won’t take longer than X, you don’t need to prepare”)
• Provide a clear next step (“does one of these 4 times work for you?”, “you can schedule any time that’s convenient for you on my calendar here”)

I recently started using Tungle, which has probably resulted in a 25% increase in responses over my previous method of “pick one of these times”.  (Just saying something open-ended like “any time next week” has had very low response rates for me.)  I don’t think Tungle is perfect, and it does require the customer to do some work, but it gives them a clear next step and emphasizes the “at your convenience” part.

Do a ride-along.

I had my coworker listen in while I did the first interview with the agreed-upon list of questions.  It gave him the opportunity to see how the questions went and how I handled tangents, and practice taking notes while listening.

Review the first notes.

After my coworker did his first interview, I read his notes and then the two of us got on the phone to discuss.   This gave me the chance to point out where I would have asked a follow-up question or tried to keep the customer talking on that topic.

I also was able to ask about emotion (“what was his reaction to this? was he excited or didn’t seem to care?”) – which was a good reminder to both of us that annotations like “!” or “very excited” or “really frustrated” are as useful as the actual words the customers are saying.

The step I’m still working on is how to best collect the responses in a readable format.   The Google spreadsheet that all of our responses feeds into does a good job of keeping all the notes in one place, but it’s damned near unreadable.  For now I’m basically copying and pasting each column into its’ own Excel sheet with one super-wide column.  This is tedious.

Nonetheless, this is the most organized I’ve been about my notes in a long time, plus we’ve near-doubled our interview velocity, which is pretty exciting.

Popularity: 5% [?]

This is another post about “later-stage customer development”.

You’ve probably heard the “medicine vs. vitamin” analogy of products — that is, to maximize your chances at success, it’s easier to sell medicine (that fix a specific, painful ill) than vitamins (that offer a vague promise of ‘feeling better’).

But I’ve realized there’s another category of product – the “gym membership” product:

• Your customers have accepted that they have a problem they really want to deal with (even if it’s not ‘life-or-death’).
• Of course there’s a free alternative, and some people get along just fine using it, but most people need the accountability, the support, and the motivation of a paid solution.
• Some people will use it religiously every day, some will use it once a week, and some will use it gung-ho for a week and then peter out for a few months.
• Some people will actually never use it, even though they keep paying you.  They’ll kind of resent you for it, though.
• There are a lot of different ways to use it to meet your goals. Some people walk in full of purpose, and know exactly how they need to use it.
• Most people need suggestions on how to use it effectively.  Otherwise they use it poorly or just wander around aimlessly.  They’ll be unsatisfied and rate you poorly.
• But if you force people through a long “initiation” session, they’ll find that obnoxious.
• Once people feel comfortable using it, they’ll find it gives them a lot more energy.  They’ll wonder how they ever got along without it!

You could also call these aspirational products.  We sign up because we want to be that kind of person.

Unlike most products — where the first and biggest challenge is getting people to give a damn enough to give you five seconds of their attention  — it isn’t that big a challenge to get people to sign up.   They see the equivalent of “better body in 90 days” and that’s enough to get them to click.

The challenge is getting people to get started and come back.   To do that, you have to anticipate and answer customer questions.  And the more subtly you do this, the more your app will feel intuitive/”just works”/delightful.

These are the first two questions that I’m trying to answer right now for both KISSmetrics and KISSinsights.

Question 1: What do I do first?

You probably have a pretty good sense for, in an ideal world, what you wish your customers would do first in order to get the most value out of your product.  However, you may be wrong.  This is where, now that you’re in later-stage customer development, it can be tremendously useful to walk someone through using your product the way you think it should be used.

One of two things is likely to happen: they’ll say, “oh, that makes sense – why doesn’t the app tell me I should start by doing that?” or they’ll say “that doesn’t make sense – why can’t I do X first?”

For KISSmetrics, we heard a lot more of the former, so we built a first-user experience designed to tell you how you should start.  Of course, we’re continuing to realize that there’s a lot more the app could be “telling” people, so we continue to tweak.  For KISSinsights, I suspect the latter is more accurate, which means we’ll need to make bigger changes to the first-user experience.

Question 2: How are other people using it?

I think this question really encompasses three things: “I want proof that other people are actually getting value from this”, “I want to see how much trouble they had to go through to get results”, and just plain curiosity.

I posted some examples for KISSinsights, which seems to have been a big help already (judging by the number of emails I’ve gotten since then).

One thing I’m still pondering, though — how do you identify the “goes to the gym every day” customer vs. the “tries to make it once or twice a week” customer, when they’ve just started to use your product?

Dealing with these questions has meant a very different kind of customer interview.  I wrote before that I was wary of telling the customer “here’s our product and here’s how you should use it” – now I think the way to think about it might be:

“Traditional” product development: “Let me tell you about us”

Early-stage customer development: “Tell me about you”

Later-stage customer development: “Let me tell you about other people like you“

Popularity: 2% [?]