Better Product Managers, and Product Management

It would be nice if we could just say “give us feedback” and our customers could just turn on the faucet and dump out all of their concerns and experiences and ideas.  But that’s not the way our brains work.

Let me illustrate.

Quick, without Googling, what were the venues for the last 4 Summer Olympics?

If you’re like me, a neat list of 4 cities did not immediately pop into your head.  In fact, even though I’m a pretty die-hard Olympics-watching fan, my brain is struggling to remember anything before Beijing.

But what about if I asked the question in this way?

From the list below, pick the venues for the last 4 Summer Olympics:

• Seoul
• Beijing
• Los Angeles
• Montreal
• Athens
• Rio de Janeiro
• Sydney
• Tokyo
• Johannesburg
• Atlanta

Easier this time, right?  Beijing, Athens, Sydney, Atlanta.

That’s an illustration of recall vs. recognition in human memory.

Recall memory involves delving into your brain, looking for specific pieces of information, finding them, confirming that they are the information you wanted, and then putting them to use.

Recognition skips the first three steps.  You are somehow prompted with the specific pieces of information, then all you have to do is confirm that they are correct and put them to use.

In both cases, the information is stored somewhere in your brain.  If I had never heard of the Olympics, I wouldn’t be able to pick the correct cities out of a list – all of the list items would’ve been similarly meaningless to me.

Why is this important?

As you might have guessed, recall memory is harder.  It’s legitimately harder for people to answer a question that doesn’t provide any contextual prompts.

If I used your product on Monday, and you email me on Wednesday saying “Any feedback?”, I’m likely to draw a blank.  Not because I don’t care about you, not because I’m a “stupid user”, but because I’ve got no context to use to dig into my recall memory.

Recall memory is like asking your brain to do push-ups.  Some people can’t do many or any pushups; others can, but we’d rather procrastinate and get around to it later.

So how do I ask for feedback in a “brain-friendly” way?

Use recognition.

Instead of asking “what should we build next?“, ask “which of these new features [list] would be most helpful to you, and why?” (Not only is this “brain-friendly”, it’s also “product manager-friendly”, allowing you to guide their feedback in alignment with your long-term product vision.  There’s no value in letting people ask for faster horses over and over again if you’re building Model Ts.)

If you suspect a specific page is confusing, don’t ask “what about the page was confusing?”, show them a picture of the page and let them point to the confusing parts.

Give customers a sentiment and allow them to agree (or disagree) and elaborate.  For example, “some customers felt this workflow was confusing – why do you think that is?”

Use immediacy.

If you ask a question while the customer is using your site, there’s nothing to remember – they just did, or are still doing, the thing you’re asking about.

In my experience, immediacy cancels out one of the biggest complaints I hear about customer feedback: “it’s useless because it’s all extremes, 1 star or 5 stars!”   This makes perfect “brain sense” – after you’ve completed an action or experience and time has passed, your memory becomes distorted towards the elements that left you with the biggest emotional impact – whether that’s joy at a gorgeous interface, or (more likely) fury at a confusing workflow.

But while your customer is experiencing something, they’re going through a series of impressions:  Ooh, nice refresh! Hmm, I don’t understand that sentence.  Oh good, this will do exactly what I wanted.  Wait, did that charge my credit card or not?  I wish this workflow was more straightforward.  This will make my life easier.  How do I undo what I just did?  Oh no!  Oh, phew, I guess it’s okay.

Wouldn’t you rather overhear that entire stream of impressions vs. a 3-star rating or “yeah, I kind of like your site”?

Use interactivity.

It’s not always possible (and sometimes downright intrusive) to get customer feedback while they’re in the middle of trying to get something done.

The next best method is to be interactive.  In-person, phone, or IM chat interviews allow you to provide context and prompts to make it easier for your customer to give you feedback. You can describe a feature “do you remember using the ’sort and select’ feature? Let me describe it.  OK, now tell me about how you used it…”  and then it’s far easier for the customer to provide valuable feedback.

(While this method is primarily qualitative, you can ask for 1-5 ratings on the phone just as easily as in a survey.  Even better, because you can confirm that the customer really understands what they’re rating instead of just picking “3″ because they don’t remember what that feature was.)

Use pictures.

There are times when you need more responses than you can scalably get through interviews.  In those situations, illustrate!  Before asking a question, include a screenshot or even a quick video, to remind customers what they’re responding to.

Unfortunately, neither SurveyMonkey nor Wufoo nor Google Forms support embedding images in between questions by default, so this is less easy than it should be.  UserTesting.com is one option, or you can hack something together with multiple Wufoo or Google Forms pages and stick your own images in between.

What’s your biggest obstacle to getting customer feedback?

For one thing, our household finally has one of the former (you may have noticed this blog was on hiatus for a few weeks).

What else?  Generally, users are afraid of doing something wrong that will damage them.  They tend to give obscure (if any) feedback on how well you’re handling them. It’s hard to tell whether a little quirkiness is normal or an issue you should be rushing to customer support for emergency help.

The other thing they have in common is that product managers tend to react with a “more information is better” attitude.  Here! Read this incredibly dense user manual / 500 pages of What to Expect The First Year!

Because, of course, nothing says reassurance like making an already distraught user think and filter and worry while they’re trying to cull out that piece or two of information they need to feel okay.

So the product/design geek in me was incredibly pleased by the worksheet that Kaiser sent us home with.

Here’s what it does right:

• PRIORITIZES.  Only TWO things that you need to worry about with a newborn.
• KEEPS IT SIMPLE: All you need is a pencil and the ability to draw circles.  (Now is NOT the time to make your users worry about forgot passwords or flaky internet access!)
• CONTACT US: The phone number if something goes wrong is prominently placed and easy to remember.

It’s easy to get caught up in the technology and think with an all-or-nothing approach – we need to provide interactive help / online and searchable / beautifully designed.  What would our users think if we included a plain, simple, black-and-white text-only sheet of paper?

They’d probably be grateful.

I’m happy to report that we’ve made it through Day 14 and circled all the circles.  It’s good to be back!

Whether it ended in a job offer or a “no thanks”, when is the last time you had a job interview for a Product Management role that you felt actually addressed your ability to do the job?

Asking about past accomplishments doesn’t separate out your role from the environment (could you do it again in a different team, in a different industry?).  Asking for PRD writing samples proves that you’re literate, but doesn’t tell you anything about how well those requirements were understood or implemented.

As Eric Ries writes in a recent VentureBeat post:

I’m not interviewing for the right answer to the questions I ask. Instead, I want to see how the candidate thinks on their feet, and whether they can engage in collaborative problem solving. So I always frame interview questions as if we were solving a real-life problem, even if the rules are a little far-fetched.

These 8 questions are things I’d want to know if I were hiring a product manager.

They are biased towards smaller companies/more startup-like environments – they assume a breadth of responsibilities that may not be relevant for a big-company Product Manager.  But since there are no “right answers”, and since I’d expect any capable Product Manager to be able to think on their feet, I’d say they’re all fair game.

1. (more…)

How can you change people’s perceptions?  Show them (or hint to them) that other people feel differently.  Think of the classic Tom Sawyer, making whitewashing look so appealing that soon he has half the town bribing him to take their turn at it:

“Like it? Well, I don’t see why I oughtn’t to like it. Does a boy get a chance to whitewash a fence every day?”

Scarcity makes things more appealing.  A couple weeks ago, there were Girl Scouts camped out near the BART exits on Market Street – with commuters lining up to buy boxes.   I know, because I was one of them.

Feeling like other people know something you don’t – also makes things more appealing.

IBM banner ad with a "replay" prompt

I was flipping between browser tabs yesterday when I saw this ad.   I hadn’t seen what played previously and I don’t particularly care about travel congestion or IBM.  But that “replay” call to action jumped out at me immediately – did I miss something? There must be some reason why I’d want to watch this!

It’s funny that this would work on me (yes, I clicked replay and watched the ad) – because I’m testing out the same concept in Loomia’s recommendations module.

I know that “popularity” is a strong driver for people to read an article or watch a video – but I’ve also seen through A/B testing that a predictive algorithm that takes context and similar readers’ behavior into account is a stronger driver.  What if we could combine them?

For all articles over a certain “popularity threshold”, we added an X People Viewed This annotation.

What this implies: not only is this probably something you want to read, but lots of other people read this.

What users think: 25,000+ people can’t be wrong.  What if I’m missing out on something useful?

The potential social cost of not being “in the know” outweighs the time cost of reading the article.