Better Product Managers, and Product Management

Surveys are an incredibly useful market and customer research tool.   But you use them too often. (Not, you know, you personally.  But ‘you’ in a global companies and organizations kind of sense.)

You shouldn’t use a survey if:

You aren’t sure which type of people you should ask to take your survey.

There are almost zero contexts where you’ll get useful data out of having “anyone” take your survey.   (And thinking, “I’ll just get thousands of responses, and then filter by some criteria later” is both a copout and unlikely.)   Do you want to hear from existing customers?  People from a certain region? People who share a common activity?  People with a specific job title?

Your “type of people” can be subjective, too — in a recent survey I conducted, I wanted to hear from was “smart people that we’d want to hire if we had the chance”.  So I distributed the survey through coworkers’ personal networks, letting them make that subjective determination.

You know which type of people you want to take your survey but have no idea how to find them.

Surveys are not an “if you build it, they will come” exercise.  Don’t waste your time on a survey if you don’t have a ready bank of people to send it to or a distribution strategy.   It’s a waste of time and social capital to send out a survey and get only 4 or 5 responses.

You’re better off conducting freeform interviews first so you can increase the “learning density” you get from each person.   For the long term, you’ll need to invest time in figuring out how to find more people.  This usually means “find where these people already are, and put yourself there” – participate in forums, join clubs, build up your network.

All of the questions you want to ask require a freeform response.

This is a sign that you don’t really know what the questions are yet, that you aren’t really sure what the problem is yet.

If you have an existing product or customer base, look at usage patterns and past feedback first.  Then start with freeform interviews, so you can tease out information in a back-and-forth context.  The survey format is bad for this type of learning.   People don’t like to write a lot, and even if they do, their first response is usually not where all the ‘meat’ is.

You don’t have a clear plan of action for how you’re going to use the results.

If you’re thinking “we might learn something” or “it would be nice to know…”, then save yourself the time (and save the time of the people who would fill out your survey).   Data without action is meaningless.

Is this information going to help you make a better decision on a specific feature or project?  Will it help you choose better wording or smarter defaults in your application?  Will it validate a specific hypothesis so that you can continue or pivot?

You can’t prioritize your list of questions down to fewer than 10.

This is another sign that ‘you aren’t sure what the problem is yet’.   Of course you have more than 10 things that you’d like to know — you probably have hundreds of things you’d like to learn — but you can’t possibly act on more than 10 at a time anyways.

I often see long surveys where the first 3-4 questions are asking about age, gender, zip code, income level.  Those are lazy questions.  If you need to segment by demographic, that should be part of your distribution plan or you should “buy an audience” from a market research firm.

You aren’t willing to write a draft, have another set of eyes review it, and then revise.

The first survey draft you write will suffer from at least one of these flaws:

• Unclear language (“wait, what are they asking?”)
• Biased language (“how much do you like feature X?”)
• Stilted language (technically makes sense but just sounds awkward)
• Stupid question (“do you want X?”  of COURSE they will say ‘yes’ – that doesn’t reveal anything about actual intent)
• Too many freeform questions (no more than 1 freeform for every 3 click-to-answer)
• Mistakenly using single-choice instead of multiple-choice, or vice-versa
• Doesn’t actually ask the question that you wanted answered
• Inadvertent rudeness (use of words with negative connotations, or a phrasing that sounds brusque or judgemental)

It is very difficult for you to catch these on your own.  I’ve been writing surveys for years, and I still always have at least one other person read it and comment on anything that is weird or confusing, and I still always have to change at least one thing.

Popularity: 2% [?]

January 26th, 2012   View Comments  

You can read part 1 here.

So how can you apply customer development techniques to large enterprise customers and existing customers?    They’re still people, so fundamentally they also have problems and pain points and constraints.  But there are some things to keep in mind if you want to maximize how productive your conversations are.

Large enterprise customers

• Expect anything they see to look ‘good’. This doesn’t mean that you need to show a working app — trust me, big company folks are plenty accustomed to being sold products by PowerPoint deck — but it does mean that you should spend a few hours or days turning your Balsamiq wireframes into visuals that are simple, specific, and polished-looking.   Don’t use lorem ipsum, take the time to write the actual text that a customer might see.  Don’t use clip art.  Correct your typos.   If you don’t, you’ll immediately take a credibility hit.
• Need a memorable narrative. These folks are constantly being pitched.  All the features and benefits and overblown language like “best-of-breed” and “cutting-edge” blend together into a haze.   What they’ll remember is a story: “Let’s say you have an employee, Jill, and here’s what Jill does each day when she…”
• Are accustomed to hearing “yes”. Sales tends to say “yes” to everything in order to close a deal; this means that saying “no” may end the conversation abruptly.  (This doesn’t mean you have to agree, it does mean you have to be very creative in how you proceed when the customer is asking for something that you have no intention of providing.)
• Are often pleasantly surprised by being asked “why”. All enterprise customers have been burned by hearing “yes” and then finding out that that “yes” has a lot of exceptions or additional cost.  Having someone who actually listens, and asks thoughtful questions, may shock them into revealing a lot more about how their business works.
• Offended by being told what to do. A multimillion dollar widget-making company does not want to hear from you that they’re making widgets wrong.  If you try to sound credible by flaunting your widget-making expertise, that is unlikely to go well.   Be humble: acknowledge that they’re the expert, and you’re the one trying to understand.  (Note: this is always good advice.)
• Have a lot of stakeholders. The end-user is probably not the buyer.  The decision-maker is probably not the implementer.  You will need to talk to a lot more people in order to validate your assumptions.   You will also need to approach them in different ways (the way marketers perform due diligence or assess your credibility is usually very different from how the IT department does it.)

Existing customers

• Are really easy to get in touch with. There’s a perception that customers “don’t want to be bothered”, and I don’t know where this comes from.  This has never been my experience!  As long as you’re clear that you’re not trying to sell something, current customers are usually eager to talk to you.  Remember, they have already invested a lot of effort in learning your product – it’s in their best interest for you to thrive as a company.
• Their top priority is their current product/service. Have you ever been to a concert where the artist kept playing all the songs from his new album instead of the greatest hits that you wanted to sing along with?   Start by answering their existing questions, and make sure you learn as much as possible about how they’re using your current product or service.  Once you’ve covered that, they will be far more receptive to answering questions about potential new use cases or products.
• Hate change. Hold off on the mockups for as long as possible.  You want the customer to recognize that they have a problem first, before you threaten them with something new and different.
• Are biased by what they’re already using. They have strong opinions on the solutions they want, and are highly motivated to push those solutions instead of talking about the problems.  It will require a lot of conscious effort to keep directing the conversation back to why/how questions.
• Need reassurance that you’re not going to de-prioritize or drop their current product. If they think that answering your questions might cause you to stop working on your current product (remember, the one they have already invested effort in learning), it’s going to be a short and pointless conversation.  You can cut this off before it starts by immediately reassuring them that their current product is safe, that this is early/exploratory research.

Hope this is useful in encouraging you to extend customer development to some new audiences.  If there are additional concerns or pratfalls you’ve encountered, I’d be happy to address them in a followup post.

Popularity: 5% [?]

January 19th, 2012   View Comments  

Customer development and lean startup techniques have been so loudly touted by, well, startups, that many people in non-startup or non-technology companies think these tools don’t apply to them.

“The rules are different when you’re talking to large enterprise customers,” they say.  ”This doesn’t work when you’re dealing with customers who are already using your products and have certain expectations,” they say.

Yes and no.  YES, the techniques I write about on this blog WILL work on your customers.    NO, you can’t blindly apply them in the same way to all types of customers.

While I’ve worked in technology startups throughout my career, I’ve actually spent much of that time working with large, traditional, non-technology corporations.  I’ve launched a few new products to new audiences — but I’ve also managed product redesigns and new features while working with outspoken and change-fearing existing customers.

There are some pretty big differences between these segments:  Consumers, Non-technology SMBs, Technology startups, Large enterprise customers, Existing customers.

This week I’ll talk about the first 3:

Consumers

• “Cold contact” methods are much less effective. Consumers tend to be more suspicious that you’re trying to sell them or scam them; they can also be fearful that a cold email means their privacy has been violated somehow.
• More likely to be polite than honest. Talking to a person in a consumer context is seen as more of a social engagement than a business one — and most people have been trained to be pleasant and avoid conflict in social situations.   Any question that can be answered with “yes” — will be.  ”How” and “why” questions will allow consumers to be more honest without feeling uncomfortable.
• Cost is more likely to be a big decision-making factor. In this context, your idea isn’t ‘validated’ unless you’ve seen concrete purchasing intent.  You can easily get consumers to identify their problem, give you details, and loooooove your solution… and then not pay you $5 for it.
• Time is undervalued. Most people are terrible at estimating how much time they spend on a certain task, how much time they waste, how long they spend on various activities.   In a work environment, we are more likely to have deadlines, be paid hourly, fill out timesheets – any of those activities make people at least somewhat more aware of their time usage.
• Don’t forget about “external stakeholders”! Just because a consumer doesn’t need to seek approval from their boss or finance department doesn’t mean there aren’t additional people who can and will veto your product.  Family members may refuse to use a new product; friends may express disapproval of changes.

Small and mid-sized non-technology businesses

• Be specific. Asking about specific tactical activities and outcomes is more likely to engage SMB folks than aspirational/”vision” concepts.
• Less motivated by “feeling smart”. Consumers, tech nerds, and people within larger companies are often happy to talk about a subject just because they care about it / are a subject matter expert.  I’ve found this to be less the case with SMB folks.  Offer them value like answering questions, sharing data, sharing curated content, making introductions for them.
• “If it ain’t broke, don’t fix it” mindset. If consumers are more likely to say “yes” (even if they don’t really mean it), SMB people are more likely to say “no”.  There are fewer people to absorb the impact of a learning curve or cleaning up a failed experiment.

Technology startups

• Harder to keep them focused on PROBLEM not solution. We can’t help it.  We spend our days building and designing, so we naturally slip into talking about solutions.  You’ll need to actively refocus the conversation, maybe many times, on the what/when/why questions.
• Easiest to reach via “cold contact” methods. As long as you’re brief, non-spammy, and respectful, you’ll get a high response rate from emails, Twitter, and follow-ups from finding people on social media/topical forums.
• Overly optimistic/aspirational. We set really high goals for ourselves, which often means we’ll tell you about all the things we want to do, intend to do, aspire to do … but never actually get around to.    Like the consumer segment, if we can answer ‘yes’ to a question, we will.
• We like sounding smart AND information. You can get an initial conversation with a technology startup person just because we love talking and thinking about stuff.  But you can build an ongoing relationship by following up with a summary of your learnings, intel on what other companies are doing (anonymized, of course), or offering us sneak previews of what’s coming next.

Next week I’ll cover the different beasts: large enterprise customers and existing customers.

Popularity: 5% [?]

January 12th, 2012   View Comments  

The biggest risk in hiring product people is…  THE UNKNOWN.

• What if we hire this person and they aren’t able to execute?
• What if they can’t negotiate with our engineers?
• What if they can’t operate under conditions of uncertainty?
• What if they worry over every little detail until they’ve totally missed the big picture?
• What if they just say “yes” to everything so we lose our focus?
• What if they blithely accept our broken processes and bad habits so that we keep making the same mistakes?

In the past month, I’ve spent quite a bit of time being the interviewee, the interviewer, and the resume mentor.  And the biggest mistake that I’ve made myself and seen others make is in being too neutral.

You don’t want to make a bad first impression.  You don’t want to come across as someone argumentative and difficult to work with.  You want to make it clear that you’re flexible and adaptable.

And that leads you into exchanges like this:

Interviewer: “Which development methodology do you prefer working with?”
PM Candidate: “I’ve worked in companies where we did waterfall, as well as agile, as well as daily releases, and I’ve been able to be effective in all of them.”

The interviewer thinks: If you were working closely with engineers and customers, you’d have directly felt the pain of what didn’t work.  Either you were pretty hands-off (bad) or you don’t reflect on your work and process to see how you can drive improvement (also bad).

Interviewer: “Tell me about a time when you’ve dealt with conflict between product management and engineering/design/sales.”
PM Candidate: “I believe in working very closely with other teams and keeping the lines of communication open so we can avoid conflict.”

The interviewer thinks: Seriously? There are always differences of opinion and conflicting priorities.  Either you just say “yes” to everything (bad), or you are oblivious to the fact that sales hates you (also bad), or your upper management takes all decisions out of your hands (maybe not your fault, but doesn’t make me want to hire you).

Interviewer: “What type of difficulties do you foresee in working with our product/culture/customers/distribution model?”
PM Candidate: “I’ve worked with X before, so I’m very confident that I’ll be able to handle the job here.  Here’s an example of a related awesome thing I’ve done in the past.”

The interviewer thinks:  If you are a good candidate, this isn’t your only job option.  You should be critically thinking about the “cons” of this job and why you might not want it.  If you haven’t, that’s a warning sign.  If you have, but you’re not comfortable articulating them, how are you going to handle day-to-day communication and negotiation here?

It’s true that expressing your opinion may lose you a potential job.  If you’ve eloquently detailed how you advocate user-centered design and incorporating customer research into the development process, the company where engineers call all the shots, is not going to offer you a job.

This is not a bad thing.  You wouldn’t do great work at that job anyways — and how are you going to land your next job with a bank of mediocre work and no positive references?  Better to dodge that bullet now.

If you have work experience, you have opinions.   As a hiring manager, I want to hear them — as well as the context that you formed them in and how you arrived at them.  Maybe you dislike a certain process because you’ve been burned by it in the past but you can see how it could be beneficial… or maybe you’re just closed-minded and resistant to change.

No one worries that you’ll be flat-out incompetent — if you were, it’d be blatantly obvious after a couple weeks, your manager could document your flaws and have you out in no time.  But an employer is petrified that you’re going to be not-quite-right in more subtle, insidious ways that won’t be noticeable until it’s too late and you’ve botched a product release or customer relationship.

The tricky thing about product managers is that every candidate is different, and every past environment is different.   Just because the last product you managed was a runaway success doesn’t guarantee that you were the reason why — maybe it was “right place at the right time”, maybe your boss made all the decisions, maybe it succeeded in spite of you.  Same thing with product flops — maybe the market tanked, maybe your boss made all the (wrong) decisions despite your recommendations, maybe there was an engineering disaster.  So your resume doesn’t really set the hiring manager’s mind at ease.

It’s your job as a candidate to make the hiring manager feel better about those risks.  And how do you do that?  By being opinionated.  Give examples.  Explain the “why” and “what I learned”.   Write blog posts.  Answer questions on Quora.  The less of a “black box” you can be, the better candidate you’ll be.

Popularity: 11% [?]

December 2nd, 2011   View Comments