Better Product Managers, and Product Management

Surveys are an incredibly useful market and customer research tool.   But you use them too often. (Not, you know, you personally.  But ‘you’ in a global companies and organizations kind of sense.)

You shouldn’t use a survey if:

You aren’t sure which type of people you should ask to take your survey.

There are almost zero contexts where you’ll get useful data out of having “anyone” take your survey.   (And thinking, “I’ll just get thousands of responses, and then filter by some criteria later” is both a copout and unlikely.)   Do you want to hear from existing customers?  People from a certain region? People who share a common activity?  People with a specific job title?

Your “type of people” can be subjective, too — in a recent survey I conducted, I wanted to hear from was “smart people that we’d want to hire if we had the chance”.  So I distributed the survey through coworkers’ personal networks, letting them make that subjective determination.

You know which type of people you want to take your survey but have no idea how to find them.

Surveys are not an “if you build it, they will come” exercise.  Don’t waste your time on a survey if you don’t have a ready bank of people to send it to or a distribution strategy.   It’s a waste of time and social capital to send out a survey and get only 4 or 5 responses.

You’re better off conducting freeform interviews first so you can increase the “learning density” you get from each person.   For the long term, you’ll need to invest time in figuring out how to find more people.  This usually means “find where these people already are, and put yourself there” – participate in forums, join clubs, build up your network.

All of the questions you want to ask require a freeform response.

This is a sign that you don’t really know what the questions are yet, that you aren’t really sure what the problem is yet.

If you have an existing product or customer base, look at usage patterns and past feedback first.  Then start with freeform interviews, so you can tease out information in a back-and-forth context.  The survey format is bad for this type of learning.   People don’t like to write a lot, and even if they do, their first response is usually not where all the ‘meat’ is.

You don’t have a clear plan of action for how you’re going to use the results.

If you’re thinking “we might learn something” or “it would be nice to know…”, then save yourself the time (and save the time of the people who would fill out your survey).   Data without action is meaningless.

Is this information going to help you make a better decision on a specific feature or project?  Will it help you choose better wording or smarter defaults in your application?  Will it validate a specific hypothesis so that you can continue or pivot?

You can’t prioritize your list of questions down to fewer than 10.

This is another sign that ‘you aren’t sure what the problem is yet’.   Of course you have more than 10 things that you’d like to know — you probably have hundreds of things you’d like to learn — but you can’t possibly act on more than 10 at a time anyways.

I often see long surveys where the first 3-4 questions are asking about age, gender, zip code, income level.  Those are lazy questions.  If you need to segment by demographic, that should be part of your distribution plan or you should “buy an audience” from a market research firm.

You aren’t willing to write a draft, have another set of eyes review it, and then revise.

The first survey draft you write will suffer from at least one of these flaws:

• Unclear language (“wait, what are they asking?”)
• Biased language (“how much do you like feature X?”)
• Stilted language (technically makes sense but just sounds awkward)
• Stupid question (“do you want X?”  of COURSE they will say ‘yes’ – that doesn’t reveal anything about actual intent)
• Too many freeform questions (no more than 1 freeform for every 3 click-to-answer)
• Mistakenly using single-choice instead of multiple-choice, or vice-versa
• Doesn’t actually ask the question that you wanted answered
• Inadvertent rudeness (use of words with negative connotations, or a phrasing that sounds brusque or judgemental)

It is very difficult for you to catch these on your own.  I’ve been writing surveys for years, and I still always have at least one other person read it and comment on anything that is weird or confusing, and I still always have to change at least one thing.

Popularity: 4% [?]

The biggest risk in hiring product people is…  THE UNKNOWN.

• What if we hire this person and they aren’t able to execute?
• What if they can’t negotiate with our engineers?
• What if they can’t operate under conditions of uncertainty?
• What if they worry over every little detail until they’ve totally missed the big picture?
• What if they just say “yes” to everything so we lose our focus?
• What if they blithely accept our broken processes and bad habits so that we keep making the same mistakes?

In the past month, I’ve spent quite a bit of time being the interviewee, the interviewer, and the resume mentor.  And the biggest mistake that I’ve made myself and seen others make is in being too neutral.

You don’t want to make a bad first impression.  You don’t want to come across as someone argumentative and difficult to work with.  You want to make it clear that you’re flexible and adaptable.

And that leads you into exchanges like this:

Interviewer: “Which development methodology do you prefer working with?”
PM Candidate: “I’ve worked in companies where we did waterfall, as well as agile, as well as daily releases, and I’ve been able to be effective in all of them.”

The interviewer thinks: If you were working closely with engineers and customers, you’d have directly felt the pain of what didn’t work.  Either you were pretty hands-off (bad) or you don’t reflect on your work and process to see how you can drive improvement (also bad).

Interviewer: “Tell me about a time when you’ve dealt with conflict between product management and engineering/design/sales.”
PM Candidate: “I believe in working very closely with other teams and keeping the lines of communication open so we can avoid conflict.”

The interviewer thinks: Seriously? There are always differences of opinion and conflicting priorities.  Either you just say “yes” to everything (bad), or you are oblivious to the fact that sales hates you (also bad), or your upper management takes all decisions out of your hands (maybe not your fault, but doesn’t make me want to hire you).

Interviewer: “What type of difficulties do you foresee in working with our product/culture/customers/distribution model?”
PM Candidate: “I’ve worked with X before, so I’m very confident that I’ll be able to handle the job here.  Here’s an example of a related awesome thing I’ve done in the past.”

The interviewer thinks:  If you are a good candidate, this isn’t your only job option.  You should be critically thinking about the “cons” of this job and why you might not want it.  If you haven’t, that’s a warning sign.  If you have, but you’re not comfortable articulating them, how are you going to handle day-to-day communication and negotiation here?

It’s true that expressing your opinion may lose you a potential job.  If you’ve eloquently detailed how you advocate user-centered design and incorporating customer research into the development process, the company where engineers call all the shots, is not going to offer you a job.

This is not a bad thing.  You wouldn’t do great work at that job anyways — and how are you going to land your next job with a bank of mediocre work and no positive references?  Better to dodge that bullet now.

If you have work experience, you have opinions.   As a hiring manager, I want to hear them — as well as the context that you formed them in and how you arrived at them.  Maybe you dislike a certain process because you’ve been burned by it in the past but you can see how it could be beneficial… or maybe you’re just closed-minded and resistant to change.

No one worries that you’ll be flat-out incompetent — if you were, it’d be blatantly obvious after a couple weeks, your manager could document your flaws and have you out in no time.  But an employer is petrified that you’re going to be not-quite-right in more subtle, insidious ways that won’t be noticeable until it’s too late and you’ve botched a product release or customer relationship.

The tricky thing about product managers is that every candidate is different, and every past environment is different.   Just because the last product you managed was a runaway success doesn’t guarantee that you were the reason why — maybe it was “right place at the right time”, maybe your boss made all the decisions, maybe it succeeded in spite of you.  Same thing with product flops — maybe the market tanked, maybe your boss made all the (wrong) decisions despite your recommendations, maybe there was an engineering disaster.  So your resume doesn’t really set the hiring manager’s mind at ease.

It’s your job as a candidate to make the hiring manager feel better about those risks.  And how do you do that?  By being opinionated.  Give examples.  Explain the “why” and “what I learned”.   Write blog posts.  Answer questions on Quora.  The less of a “black box” you can be, the better candidate you’ll be.

Popularity: 11% [?]

Years ago, when I was new to cooking, this happened to me a bunch of times:

• Invite people over for dinner
• Find a recipe, skim it, buy the ingredients
• Late afternoon, start cooking–
• …and realize that I’d missed the part where something needed to rise for 12 hours, or chill for 4 hours, or roast for 4 hours…

In other words, this dish was not going to be ready for dinner, no matter how badly I needed it.  I’d simply overlooked that there was a long cooking time, and now I had a choice of trying to make the best of a half-cooked entree or quickly whipping together an inadequate substitute.

Either way, my dinner guests were not going to be happy.

There are a lot of things in product management that have a long cooking time.  And somehow, we overlook that, until we really, really need whatever it is.

Some of these things are:

• Case Studies
• Educational content
• Reference customers
• Usage and retention metrics
• Revenue and customer lifetime value metrics

You probably don’t have all of these today (even I don’t have these to the degree I’d like to).  You may think you’re not ready (“I can’t think about case studies yet – I just released my beta last week”) — but you are ready to make a plan.

Ask yourself, what can I do today to make sure that I have this data 6 months from now?   Work backwards and write everything down, no matter how obvious it may seem:

To have case studies, you need successful customers who like you enough to collaborate with you.  To get there, you need:

• to have developed a good working relationship with customers
• to have identified customers who had the potential to be successes and nurtured them and worked with them to make sure they do succeed
• to know “what a successful customer looks like”

Even if you just launched your beta last week, you can start with that bottom rung: what do you think a successful customer might look like?  Write down your hypothesis.  That’s the first step.

For any of these “long-cooking” items, it may seem like a long, long ways off.  But 6 months from now, or 12 months from now, will come a lot faster than you think.  Start reading the recipe now.

Popularity: 5% [?]

You have a target market.  You do customer development.  But do you have personas?

Personas are, in my opinion, the often-neglected third leg of the stool in creating awesome products.  They live in between individual customer interviews – Bob X. says this — and a wider target market profile — we’re targeting small businesses making over $10MM revenues with these characteristics.

Persona Analysis is the process of identifying individual person representatives for your audience and fleshing out their traits, likes and dislikes, frustrations and limitations.  Personas are frequently used in the interaction design / user experience worlds, but much less so in product management.

For any product, there are different types of people who:

• need your product / have already bought your product
• are able to get value from your product
• have money to spend on your product

These different types of customers have different job titles, priorities, responsibilities, and technical abilities.   These differences affect how they decide to buy your product, what frustrates them or excites them about your product, and how you can help them start using your product quickly/effectively.

If you’re a startup, you are ready for personas as soon as you start hearing different types of answers from your interviews or you start witnessing different types of behaviors among your users.  (“Gita said she’d never use Wizard X because she’d rather configure it manually” / “Devon loves Wizard X because he finds manual configuration too difficult!”)

What does a persona look like?

Some people spend a lot of time giving their personas pictures, names, and back-stories.   To me, the only reason to spend time on that is if helps your team connect with your personas.  (In my experience, too much “marketing” of personas makes engineers distrust them.)

Here are the questions I asked to flesh out our personas.  They are geared towards a B2B software product, but many are actually fairly applicable to consumer and/or physical products as well:

• What is this person’s comfort level with technology?
• Why are they interested in solving the problem that your product solves? (i.e. In what way will it make their life better?)
• What challenges does this person face in their job? (in general – not specific to your product)
• What capabilities help this person do their job more effectively?
• What are this person’s primary limitations to doing their job well? (time/money/technical skill/access to information)?
• What are this person’s primary limitations to trying a new product/process/concept? (time/money/approval/technical skill/access to information/fear)
• How has this person been burned by products in the past? (didn’t do what they promised/too difficult to set up/limited flexibility/limited power)
• What claims would this person likely be skeptical of?
• How does this person tend to do due diligence / verify information?
• Which teams in their organization do they work closely with?
• What is this person afraid of? (i.e. what do they perceive as a risk that will damage their company or their ability to do their job)

When I looked at our actual customer interview notes, three distinct “personas” emerged.

(IMHO: 2-4 is probably ‘normal’; if you have only 1 persona you don’t have enough customers yet and if you have more than 4 you may be asking the wrong questions or have an overly generic product.)

For each one, I wrote up:

• a 2-sentence summary of their job role
• [Person] can do her job effectively when …
• [Person] is frustrated by / skeptical of …
• [Person] needs to be convinced …
• How do you market to / communicate with [Person]?

Laying them out side by side gave us a new appreciation of our customer types — and made it really clear how people trying to solve the same goal may approach it from almost completely opposite directions.

Personas need updating

If you are a startup, or are launching a new product, your personas are in flux.

You’ve read Crossing the Chasm – this is where it applies.  In the beginning, you will probably have 1 persona – the most technical, the most power-user, the most early-adopter.  These people are great: they give tons of feedback, they’re smart, they’re patient while you figure out what the heck you’re doing.  They’re also NOT what you want to build your product around (unless your aim is a niche market play).

So if you create personas and you only have 1 — try again in 3 months.  If you only have 2 — try again in 6 months.  Even if you think you have it covered, totally — revisit again in 6 months.  The internet keeps changing and customer expectations keep changing along with it, so these are not a “set it and forget it” exercise.

Popularity: 9% [?]