Better Product Managers, and Product Management

How to Fail Fast: Checklist to Lead You to the Next Experiment

This post is Part 2 in a series.  Read Part 1.

You’ve seen the signs and know you need to change direction in order to have a successful product.  But how do you select the best possible next experiment?

There’s no sense in failing fast if you aren’t able to learn from your failed experiment.  And that’s another rathole that well-intentioned product managers fall into.  Everyone wants to learn “as much as possible” from a failed project; what they don’t do is follow a rigorous checklist designed to shine a light on the specific critical elements that stand in the way of success.  Postmortem meetings generate a lot of opinions.  Non-focused user surveys generate a lot of opinions.  Neither of these are likely to generate the right next step.

This is a checklist of questions to ask based on my experiences in product strategy changes.   Is this a lot of questions?  Yes!  And you should know the answer to all of them.  I can’t guarantee it’s complete, but it’s a good framework for documenting and challenging your assumptions.  As you answer these questions, it will become clearer what user and market research you need in order to map your next move.

Questions About Customers

  • If you acquire 100% of your target customer base at your current monetization per user, would you have a profitable product?
  • Are your current customers generating profits?  (If they are generating less revenue than it costs to support them, the answer is “no”.  If they are not generating revenues, the answer is “no”.)
  • Have your customers paid for your product?  (If the answer is “no”, continue to the next, vastly less reliable question)
  • Have your customers demonstrated willingness to pay for your product?
  • Are your customers able to pay for your product?
  • Is there room for your customers to “grow” into more committed, profitable customers by using more features or premium services?
  • Have your customers demonstrated loyalty via low attrition rates, word of mouth recommendations, referrals?

Questions About Product

  • Does your product solve a problem that your customers are motivated to solve (and have demonstrated this motivation through investing time or money)?
  • Can you continue to offer your product with the people you have and the money you’re making?  (Or does “something” need to happen to convert people into profits?)
  • If your customer base doubled/tripled/grew exponentially, would you be able to continue offering your product without fundamentally altering the features or pricing?
  • Is your product winning more customers away from the alternatives? (Alternatives are not always other products – “live with the problem” and “do it manually/offline” are entirely valid alternatives)
  • What percentage of people convert to customers? (Complete registration, purchase your product, begin using)
  • What percentage of customers successfully complete a desirable “advanced” action? (For example, in a bill pay product, you want customers to complete the full process of adding a biller and making a payment.)
  • Have you watched users successfully complete the key tasks in your product? (Analytics may show that users are completing an action successfully, even when watching them in a user testing situation reveals that it takes them too many clicks and they are getting progressively more frustrated.)

Questions About Pricing

  • Do your customers have limitations which may affect the way in which they are able to pay for your product? (For example, paying in one lump sum may be desirable in order to zero out a specific budget line item; or paying monthly fees may eliminate the need to get management spending approvals)
  • Is the amount you are charging adequate to cover sunk costs such as one-time deployments?
  • Is the amount you are charging adequate to cover ongoing costs such as customer service?
  • Do your customers have variable needs for services and/or features such that tiered pricing would make sense?
  • Have customers expressed interest in a different pricing plan (subscription vs. one-time, base plus usage, etc.)
  • If you are not directly charging customers, is your revenue dependent on a third-party factor? (For example, ad-supported models are dependent on ad CPMs)

Questions About Distribution

  • How many customers have found out about your product?
  • What is your acquisition cost per customer?
  • What percentage of your customers have referred a customer, blogged or twittered about you, or invited friends?   (For a viral distribution strategy, each customer must share the product with > 1.0 people)
  • What incentive do your customers have to talk about your product?
  • Are there obstacles to people finding your product?  (For example, site downtime, required to register before any preview, open by invitation-only, hard-to-remember URL?)

Questions About Brand

  • Can your customers spell and pronounce your product name?
  • Are you free from trademark issues so that you can market under that product name?
  • Does your product name “match” your intended customer audience? (This isn’t always a blocker, but having a “fun-sounding” name when you’re selling to enterprise customers can be another obstacle to overcome.)
  • Does your product look-and-feel “match” your intended customer audience?  (Hard-core computer gamers don’t trust a white-background website with calm blue accents; investors shopping for a new brokerage don’t trust a black-background website with red text.)
  • Are your customers able to articulate your value proposition?  If you ask them why to use your product, does their answer match your pitch?
  • Do the associations that people (customers and non-customers) have with your brand name and look-and-feel match the “feel” that you want the product to convey?

This post is Part 2 in a series.  Read Part 1. (Stay tuned for Part 3 next week!)

Popularity: 3% [?]

Popularity: 3% [?]
  • http://www.outsideinview.com/ Jennifer D.

    Great information. Very practical lsit that works the high points without forgetting the focuse.

    Not only is this a good list as you review product line and viability; but, also to turn into questions if you are contemplating work with a startup.

  • Artie W

    This is an awesome post!!

  • http://twitter.com/trevorrotzien Trevor Rotzien

    Great checklist! Not only good for Product Managers, but for the entire leadership of product companies to at least understand.

  • Cindy

    Thanks for the comments! As with many of my posts, a lot of questions came from “Wow, I wish I’d asked this question before X happened…”

    The branding one is surprisingly tricky – it seems so obvious that someone MUST HAVE checked to make sure there’s no existing trademark for that name, to make sure people outside the company can pronounce it. And yet I’ve been on teams bitten by this MULTIPLE times. Don’t pick an internal name, don’t put a name on a single PPT, until you’ve checked the trademark/pronunciation/negative connotations out!

  • http://productmanagementtips.com Gopal Shenoy

    Great post Cindy. I have already tagged it on delicious because I know I will be referencing this post for a long time.

  • http://twitter.com/jidoctor jidoctor

    Just commented on @cindyalvarez blog “How to Fail Fast” @ http://snurl.com/i0j6p; good list for review (& asking interview ?s w/ startups)

    This comment was originally posted on Twitter

  • http://twitter.com/mjbroadbent mjbroadbent

    RT @cindyalvarez: New blog post: How to Fail Fast: Checklist to Lead You to the Next Experiment (pls comment!) http://snurl.com/i0j6p

    This comment was originally posted on Twitter

  • http://twitter.com/mikegee mikegee

    How to Fail Fast: Checklist to Lead You to the Next Experiment http://bit.ly/dVSbj

    This comment was originally posted on Twitter

  • http://twitter.com/josh_baker josh_baker

    How to Fail Fast: Checklist to Lead You to the Next Experiment http://bit.ly/dVSbj (via @mikegee)

    This comment was originally posted on Twitter

  • http://twitter.com/Jim_Holland Jim_Holland

    To have a successful product, you may need to change direction-Great questions/How To’s @cindyalvarez http://bit.ly/AwqeF #prodmgmt

    This comment was originally posted on Twitter

  • http://twitter.com/productmgt productmgt

    You’ve seen the signs and need to change direction in order to have a successful product http://tr.im/lHr2 (thx @cindyalvarez) #prodmgmt

    This comment was originally posted on Twitter

  • http://twitter.com/cindyalvarez cindyalvarez

    My 4-post series on “How to Fail Fast”: http://sn.im/hhe95, http://snurl.com/i0j6p, http://snurl.com/ig987, http://sn.im/ivzbu #prodmgmt

    This comment was originally posted on Twitter

blog comments powered by Disqus