Better Product Managers, and Product Management

Revenue seems like the one part of the AARRR metrics that shouldn’t need explaining.  More money = better, right?  It hardly takes a rocket scientist to know that.

But as you’ve probably figured out by now, my personal bias is towards starting with metrics where you can immediately translate them into a natural-language statement (“this means X customer type is more likely to pay us”).

So instead of talking about customer lifetime value or average revenue per user, I’m going to talk about revenue in terms of two simple questions you should be asking to start with:

• What percentage of Activated customers have given us money, even once?
• What types of customers were more likely to give us money?

These are both fairly short-term questions, but what I like about them is that they can immediately narrow where you should focus on improving next.

(Everyone ought to be improving their customer lifetime value, but that’s about as vague as your New Year’s Resolution to lose 10 pounds.  And about as likely to succeed, unless you break it down.)

So let’s look at some examples.

What percentage of Activated customers have given us money, even once?

Let’s look at fake-Amazon.com:

You’d have defined Activated customers as the people who were had reached the potential to get value — in this case, they found a product that was interesting enough for them to view in detail. This is your potential pool from which to extract Revenues.

How many of those people give you money?

In this example, 2.2% of your potential pool is giving you money.   So now the obvious question is, how can I get more of this potential pool of pre-qualified, full-of-intent, people to give us money?

I’d expect that it’s quite common for someone to view a product and not proceed with purchase — you could be just browsing, or perhaps you were just curious about that electronic toenail polisher.  But here I’d hone in on the big dropoff between “Added to Cart” and “Completed Purchase” — and try to figure out why via customer interviews, user testing, or a KISSinsights survey.

If I could win back just 5% of those “Added to Cart” people, that’s more than 150 additional purchases.  Assuming a $20 average sale, that’s more than $3,000 additional revenue per day, and the solution is probably just fixing something that you should’ve fixed anyways.

For a second example, let’s look at fake-Netflix.com:

This would probably signal to me that my focus belongs on getting more Acquired users, versus trying to improve upon either of these steps.

What types of customers were more likely to give us money?

Most of us are targeting more than one type of customer.  I strongly suggest collecting that information — either via the registration form, or some required post-signup configuration — and using it to confirm where your dollars are coming from.

In this example, suppose you were selling a website service, and asked your customers what type of site they maintained:

The overall percentage of upgraders — 2.8% is interesting — but a lot less interesting than the fact that people running e-commerce sites are dramatically more likely to pay you than other types of customers.

This tells me that, rather than trying to create a “generic” marketing homepage that appeals to all customers, you should try tailoring your marketing site to specifically appeal to e-commerce customers.

Once you’ve experimented and improved some of the more obvious areas, you’ll feel a lot more comfortable tackling the more complex “lifecycle” metrics (and these can only help those, as well).

Popularity: 67% [?]

In a continuation from last week, I’ll talk about how the RRRs might be seen in the context of Facebook (and other examples).

Acquiring and Activating a customer is like getting a person inside your restaurant and sitting down with a menu in front of them.  But that’s no guarantee they’ll come back and eat there again.  In fact, there’s still the potential of you screwing up their experience so badly that they walk out without ordering.

Retention

Here’s the challenge with retention: most people think in terms of questions like “What percentage of our customers are logging in daily/weekly?” or “What percentage of our customers stick around for 3/6/12 months?”

Those are excellent questions, but they take too damn long to give you data.  If you’re early in your product lifecycle, you can’t afford to wait 3 or 6 months to get a sense of your Retention.  I recommend measuring in 2 ways: Retention threshold and ongoing Retention.

Your Retention threshold is analogous to the Activation threshold – it’s the point where your product has become a part of the customers’ behavior; something that they’re willing to invest time in.

How do you know what behaviors correspond to this threshold?  You don’t, at first.

This is where customer interviews can be really useful.  Don’t try to be too clever — just ask, “What’s the thing that convinced you to be a loyal customer? / What’s the thing that brings you back to our site regularly?” and listen to what people tell you.   A dozen conversations will get you to answers much, much faster than trying to tease this out with quantitative analytics alone.

So let’s guess that Facebook had done these interviews and identified a bunch of different behaviors, not all of them easily measurable:

• “when I started sharing all my photos via Facebook”
• “Facebook is the first thing I check in the morning and the last thing I check before I sleep”
• “I realized it was easier to invite people to my events using Facebook than Evite or SMSing”

Now, not all customers are going to do all things, but you could use this input to choose a few events as a reasonable proxy for crossing the Retention threshold — let’s say “Created a photo album”, “Created an event and invited at least 1 person”, “Set Facebook as browser homepage”.

You might see a funnel like this:

Note that all three behaviors were combined into one “retention action” and then broken out in the table beneath.

This gives you a few into how many people are crossing over the Retention threshold, and which behavior path they’re more likely to take.  You can then experiment with “nudging” people to complete these actions faster (i.e. via an email campaign encouraging customers to upload a photo album after a holiday weekend, or an interstitial prompting them to set your site as their homepage).

This is just a proxy; it’s not something I’d get obsessed with.  But it’s a good stopgap measure until someone has been a customer for long enough that a more standard “they keep logging in / they keep paying me” metric kicks in.

I’m going to talk less about ongoing Retention because it seems like it’s more widely understood.  My main peeve there is when companies blindly use “login frequency” as a measure of retention.  How often I log in is not equal to “how valuable I find your product”.

The real equation you’re trying to solve for is:  “Of the times when my customer has [these problems], what percentage of the time are they turning to me to solve them?”

Facebook’s goal is to be an everyday app and insinuate itself into your social world.  Since you’re part of your social world every day, their goal should be for you to log in daily.   Compare that to LinkedIn — it’s not every day that I need to check out someone’s professional credentials or research potential hires.  When I have that need, 100% of the time I turn to LinkedIn.  But I don’t have that need that often.

(That may be another problem, but I’ll save that for a different post.)

Referrals

There are 2 ways of looking at Referrals:

• What percentage of my customers will, if prompted, refer my product to others?
• What percentage of my customers have come from referrals?

A great example of #1 is Dropbox:

Dropbox offers an incentive for customers to refer the product to friends, which has several points of optimization:

• Do people care enough / trust you enough to start a referral process?
• Is the workflow of referring the product easy and fast enough for them to complete it?
• Do the referred / invited people care enough to come try it out?
• Is the signup process for invited people streamlined enough for them to complete it?

If explicit referrals are key to your distribution strategy, you’ll need to optimize the hell out of each of these stages.  That is a big if.  There are many companies who grow just fine without an explicit referral strategy, and there are many companies who may benefit from this in the future but aren’t ready yet.  If this doesn’t “fit” with you now, don’t stress about it.

Let’s say you’re not optimizing a referral workflow.   KISSmetrics is not.  KISSinsights attaches a signup call-to-action on our free surveys: we get customers that way, but it’s a very different workflow than “Your friend X endorses that you use Product Y”.   We could optimize that a lot more.  But in the meantime, some simple questions you can ask are:

• What percentage of signups originated from Twitter or Facebook referrers?
• What percentage of visitors who originated from Twitter/Facebook converted?
• What percentage of signups originated from a “click here to get your account” or “powered by” link?
• What percentage of visitors who originated from “click here to get your account” or “powered by” links converted?

These can serve as an indicator to let you know if you should start focusing more on one of these channels.  We get a lot of Twitter attention — i.e. lots of Acquired people — but the conversion rate isn’t super strong.  If it was, you can bet I’d be prompting customers to Tweet about us more.

Another indicator: if “passive referral” folks account for more than a few percent of your customers, that may indicate that your homepage isn’t adequately communicating your value proposition.  So only folks who’ve gotten the added credibility of a tweet from a trusted contact or seeing you live on another site are actually converting.  If that’s the case, time to start A/B testing new homepage marketing copy!

OK, I know I said I was covering all 3 R’s this week, but I think Revenue deserves its’ own entry.  So you’ll have to wait one more week…

Popularity: 7% [?]

Startups, you all know about the AARRR metrics.  But it can be tricky to figure out how to apply them to your site.

So I’ve taken a site we’re all familiar with — Facebook — and walked through how they might apply these metrics.   (Note: Facebook is not a KISSmetrics customer nor associated with us, and I am in no way implying that their analytics are this rudimentary.  Also, all numbers below are totally, and hastily, made up.)

We’ll start this week with the A’s: Acquisition and Activation.

Acquisition

Acquiring a visitors means that they came to your site and did something to show interest.  People who bounce in and off your site in less than 5 seconds are not acquired.  They probably won’t even remember your name tomorrow.  Same goes for people who open your site in one forgotten browser tab and never look at it.

What should you use to measure “showed interest”?  It will vary based on what options your site offers to customers.  For an information-dense homepage, “remaining on the site for 20 seconds” might be a good proxy for interest.  For a shopping site, “clicking on navigation, conducting a product search, or clicking on a feature product” would show interest.

In the Facebook case, their homepage has the signup form embedded right there, with a very low barrier to entry.  I would consider “submitted signup form” to show interest:

So you might have a funnel that looked like this:
If your already-converted customers come to that homepage, you’re going to see skewed-low numbers.  (Obviously, if I’m already a customer, I am not going to continue to “Submitted Signup Form”, I’m going to click the sign-in link. )

One easy high-level way to “correct” for this skew is simply to create a complementary funnel for Viewed Homepage -> Signed In and subtract that percentage:

• 10,000 people view homepage and then 100 submit signup form (1% signup rate)
• 10,000 people view homepage and then 5,000 click the sign-in link
• those 5,000 were not actually candidates for signup (they’re already a customer) so assume more like a 2% signup rate

Not exact but good enough to get a feel for it.

There’s a ton you can dig into here around which channels, ads, search terms are leading to the highest acquisition.  That said, most companies I talk to are focusing too much on Acquisition too early. By far the biggest problem and highest risk is that no one will care about your product/site.  (Or be able to use it.)

I’d worry about getting more of the few people you do have, to give a damn, than trying to optimize/maximize how many more people you can get.  (Brant Cooper writes more about that, and the order in which you should tackle the AARRR.)

Activation

Like Acquisition, Activation is not immediate.  A person who has simply completed a signup is not necessarily activated unless they can immediately start using and benefitting from your product.

I define Activation as the point at which your potential customer is able to get value from your product.  In other words, they have completed those obstacles — entering information, configuring settings, installing code — which are required before they can do anything useful.

The Facebook example works particularly well here.  Until you’ve added at least one friend, Facebook is useless.  (“what is the sound of one friend friending?”)

So at minimum, you might track a report like this:

But how do you know if “added 1 friend” is enough of a proxy for Activation?  The short answer is, you don’t. You’ll have to guess, with some help from talking to customers and user-testing.

(After you have a fair amount of data, also, you’ll be able to do things like look at the people who log in regularly and identify their least common denominator.  But most people don’t have that much data to start — and for most sites who don’t enjoy the traffic levels of Facebook, quite honestly the patterns don’t just leap out like that.)

As it happens, I sat with my mom while she signed up for Facebook  (the perils of being related to me; I am probably surreptitiously conducting a user test on you).  Adding 1 friend wasn’t that exciting.  Sync’ing her Gmail contacts wasn’t that exciting (after all, the people she emails are the people she already sees all the time.)

What “tipped” her experience was searching for — and finding — long-lost relatives and getting friend requests from people she’d lost touch with.

These “tipping point” experiences are things you’d want to measure. You’d also want to measure the customer behaviors that facilitate these experiences — for example, if a user does not fill out their Facebook profile, they are unlikely to be ‘found’ by others.  Adding your high school and college affiliations don’t directly bring the customer value, but they indirectly lead to value by making that person findable.

So you might track a report like this:

Of course, there isn’t going to be a “perfect” proxy for Activation, so you may want to start with several different “hypotheses” — different sets of criteria that you look at and then triangulate by talking to customers and looking at longer-term data.

You’ll also want to use qualitative research to understand why some customers perform these actions (“I figured I’d get this benefit”) and others don’t (“I was afraid this would happen”) so that you can move more people to more quickly proceed from Acquired to Activated.

Next week: the RRRs!

Popularity: 19% [?]

You may be wondering: where do I start?

Asking questions helps me make sense of metrics.  It’s easier for me to think in terms of questions first, then look for the ways to answer them (whether that involves quantitative analytics, qualitative research, user testing, or some other tool).

Not all of these questions will be relevant for your business, but seeing a checklist can help you identify the questions that do make sense for you.  (It’s also useful to revisit this list from time to time, because as your business evolves, the questions you focus on will change.)

Acquisition

• What sources are sending visitors to my site?
• What are visitors searching for that brings them to my site?
• What percentage of my visitors are coming from search / direct traffic / paid search / ads / referrals?
• Which ad campaigns are convincing people to visit my site?
• For the people coming to my site from direct traffic, how did they find out about us?
• If a particular (non-paid) source is sending a lot of traffic to my site, how are they describing us?

Activation

• Of all visitors to your site, what percentage of them convert to becoming a customer (signing up, subscribing, purchasing, starting a trial)?
• Of the visitors who fail to become customers, where are they dropping off?
• Which sources are most effectively sending me visitors who become customers?
• Which sources are sending me lots of traffic that is not converting?
• Which ad campaigns are most effectively sending me visitors who become customers?
• Are visitors who came via search more likely to become customers?
• How many times do visitors come to your site before they convert to customers?
• How much time (in hours/days/weeks) does it take for a visitor to your site to convert to being a customer?
• What is the first source that sent a customer to your site?
• What is the last source that sent a customer to your site immediately before they became customers?
• How does signup workflow (i.e. Facebook Connect vs. traditional signup, mobile vs. web signup, homepage vs. third-party partner) affect conversion rate?
• Were visitors who viewed a video or product tour more likely to convert to becoming a customer?
• How does collection of billing information (i.e. collected up-front, billed later, free trial to start, etc.) affect conversion rate?
• What percentage of customers experience an error during the signup process?
• Of the customers who experience an error during the signup process, how many persist and become customers?

Retention

• What percentage of customers come back and log in again within a week / month / 3 months / etc. ?
• What percentage of customers successfully complete configuration / setup such that they’re able to use your product?
• Are customers who signed up from a specific source more likely to complete setup / configuration?
• Of the customers who do not complete configuration / setup, where did they drop off?
• Of the customers who sign up for a free trial, how long does it take for them to start actively using their trial?
• How long does it take for the average customer to go from initial signup to successfully using the core features of your product?
• What percentage of your customers are successfully using some advanced features of your product?
• Which sources are most effectively sending me visitors who become customers who return?
• Are customers with a specific demographic profile more likely to successfully return and use your product?
• If you’ve offered discounts or coupon codes, are customers who used those codes more or less likely to successfully return and use your product?
• Are customers who sign up because of a referral more likely to return and use your product more actively than customers who did not come in via a referral?
• What percentage of your customers stop using / paying for your product each month?

Referral

• What percentage of your customers are actively recommending your product?
• What percentage of your incoming customers are coming in via referral?
• How does referral method (Facebook, Twitter, email invitations) affect number of referrals sent?
• How does referral method (Facebook, Twitter, email invitations) affect number of referrals that the recipient acts upon?
• How does incentive affect referral behavior (both sending and acting upon)?
• Who are your most active referrers / evangelists?
• Are customers with a specific demographic profile more likely to recommend your product?

Revenues

• How much in revenues are you making each month?
• Are you on track to increase revenues from last month?
• What percentage of your revenues are coming from new vs. returning customers?
• How do your revenues break down between various plans/service levels/products?
• Which of your customers are providing you with the most revenue?
• How long does your average customer remain a paying customer?
• How much time elapses between return purchases?
• What is your average customer lifetime value?
• Which sources are providing you with the most profitable customers?
• Are customers with a specific demographic profile more likely to be more (or less) profitable?
• What percentage of my customers are canceling due to billing errors as opposed to deliberate cancelation?
• What percentage of purchases are coming from returning customers?
• What percentage of your customers are upgrading from a free to paid plan?
• What percentage of your customers respond to a cross-sell or upsell purchase?
• If you offer multiple upsell calls to action, which ones are most successfully driving upgrades / additional purchases?
• If you offer multiple payment options, which one is driving the highest percentage of successfully completed transactions?
• What percentage of customers are experiencing an error during the billing / purchase process?
• Of the customers who experience an error, how many persist and complete the billing / purchase process?

Popularity: 28% [?]