Getting Beyond Beta: Measuring Your Audience Breakdown
Exactly how you measure your audience breakdown will depend on what type of software, physical product, web app or service you have. But fundamentally, all audiences go through some sort of decision funnel:
100% of your audience shows up.
Some % shows some sign of interest.
Some % commits an investment of time, personal information, and/or money into your product or service.
Looking at the numbers in between will prompt a lot of questions that you may not know the answer to, but should. I can’t tell you what you’ll learn, but I can walk through some of the questions that your numbers should drive you to ask.
For ease of discussion, we’ll assume we’re looking at an online product, say, a sock-matching service. (Same general rules apply for physical products, but data collection is more complicated.)
- Homepage: 20,000 views
- Register: 2,500 views
- Confirm registration: 600 views
20,000 views to the homepage
I hear a lot of product managers or marketers brag about page views. There are only two times when this is relevant:
1) It’s a really, really, really, really big number AND you’ve sold out ad inventory AND your overhead expenses are really low. (Congratulations, you’re hotornot.com or icanhascheezburger.com, and you’re raking in the $.)
2) It’s not a very big number AND you spent tons of $ on ad buys or direct mailers hoping to get more people in the door. (Sorry, you spent that marketing budget poorly.)
EVERY OTHER TIME, pageviews numbers is just a denominator.
Pageviews don’t translate into happy users,they don’t translate into revenues. They’re only relevant in comparison to the number of people who showed interest, and people who eventually convert to real customers.
For our sock matching service, we need happily socked users who will pay us – not flip-flop wearing visitors who come by and don’t see anything that serves their needs.
2,500 views to registration
I say “registration”, but generically you want to think in terms of “showed some sign of interest”. What percentage of your audience clicked on a “learn more”, or watched a demo, or clicked into the “Solutions” section of your site?
2,500/20,000. Some of those 20,000 might have been repeat visits from the same user, but that doesn’t necessarily matter. If it takes repeat visits before a potential customer is willing to show some interest, something’s not resonating strongly enough with them. So, 2,500/20,000 = 12.5% of your audience showed some interest. 87.5% did not.
87.5% is a big number. Before we move on, let’s think about what that could mean.
Visitors could’ve been expecting something different (like people searching for “green living” and landing on a site selling green sofas rather than an eco-friendly site).
In Google Analytics, you can look for the search terms that sent visitors to you. Do they make sense? Are they terms that you feel are relevant to your business? If not, look at the terms that do make sense and make sure your content contains more of them.
Look at the sites that referred visitors to you. Did you get a huge wave of traffic because lifehacker.com or oprah.com linked to you? Spikes like that might include lots of people that you aren’t targeting to be customers.
If you aren’t sure why so many people left without showing interest, quick surveys can help. You could also use SurveyMonkey or 4Q to show very short surveys to people when they navigate away from your site. I’d advocate non-open-ended questions (“Did you realize that Sock Sorters would prevent you from ever wearing navy and black mismatched socks again?”) – you get less information but a much higher response rate.
Visitors could’ve been completely confused by what your product was.
A good tool for this is the 5-second test. It’s an online tool which will allow you to show a page to users for 5 seconds and then find out what they remembered about it. You don’t need actual customers for this – any subjects you can find will reveal a lot about how people interpret the messaging you put out.
Find a few highly targeted customers and interview them. You know that person X desperately needs sock sorting services, so ask them about their sock behavior. What problems do they have, what steps do they go through, what things frustrate them. Then walk through your site with them. Do they “see” that you’re solving their problem? Did they use words to describe their problem that are completely different from the language on the site?
Visitors could’ve been seen something intimidating/unprofessional/scary that made them distrust you.
The 5-second test would work well here. So would a quick heuristic analysis by a neutral user experience person. An experienced UX person can recognize certain things – lack of a secure site icon, specific phrases that don’t sound ‘right’, conventions that look spammy – that might not jump out at someone who has seen the page every day for months.
They just don’t see the button.
Visitors can’t respond to your call to action if they don’t notice it. Your call to action may not pop enough. Your language may be unclear. CrazyEgg is a simple javascript software that shows where visitors are mousing and scrolling on your site. If their activity is concentrated in one area and your call to action is somewhere else, a simple move could double your conversions.
Or you may not load in time. With my current product, a module that loads in third-party sites, we found that many visitors were skimming the page so quickly that they were gone before we appeared on the page! This was a critical discovery: it immediately shifted our product priorities. There’s no sense building extra features to make 20% of the population happier when there’s an untapped market of 80% who never see the product!
600 views to confirm registration
Again, “confirmed registration” is shorthand for “successful conversion”. You’ve got yourself a customer! Now all you have to do is keep them.
But wait – not everyone who expressed interest turned into a successful conversion. 600/2,500 = 24% of people who expressed interest, who are definitely target customers, actually took the next step to become one.
Now, how bad (or good) is that? It depends on how high the investment (or friction) is. If you have a free product with one-click registration and only 24% of interested visitors convert, that’s pretty crappy. If you have an expensive product, or one that requires the visitor to do several steps of setup, that’s not terrible. (Most banks would be thrilled if 24% of people who started setting up bill pay actually finished setup and paid a bill, for example.)
Let’s assume we’re not happy with 24% conversion (76% non-conversion). What can we look at to understand more about the 76% who didn’t make it through?
Did the people who dropped out come from a common origin?
In Google Analytics, you can look at Traffic Sources to find out more about your visitors based on who sent them to you. One thing I noticed years ago when I worked for a gaming startup, is that hard-core, highly tech-savvy bloggers would write enthusiastic reviews of tools. Their fans would race over… but not being as tech-savvy, were not as willing to invest in learning those tools. The audience liked the idea of gaming tools, but needed clearer messaging and an easier user experience to benefit from them.
Did the people who dropped out visit during specific times?
In my experience doing consumer interviews for financial software, people commonly said, “Oh, I started setup while I was at work, then realized I needed information that I had in my desk at home” or “It sounded interesting, but I couldn’t spend that much time while at work so I meant to come home and do it later.” Two ways to help with this: clearly set time and materials expectations upfront (you will need 5 minutes and your latest utility bill); send an email reminder to come back and finish later. I implemented the expectation-setting with our products at Yodlee and it (with other best practices) contributed to a ~20% uptick in completed registrations for our credit card bill payment application.
Obviously, if you have a multi-step setup, it should be easy to stop and return later without having to re-enter data. (Adaptive Path has an interesting case study on redesigning PayCycle’s setup flow to allow for this.)
Did the people who dropped out all leave on a specific step?
You can look at the Exit Pages in Google Analytics to see this. Frequently, the page that asks for money is a dropoff page; if money’s not an issue, design often is.
How this knowledge should affect your roadmap
Customers need a voice, and often are the source of great product innovations. But it’s easy to “innovate yourself into a corner” if you center your product roadmap decisions around your existing customers.
In the example above, we started with 20,000 people in our audience, got to 2,500 interested people, and ended with 600 customers. If those 600 customers were paying you $1 each, you’d be making $600. To double your revenues, you’d have to convince these customers to buy twice as much stuff, or you’d have to make them so ecstatically happy that you had 100% new customer referrals. Both of these are possible, but really, really hard.
If you look at the 1,900 people who expressed interest but didn’t become customers, you only need to make about 30% of them happy in order to double your customer base and revenues. Creating a feature that is neutral to your customers but a differentiator to your uncomfortable potential customers has a much higher potential return.
-->
