Better Product Managers, and Product Management

There is something you want to improve about your site or product.  (If there wasn’t, you probably wouldn’t find this blog terribly interesting.)

Maybe you’re unhappy with the low percentage of visitors who continue past your homepage.  Maybe too many people are abandoning their full carts.  Maybe you’re getting a ton of customer support requests related to a specific feature.

But where do you start?  It can feel very daunting, which is why most people are not doing as much active testing and improvement as they should.

But don’t worry – it gets much easier when you have a plan.   We ran a ton of focused A/B tests on KISSinsights last year and more than doubled our homepage to signup conversion rate.  Here’s how we started:

Step 1:  Write it down.

No, you won’t “just remember”.  (And even if you do, the rest of your team certainly won’t).    Fire up a new doc and write down your problem:  Not enough visitors continuing past the KISSinsights homepage. And put the date at the top so you remember when you started.

Step 2: Quantify it.

How bad has the problem been over the past 7 days?  I like ‘past 7 days’ because it includes both weekdays and weekends — which tend to have very different usage patterns — but is still pretty recent, so it’s not affected by things you did months ago.

If you’re using a percentage (i.e. only 1.1% of customers continued beyond the homepage), be sure to include the raw numbers as well (11 visitors out of 1000).

If you’re using a delta (i.e. 14 customer submitted service tickets related to feature X), be sure to include a comparison (usually we don’t get more than 3 service tickets on any one specific feature).

As you see in the delta example, your “quantify” doesn’t have to be super-precise.  I’ve written down things like “support emails seem angrier this week – usually we only get one complainer and this week we’ve had 5 already” — that’s pretty subjective, but it’s still something that I can come back to later and compare to.

Step 3: Ask yourself why.

Quickly take a look at the problem page, workflow, or feature.  Do you see anything that stands out?

…but don’t spend a lot of time on this.  You’re probably not that useful.

Step 4: Ask the data why.

Here are some techniques for figuring out why:

• Check your browser/OS stats: By far the simplest explanation; if the problem is extremely pronounced among IE8 users, there’s almost certainly a bug.  These are the easiest – fix it and you’ll probably see an instant improvement bump.
• Ask an unfamiliar person: Grab someone outside your company — anyone; they don’t need to match your target customer profile — and ask them to use the page/workflow/feature in question and ‘think out loud’.   Things to look for – are they confused by the copy? Do they hesitate because they have a question that’s gone unanswered?  Do they scroll up and down looking for the button to click?
• Check your heatmap: Use a tool like Crazy Egg to check where people are clicking.  In our case, when we had a 5-field form, we could clearly see that most people only clicked on the first 2 fields – signaling that we were asking too much effort of them.  We reduced that form to 2 fields and conversion bumped up immediately.   Heatmaps can also make it obvious that people aren’t scrolling below the fold, or that they’re trying to click on a non-clickable element, or clicking frequently on a ‘distraction’ item instead of the next step.
• Ask a quick question [shameless plug]: With KISSinsights, you can configure a question that pops up after a time delay – we asked what other information customers needed to convince them to sign up, and their responses gave us a to-do list of copy to reword and information to add.
• Ask a successful customer: It’s often easier to find someone who succeeded despite the problem (than someone who dropped off).  Try contacting a customer who completed the workflow or purchase and ask, “Was there anything you found confusing about the process?  Was there information that you expected to see there that you couldn’t find?”  They may well remember and be able to tell you the things that made them hesitate, even though they went ahead and continued.
• Answer support questions with your own questions: When replying to your customer support emails, ask a question of your own (“May I ask you a question?  Have you tried feature X? Is there anything about it you found confusing or difficult?”)

Step 5: Brainstorm.

More heads are better than one.  And now you have the data to bring this problem to your team.   At KISSmetrics, we use a format like this:

Here’s the problem and the current performance data.

Here are some issues we’ve identified that are causing or contributing to the problem.

What can we quickly do to try to fix those issues?

Remember that the operative word here is “quickly” — you don’t want to spend the next two weeks brainstorming the “perfect” solution (which doesn’t exist anyways.)  If your team has that tendency, one tip can be to set time limits on the solution period, i.e.  It’s Monday morning.  Let’s think of possible solutions until our noon Tuesday meeting and then we’ll pick the best one or ones to start implementing.

This should give you a good kick-start on making it better.  Next week I’ll cover how you pick your solution, deploy, and measure.

Popularity: 5% [?]

Popularity: 5% [?] blog comments powered by Disqus