Cindy Alvarez

Listening and Acting: Power Users vs. Mainstream Users

Last week I wrote about the huge gaps that can (and probably do) exist between the power users and the mainstream users of your product.

But what do you DO about it? Power users are your evangelists, your backup QA testers, and your visionaries. They're also by far the easiest to get in touch with (just try to avoid them -- they'll find you.) It would be stupid to blindly ignore their feedback. It's also really hard to tell sometimes: is this an issue that only affects the power users? Or is it a universal thing, and they were just the first to detect it?

Here's how I try to evaluate feedback, bug reports, suggestions:

Understand When, How Often, Why

The fastest way to ask these questions is just to shoot the person an email asking followup questions -- most people will be able to answer. I often also add "if it's easier to explain over the phone or chat, here's my contact info").

You'll hear things like "...it happens when I upload a file. What? Oh, the file is 700MB, does that make a difference?" or "well, I wanted to match our obscure data format, so I changed every single one of the settings..."

Look at Non-Average Data

Averages are really easy to measure, and lots of tools default to "average" values. But the "average" value hides a ton of variability. Let's say you have 10 users who sent 1, 1, 1, 1, 2, 4, 4, 4, 800 messages.

The average number of messages sent is 81.8. The median number of messages sent is 2.

That's a 40x jump. That's a good multiplier to keep in your head when you read feature requests. Just by asking yourself, "is this a situation that only affects the 40x user?", you'll more effectively prioritize feature requests, complaints (and compliments).

Go Find Some Normal Users

Want to know if this problem is affecting mainstream users? Ask them. You don't need a lot of data -- just a couple of anecdotes can provide a very useful sanity check on your assumptions.

I find mainstream users to talk to by:

Usually the "conversation" can be as simple as: "One of our other customers described this problem. Does this sound familiar to you? / Has this ever happened to you?"

If they have no idea what you're talking about, this may be an ignorable power-user problem.

If their response is a "yes", a "well, sorta...", or a related followup question -- you probably have an issue that affects a significant percentage of users. Now it's time to dig deeper into the when/how often/why.

Think about None, A Few, and A Lot Use Cases

Settings, features, preferences, limits, and displays are often very different based on 'amount' use cases. We tend to spec out or mock up "optimal" use cases -- here's how it'll work with an average number of users, an average number of items, an average file size. Too bad our users aren't so average.

Instead, think about how this change/solution will work:

If you're lucky, you have an admin tool where you can emulate these situations. If not, it's worthwhile to take some extra time and mock up options.

And finally: be prepared for complaints

The corollary of "if you're not embarrassed by the MVP of your product, you waited too long to launch it" is "if your power users aren't angry at you, you're not going to succeed in the mainstream."

Sorry everyone: I wish we could build that crazy feature too. But we're not going to.