Is Your Company Mature Enough to Learn from User Research?

I’ve worked on products that were great and ones that were kind of atrocious.  And through both ends of the spectrum, I can confidently say that if you’re doing research right, what you learn will make you cringe.

  • You’ll learn that the person who rates you a 10 out of 10 uses less than a quarter of your product’s functionality.
  • You’ll watch someone struggle and mutter under their breath as they try to complete a task that is both critical and (YOU THOUGHT) simple.
  • You’ll see that, even when individual tasks are easy to do, it’s tricky to string them together.
  • You’ll see people who don’t struggle with your app at all — but also are politely but utterly disinterested in using it.

This took me by surprise the first few times I conducted my own user research, because I had seen other peoples’ slide decks, and those decks always painted wholly optimistic pictures: 6 out of 8 participants completed the task without hesitating! 7 out of 8 would ‘definitely’ use this app!   Quotes like “really easy to figure out” or “would allow me to work a lot faster” sprinkled throughout.

Was I just doing it all wrong?

I can’t blame those early researchers: they feared (probably correctly) that their clients would shoot the messenger.  If research brings unwelcome news, well, do we really need researchers after all?

It’s a sign of the maturity of a company — and I don’t mean maturity as in series B startup or Fortune 500 — how they conduct research and how they accept it.

About a month ago, one of the researchers on my team at Yammer shared a presentation on how mainstream users use our product.  It was depressing.

It signaled that, for all the progress we’ve made, we have so much farther to go.  And yet it made me proud to see that the engineers and product managers in the audience asked respectful questions and encouraged follow-up research so we could learn more.  I didn’t hear mutters or complaints in the hallway.  We might not have been happy to hear the feedback, but we didn’t attempt to reject it, either.

Of course there is an art to delivering bad news: one can’t simply dump a bunch of negative quotes and failure rates and drop the mic.  But when I hear usability test readouts/see slide decks/read blog posts where every bit of news is good news, I cringe.  Those are companies that have much bigger problems than a probably-unusable product.

3 Responses to “Is Your Company Mature Enough to Learn from User Research?”

kevinowens:

Great post Cindy and something that definitely rings true with the products and companies I have been involved with. The executives, product managers, sales and marketing all need to have the maturity to listen to constructive feedback and commit to making the change.

Stuart Moore:

Companies struggle with the reality of facing up to feedback during beta / preview releases because of the time and effort invested in building the new feature. It is all great when the feedback is positive, or only minor changes are required. But if there is insufficient time to resolve the issues before release then there is really only one course of action, hide / remove the feature and then put the effort into getting it right for the next release. Unfortunately, this is where the conflict comes in, pulling the feature means accepting failure – keeping it in justifies the effort and helps mask the failure. We have to accept that we make mistakes, the point is to learn – why release a feature that may cause confusion, leave customers disappointed or worse result in support calls. The bravest course of action is to acknowledge the mistake and pull it.

UXR:

So true. I feel the user researcher’s job is to present the data and recommendations based on the analysis of that data. Good or bad! In fact, I think they need to hear the negative things more. I had a UXR coworker once who was afraid to say negative things in their reports. It comes down to personal and professional integrity and in making a real difference.

Leave a Reply