Hybrid Feedback is Stronger
As last week’s commenters pointed out, there’s a challenge in offering multiple choices vs. asking for freeform responses: You might get more responses but still be missing the root cause of customers’ concerns/problems/ideas.
They’re right. Freeform answers alone are flawed. Multiple choice options alone are flawed. You need to use them both together in order to generate unstoppable, reliable hybrid feedback!
There are two approaches to cultivating your hybrid feedback – pick the one that is most relevant to your situation.
Interview First, Then Survey
Use this method when: You have no idea what you’d even suggest as multiple choice options.
Example: People are really excited when they sign up for your book exchange website. But almost no one is completing a successful exchange, and honestly, you have no idea why. You can’t see a pattern to where people are dropping out of your workflow, and you’re not getting a lot of bug reports or complaints in your support inbox.
Where to start?: Talk to people.
You can call and ask questions about their book exchange needs (see earlier post on “what you should be learning?”), or ask a couple people to go through your website while you watch over their shoulder or using UserTesting.com, or some combination of the three.
Once you’ve talked to 5-10 people, you will usually see some patterns – concerns or obstacles that are affecting more than one person and that seem pretty plausible based on what you know of your own product.
You may find that the problems reported in these first 5-10 conversations/user tests are so fundamental that they’re actually preventing you from getting deeper feedback! (For example, if people are having problems logging in, they’ll never get to use your core product enough to give you useful feedback on it.) In that case, skip the survey and start fixing!
Or you can use this initial feedback to populate your multiple choice options, and run the numbers.
Personally (and yes, this is my opinion, not proven data), when I see a multiple choice question from a company and all of the options are really well-written and show a deep understanding of their product, I feel like they really care and am more likely to spend extra time writing out an “Other” response.
Survey First, Then Interview
Use this method when: You want to guide the conversation to specific, relevant options. OR You have a pretty good guess as to what the potential responses are.
Example: You have several large features on your product roadmap, all of which are aligned with your product vision, have supporting market research, and will provide different customer benefits. But you’d like to validate that they really will be valuable to the customer, and get some subjective feedback on which one will “delight” your customers the most.
Where to start?: Survey.
Questions that may work for this context:
- Which of [list options] features are you most excited about?
- Which of [list tasks] do you use most frequently?
- Would one of these [list options] have convinced you to complete your purchase?
- Would you be willing to be a beta tester for one of these upcoming features [list]
- Have you experienced one of these issues [list]? How did it affect you?
Once you’ve gotten 20-30 responses, you will usually see a clear winner (or two) emerge.
But now, you have to make sure you properly “decode” that feedback. You need to understand the “what else” and the “why”.
What else? It may be true that the majority of your customers are most excited about [feature X], but they are assuming that it will magically “just work”. It’s your job to understand and solve for things like Will this change my workflow? Will this involve new people or exclude people who previously used it? My boss is used to [competitor product] so she’ll want it to work like that does.
Why? It may be true that everyone is selecting “add customer reviews and commenting” as their preferred next feature. But it may not be for the reason you assume.
Let me illustrate with a personal example:
CA: “I didn’t end up buying that [group buying site] Pilates deal, even though it was a good discount.”
Friend: “Why not?”
CA: “It wasn’t clear when you could use it — It would’ve been a waste of money if the classes were only at a time when I couldn’t go. They should make the vendors give them more information or update their websites to be more clear.”
Friend: “Did you realize that [group buying site] offers a money-back guarantee? You could’ve bought it and returned it if it wasn’t the right schedule.”
CA: “Ohhhh. I didn’t know that! I would’ve bought it if I’d known that!”
Which is easier: getting hundreds of vendors to submit information, and then wrangling that information into a CMS; or just highlighting a money-back guarantee icon?
So that’s why you then follow up with interviews. Get details, try to disprove your assumptions, and then you’ll finally have the full understanding of your feedback.
100% of your audience shows up.