Making it Better: How do I close the loop?
If you read last week’s blog post and followed through, you now have:
- picked a fast and measurable improvement to try
- figured out what part of your audience was going to see it
- launched it
You might think the hard part is done now.
But this is exactly where most people fall apart — they fail to close the loop.
Closing the Loop
By “closing the loop”, I mean:
- Validating that your change worked
- Measuring how big the effect was
- Following up with your team so they know what happened and what you learned
The last step is what makes or breaks your ability to keep making your product better. You can brute-force your team through a couple rounds of improvement without it, but it’s not sustainable.
When you have a developer who spent a full day reworking a feature, and they never hear that whether it made any difference on usage, it’s going to be harder to keep them motivated. This is especially important when you’re doing quick iterations — workflow changes and copy tweaks are not, generally, the meaty problems that developers are intrinsically excited to work on. But if those “boring” changes lead to more customers, that’s something to be excited about.
When you’re solving an unknown problem — and really, the vast majority of “making it better” problems are unknown — you benefit from using all the brains in your company. But when you don’t follow up with what you learned, you’re handicapping those brains. Suppose you learned that removing configuration options increased conversion, but you don’t share that with your team. How are they going to know to stop thinking up power-user features?
If you are like me, you probably think you are being very clear in your followups, and you probably are not.
How to be really clear when closing the loop
We had the best results with KISSinsights when I gave a verbal followup in this approximate format:
- Remember, our goal was to improve [metric]
- We started at [number]
- We deployed [these changes] to try and move that number
- Now we’re at [number]
- We think these changes did work / might work but we want to watch for another week / didn’t work
- Here’s any anecdotal evidence from customer interviews, support emails, tweets, etc.
My goal for moving forward is to share this information in writing as well – discussing it in meetings is great, but lots of people just do better grokking the written word. I also think, as we move from very short-term goals to longer-term goals, it will be more useful to have easily-searchable written records to look back on.
Popularity: 4% [?]
