Watch people, not clicks
Many startup founders have this unhealthy fascination with data, metrics and analytics. They think all these numbers contain the answers to the mystery of business success.
More often than not, it’s just a dangerous distraction. I’ve learned this from experience, and today I want to share with you why you shouldn’t be too fixated on the numbers, and what to do instead.
The false promise of data-driven decision making
As entrepreneurs, we now have access to incredible amounts of data. Every interaction that happens on our websites and apps can be measured, quantified and analyzed.
In our euphoric embrace of data, we easily forget to take a step back and look at the bigger picture.
"When we let data drive our marketing, we all too often optimize for things that are easy to measure, not necessarily what matters most." - Ezra Fishman in The Dangers of Data-Driven Marketing
Startups are chaotic and messy. It can be frustrating to try and make sense out of it. It’s tempting to think you can A/B-test your way out of the mess. Just let the numbers decide!
But there are so many crucial elements to the success of your startup which can’t be quantified.
Statistical significance, local maximum & other stuff that’ll drive you crazy
If you’re running an early-stage startup, you probably don’t even have enough traffic volume to gain reliable insights from A/B testing in any decent amount of time, especially not at the speed of startup.
Peep Laja from ConversionXL recommends running tests until you get at least 250 conversions per variation, or even better 350-400. And even if you get all the little things right that can mess with A/B tests big time, there’s still the chance that you’re optimizing your way into a dead-end.
Get the big stuff right before you optimize for details
There’s surely value in data and A/B testing. But more likely than not, you, dear startup founder, shouldn’t spend too much time on optimizing for little wins, and instead swing for the fences.
Once you become an online-behemoth, sure, go ahead and start split testing button colors, but if you’re running one of these, you probably won’t be reading this blog. (Hey, Jeff Bezos, if you’re reading this, say hi in the comments!)
Numerical noise
Maybe you’ve got a fancy analytics platform with custom dashboards that allow you to constantly stay updated on all your KPIs… congratulations!
But I see too many founders just drowning in an ocean of data. How do you extract actionable insights from all this? How do you make the data meaningful?
If the data doesn’t translate into wisdom, then metrics are just a waste of time. You won’t find the answers in the analytics.
Do it like Pinterest
Back in the days when Ben Silbermann’s little startup was not yet a $5 billion company, he would walk into coffee shops and ask strangers to try out his website Pinterest.com.
He’d look over their shoulder, and observe them interacting with the site and ask them questions to better understand how he could improve the design and user experience.
That’s exactly what you should be doing. It's why I keep saying that startups should do things that don't scale, even if they ultimately want to become scalable startups. I care so much about this topic that I'm writing a practical how-to book for unscalable startup growth strategies.
Have random strangers use your product, and see how they react. Not just what they do on the screen, but also how they do it.
Watch their body language, their facial expressions, ask them how they feel when using your app. Watch how they use your website in the context of everything else that’s going on.
If you’ve got a B2B product, visit your users in their offices and see how they’re using it, and how they interact with it in the ecosystem of other things they use during their workday.
Grown-up startups should do this too.
Keep doing this even if you’re a mature company, even if you have millions of users and statistical significance, you still want to do these one-on-one observations regularly.
There’s so much you will learn from these. And you’ll be able to take away very different insights depending upon who you’re watching:
- a first time user,
- someone who uses your product for a week and
- someone who uses it already for a whole year.
A lot of the tiny details that might seem insignificant during the first week can turn into monstrocities within a year. There's a ton of value in observing your users "in the wild".
