You gotta love good survivorship bias stories like this. Especially those written by VC armchair quarterbacks. It's always the same examples reiterated over and over again.
I felt that Superhuman was super good at making investors and VCs enthused, but that’s not product market fit.
I guess it’s VC product fit. Or VC salesman fit or something.
As the other comment in this thread asks “the proof is in the pudding, does Superhuman have product market fit today?”
You should ignore this article. They ultimately did not get PMF
My experience with Superhuman has been terrible. Back in the day, I tried to sign up. Couldn't. Then they started allowing sign-ups, but there was a mandatory onboarding/sales call to get started...
Finally, some time ago managed to sign up in hopes it could help to manage multiple inboxes with a unified inbox for all of my accounts. Nope doesn't have that. Canceled my subscription immediately yet I kept receiving their spam for a while.
Overall, a very scammy vibe.
Does anyone know how Superhuman is doing these days?
It's been a bit quiet since their insane hype-cycle during the time this article was published.
Notably, Rahul copied the well known Sean Ellis metric then failed to credit the originator - and got famous. Kind of sketchy in my opinion.
obviously no real content in TFA, upvoters can elaborate why this is interesting
Is it just me, or is this bad statistics?
With a small sample size and large numbers of personas / categories you would expect to see a positive bump, even if there was no statistical relationship between the persona and the preference. Since you are only eliminating categories that don't happen to be represented in the subset you are testing, you can only ever actually go up.
For demonstration I rolled 20 dice randomly for 6 personas and 3 categories of preference:
1, 5, 4, 4, 4, 2, 6, 1, 4, 2, 1, 2, 5, 6, 3, 6, 3, 2, 1, 3
A, A, A, A, B, C, B, A, C, A, C, A, B, C, A, A, B, A, B, B
A = 10, B = 6, C = 4; Which gives me 20% for C
I restrict myself to just the numbers that voted for C (2, 4, 1, 6) removing all 3 and 5s
I now am left with:
1, 4, 4, 4, 2, 6, 1, 4, 2, 1, 2, 6, 6, 2, 1
A, A, A, B, C, B, A, C, A, C, A, C, A, A, B
This now gives me A = 8, B = 3, C = 4
And now I get 27%, a nice 35% boost! Even better than Superhuman's 10% boost. But this is all an illusion, there was absolutely no dependency between the persona and preference here, which you would only see with a large enough sample size.
Having followed this exact playbook to validate several products I can confidently say this will give you false positives and the only reliable way to determine when you have PMF is: accidentally get PMF on something so that you know what it feels like (it’s unmistakeable).
Peter Reinhardt from Segment has a must-watch talk for anyone interested in this topic https://youtu.be/_6pl5GG8RQ4?si=pogHC45L58U7K6mW
Never used Superhuman - has to be one of the most hyped startups with the least amount of IRL users that I know of
The article discusses a way to test product market fit prior to launching. After launch its obvious. But then the way it tests product market fit is asking its users if they would be disappointed if the product didn't exist.
But if you have users you can simply test product market fit to see engagement. If people are engaged, that's a good sign. If you have people that are signed up and aren't engaged, that's a bad sign.
The author uses Slack as an example of a company that has product market fit and over 50% of users surveyed would say they'd be "very disappointed" without Slack (the magic number to beat is 40%). I wonder how that poll would fare for Microsoft Teams, that has 320 million MAU compared to around 40 million for Slack (roughly, few years old but order of magnitude is ~10x). Does Slack have more of a product market fit than Teams because very few people would be disappointed if Teams stopped existing? Or is usage a better metric?