AI Product Sense Drill
An ordinary product case asks you to design for a user in a system that behaves the same way every time. An AI case adds a component that is right most of the time and confidently wrong the rest, and that moves every interesting decision: what the model does unsupervised, how you would notice it failing, what the user sees when it does. This is why a recited framework fails so visibly here. A framework tells you which boxes to fill. It does not tell you where to put the line between what a model decides and what a person decides, and that line is the answer.
Fifteen decisions across three products. Every choice returns the reasoning, not a score.
Pick a product
Three shapes, three different answers. What changes between them is not the framework, it is which failure is expensive.
Weekly active users have plateaued. Developers who stay are extremely engaged, and most people who try it never return after the first session. Leadership wants an order-of-magnitude increase in weekly actives.
Three engineers. One quarter. The model itself is not yours to change.
Where do you start?
A first session fails most often when the assistant edits several files and gets one wrong. What do you change?
How would you know the change worked, before users tell you?
What is the failure you would most want to catch, and how would a user experience it?
What do you take to leadership as the measure, and what would make you stop?
Where that leaves you
Keep going.
0 of 5 decisions made.
Keep going
All labsNext in Product & Data · 12 min
Product Metrics Tree
DAU is a vanity metric for most products. Pick the product shape, get the right north star, find the guardrail your change might break.
Valuation · 3 min
Name That Multiple
A Wordle for finance. Guess a company’s EV/EBITDA band from the business quality, learn what drives it, share your score.