Routing

Run this on
every question

One decision, one lookup, three questions and one gate, asked before you open a query editor. Each exit names what you hand back.

Knowledge Layer knowledge repo · what was already tried on this surface, wins and losses alike prior store · what this metric has actually moved by, so the MDE is a number read before the question is scoped, written when the decision is made settled, stale, or new? and what is the prior? a question arrives who decides what, on this answer? do I trust the metric and the definition of success? is this description, a forecast, or a change? is the decision made once, or continuously? who assigned the treatment? Backlog nobody decides on it, or it is already decided hands back: a backlog line, not a method Measurement Framework fix the definition, the source of truth, the registry hands back: a number the rest of the map can stand on Exploratory Analytics deep dives · funnels · segmentation · opp sizing hands back: a hypothesis Statistical Modeling forecast · optimizer · uplift model hands back: a forecast, a ranking, an allocation Experimentation A/B · switchback · power · sequential · HTE hands back: an effect size Causal Inference match the method to how assignment happened hands back: an effect size, plus its assumptions Decision ship · kill · reallocate a named choice none, or made unless the lookup settles it yes a change once one ship-or-kill call no description what · where · how many · who a forecast how many will we get · for a plan continuously per user · per day · at volume you can, or could have it already happened 1 2 effect size effect size a model that acts on users still needs an experiment to prove it moves anything a hypothesis is a change question, so it re-enters here what ships changes the data, so the next question enters a different world what we learned

you can, or could haveinto Experimentation

Before it ships, it is a choice

Once it has launched, can you randomise? is a fact about the past. Asked early, it is a design decision you still own:

  • hold out a slice of users
  • stagger the rollout by market or by cohort
  • randomise the prompt, not the feature

it already happenedinto Causal Inference

Pick the method by how assignment happened

ShapeMethodUse it when
ITSone unit, one switch date, no untreated twin
DiDtreated and untreated units, both seen before and after
Synthetic controlone treated unit, many untreated ones to blend
Matchinggroups differ on things you can actually observe
IVsomething shifted uptake without touching the outcome

If none of these fit, you do not have a comparison group. Say that out loud rather than shipping the number anyway.