Are they different?
Does the drug group really differ from the placebo group, or is it luck?
Each set is 15 points on a circle. Real random points leave big and small gaps; made-up points are too even.
One circle has truly random points. Which?
The rule-made sets are too evenly spaced.
Your turn: can you make up random numbers?
Click 15 points on the circle, as randomly as you can.
Optional: how the gap score works (you can skip this)
Walk around the circle and measure each gap between neighboring points, as a share of the whole circle. Square each gap, add them up, and multiply by the number of points: \(s = T \sum (\text{gap}/L)^2\), with \(T = 15\) points and \(L = 100\) the circle's length.
Perfectly even points have every gap equal to 1/15, so \(s = 1\). All points at one spot leave one gap equal to the whole circle, so \(s = 15\).
Random points score \(2T/(T+1) = 1.875\) on average, and most land between 1.4 and 2.6.
Scores of 10,000 random sets. Yours is marked.
Idea from V. I. Arnold, Euler Groups and Arithmetics of Geometric Progressions, paragraph 9.
Same or different?
Each round is a small trial of a drug against a placebo (a dummy pill). The drug's true response rate (the share of all patients it would help) is hidden; in half the trials it equals the placebo's.
1. Patients per group
2. Your call
filled = improved outline = did not.
bright = what the trial recorded pale = what would have happened on the other treatment
Randomization
In physics, if you repeat an experiment the same way, you get almost the same result. So if you change one thing and the result changes, that change caused it.
Patients are not like that. Two patients on the same drug can do very differently, because of health, age, habits and things we cannot control.
Sicker patients tend to do worse no matter what they take. If more of them end up on placebo, the placebo group does worse because of who is in it, and the drug looks better even if it does nothing.
So we flip a coin for each patient: heads gets the drug, tails gets placebo. Then healthy and sick patients end up in both groups in about the same numbers. The coin also splits things we never measured, like sleep or diet. What is left is luck, and the randomization test below shows how big luck can be.
Idea from D. R. Cox, Planning of Experiments (1958).
Each column is one of 40 patients: top icon = result on the drug, bottom = on placebo.
The colored bar is each patient's health at the start: orange sicker, teal healthier.
Who decides who gets the drug? Try both. When patients choose, healthier ones tend to pick the drug. Watch how far the trial's answer lands from the true effect.
Why the gap is not the drug
The gap you see is the drug's effect on the people who took it, plus how different the groups already were.
Do the groups match?
A fair comparison needs groups alike before the drug, including in what nobody measured.
| Trait | Drug | Placebo | Match? |
|---|
The randomization test
In a trial, a coin flip decides who gets the drug. Suppose the drug does nothing, so each patient's result stays fixed. Re-randomizing flips the coins again and asks how big a gap luck makes.
Observed
After re-randomizing
Press "Re-randomize once".
Predict, then check
Examples
Each card re-randomizes 2,000 times in simulation. Orange bars are gaps at least as big as the real one.
Takeaway
- Chance makes differences too.
- A test asks: is the gap bigger than chance usually makes?
- Small trials fool everyone. Big trials let real gaps stand out.
This is why trials randomize: the coin flip itself tells us what gaps chance makes. The share of re-randomizations with a gap at least as big as the real one is the p-value.