Module 7

Peeking at the data: looks during a trial

A drug does nothing. If you check the results again and again, will you fool yourself?

Try an example:

The truth, and what the trial sees

Nobody can see the true response rates. The trial sees only patients, a few more at each look.

True (hidden)

Seen so far

Placebo (a dummy pill)

Drug

filled = got better outline = did not faded = not yet enrolled

Look	Patients per group	Got better: placebo	Got better: drug	Gap	p, one-sided

A trial with planned looks

The strict early bars come from a spending formula (Lan-DeMets), so they differ a little from the textbook O’Brien-Fleming bars. At each look an independent safety board (DSMB) checks the data.

1. How many looks?

2. Which rule?

3. Does the drug work?

Run

Each trial: 100 patients. Each simulated patient's result is a measurement on a bell curve. This approximates yes/no outcomes in big trials.

z: how big the gap is compared with luck. Above 1.96 on the drug’s side is the usual bar, one-sided p below 0.025 (module 2). Only the drug’s good side is drawn.

100 simulated trials, one dot each

StoppedNever stopped

Chance that a useless drug gives a false alarm

Every look at one-sided p < 0.025 (usual bar)Stricter bars (Pocock or O’Brien-Fleming-type)

False alarm: the trial stops though the drug does nothing.

Will the trial win if it keeps going?

Posterior: how sure we are today that the drug is better. Predictive probability: the chance the trial ends in a win if it runs to the end, given the data so far.

Plan: 100 patients per arm. Halfway, placebo has 15 of 50 better (30%). Pick the drug’s count.

Drug: better so far, of 50

Wins at the end (final posterior above 0.975)No win

Next, module 9 (CALGB 49907) uses this chance to stop a trial for futility.

Predict, then check

Real trials that stopped early

CALGB 49907 (cancer trial): stopped early.
ASTIN (stroke drug): stopped for futility. The trial had little chance of meeting its planned goal, so enrollment stopped.
DAWN (clot removal after stroke): stopped enrollment at 206 patients because the predictive probability of a win (module 7, below) passed 95%. At the end, the chance clot removal was better was above 99.9%.

Sources: Sokolova and Sokolov (2026), Clinical Trials in Practice, which cites Krams et al. (2003, ASTIN) and Nogueira et al. (2018), NEJM, DAWN.

Takeaway

Every extra look is another chance for a false alarm.
Plan the looks first, and use stricter bars so false alarms stay at 2.5%.
Stopping early saves patients, but the early estimate tends to be too big.
Predictive probability is the chance of a win at the end, not how sure we are today.