Module 3

Errors, power and sample size

How many patients does a trial need before we can trust its answer?

Try an example:

Two ways to be wrong

A trial is like a court case. The drug is "innocent" until the data convict it.

A false alarm is a Type I error (chance α); a miss is a Type II error (chance β). Power, the chance to catch a drug that works, is 1 minus β.

1. The drug

Patients on the old treatment improve 30% of the time.

of patients improve on the new drug

2. The trial

Patients in each group:

3. The bar for a clear difference

The bar: how far from "no difference" a result must land, in units of luck, to count as a clear difference in the drug's favor (one-sided). Bar:

Where trial results land: orange if the drug does nothing, teal if it works.

Assume the drug does nothing

One dot per simulated trial.

The truth, and what 20 trials see

Each card is one trial, with the group size and bar from the lab. Trials see only their own patients, never the true rates.

The truth (hidden from the trials)

Choosing the sample size

Before a trial starts, the team picks the smallest gain worth catching, the false-alarm bar, and the power they want. It also estimates how much patients vary. The only thing left to choose is the number of patients.

The bar is one-sided 2.5% (z = 1.96), as in the lab above.

Gain worth catching (old rate 30%):

Power you want:

Where does the curve reach the dashed line? Patients per group:

A shortcut

\[ n \approx \frac{(z_{\alpha} + z_{\beta})^2\,\big[p_0(1-p_0) + p_1(1-p_1)\big]}{(p_1-p_0)^2} \]

n is patients per group. \(p_0\) and \(p_1\) are the old and new improvement rates. \(z_\alpha = 1.96\) for the bar. \(z_\beta = 0.84\) for 80% power, 1.28 for 90%. The top is how much patients vary. The bottom is the gain, squared. It lands within a few patients of the curve.

More in Clinical Trials in Practice, chapter 11.

Halve the gain and you need about 4 times the patients. Going from 80% to 90% power needs about a third more.

Predict, then check

Real trials, real numbers

Planning a trial means asking: how many patients?

A worked design

The drug improves a score by 3 points. Patients vary by about 10 points.

Chance to catch it (power):

Improvement (points): Patient variation (points):

At the starting values, Clinical Trials in Practice also finds 467 patients in total.

Rare side effects need crowds

ICH E1 asks for about 1,500 patients exposed to a drug taken long-term for a non-life-threatening condition. Pick a true side-effect rate.

Sources: design numbers from Sokolova and Sokolov (2026), Clinical Trials in Practice. Safety numbers: ICH E1 guideline.

Regulator's corner

The ICH E20 draft describes a false alarm as "conclusions that truly inefficacious treatments are efficacious" (useless drugs called useful). That is our α, the Type I error.

ICH E20, section 3.3 (draft, June 2025, not final). Read the guideline (FDA).

Takeaway

A trial can raise a false alarm (Type I, α) or miss a drug that works (Type II, β). Power is 1 minus β.
More patients help detect a working drug without raising the false-alarm rate. Moving the bar trades one error against the other.
Half the effect costs about four times the patients.