6 Introduction to Sample Size Determination

Outline of points mentioned in ICH E9 Section 3.5 ‘Sample Size’.

Sample size specification in study protocol/report

• •

  primary variable (endpoint used to measure efficacy)

• •

  test statistic/form of analysis

• •

  null hypothesis

• •

  alternative ’working’ hypothesis

• •

  Error rates (defined over-leaf)

  • –

    type I error rate (5% two-sided test)

  • –

    type II error rate (10 or 20%)

• •

  how to deal with withdrawals and protocol violations.

Additional specifications

• •

  details of sample size method (including estimates of variances, differences to be detected)

• •

  sensitivity analysis: assess impact of changes in parameter values

• •

  for confirmatory trials (phase III): “…assumptions should normally be based on published data or on the results of earlier trials”

• •

  multiplicity considerations (controlling error rates).

Analysis set (primary analysis)

• •

  ITT

• •

  per protocol.

Classical hypothesis testing $\to$ use p-values to determine which of two competing hypotheses to draw from available data, $H_0$ versus $H_1$, say.

P-value: the probability $p$ of obtaining a test result as extreme or more extreme than that observed assuming the null hypothesis $H_0$ is true.

Choose the size of test is given by the value $\alpha$.

If reject $H_0$ and conclude data inconsistent with null. Often $\alpha =0.05$ is used.

Methods based upon experimental data carry some risk of drawing a false conclusion

		Truth
		$H_0$ true	$H_1$ true
Decision	Fail to reject $H_0$	-	Type II error
made	Reject $H_0$	Type I error	-

Unnumbered Figure: Link

• •

  type I error rate $\alpha$

• •

  type II error rate $\beta$

• •

  critical value $t_{crit}$

• •

  power $=1-\beta =P(\text{reject}{H}_0\mid H_1\text{ true})$

Consider (approx ) normally distributed test statistic $Z\mathrm{\sim }N\mathbf{}\mathrm{(}\theta \mathrm{,}\mathrm{1}\mathrm{)}$

$\theta =0$ under $H_{\mathrm{0}}$ and $\theta \mathrm{=}{\theta }^{\mathrm{\star }}$ under $H_{\mathrm{1}}$

Set power equal to target value: $P\mathrm{(}Z\mathrm{\ge }{\mathrm{\Phi }}^{\mathrm{-}\mathrm{1}}\mathrm{(}\mathrm{1}\mathrm{-}\alpha \mathrm{)}\mathrm{\mid }{H}_{\mathrm{1}}\mathbf{\text{ true}}\mathrm{)}\mathrm{=}\mathrm{1}\mathrm{-}\beta$

$$\Rightarrow 1-\mathrm{\Phi }\left({\mathrm{\Phi }}^{-1}(1-\alpha )-{\theta }^{\star }\right)=1-\beta$$

$$\Rightarrow {\theta }^{\star }={\mathrm{\Phi }}^{-1}(1-\alpha )+{\mathrm{\Phi }}^{-1}(1-\beta )$$

Note $\theta \mathrm{=}\theta \mathbf{}\mathrm{(}n\mathrm{)}$, solve equation for $n$.

Data

$Z_i\sim N(\mathrm{\Delta },{\sigma }^2)$, $i=1,\mathrm{\dots },n$ iid with known variance ${\sigma }^2$.

Hypotheses

$H_0:\mathrm{\Delta }=0$ vs. $H_1:\mathrm{\Delta }>0$.

Test statistic

$Z=\sqrt{n}\frac{\overline{Z}}{\sigma }$ with $\overline{Z}=\frac{1}{n}{\sum }_{i=1}^n{Z}_i$.

Critical value

${\mathrm{\Phi }}^{-1}(1-\alpha )$.

Desired power: if $\mathrm{\Delta }={\mathrm{\Delta }}^{\star }$ (smallest clinically relevant difference), probability for rejection of the null hypothesis at least $1-\beta$.

Non-centrality parameter

${\theta }^{\star }=\sqrt{n}{\displaystyle \frac{{\mathrm{\Delta }}^{\star }}{\sigma }}$.

Required sample size

$$n=\frac{{({\mathrm{\Phi }}^{-1}(\alpha )+{\mathrm{\Phi }}^{-1}(\beta ))}^2}{{\mathrm{\Delta }}^{\star 2}}{\sigma }^2$$

Data

$X_{ij}\sim N({\mu }_i,{\sigma }^2)$, $i=1,2$, $j=1,\mathrm{\dots },n_i$ with $n_1=rn$ and $n_2=(1-r)n$ iid. Denote $\mathrm{\Delta }={\mu }_1-{\mu }_2$.

Hypotheses

$H_0:\mathrm{\Delta }=0$ vs. $H_1:\mathrm{\Delta }={\mathrm{\Delta }}^{*}>0$.

Test statistic

$T=\sqrt{r(1-r)n}{\displaystyle \frac{{\overline{X}}_1-{\overline{X}}_2}{\sigma }}$.

Variance

${\sigma }^2$ known.

Exercise day 3: non-centrality parameter? sample size?

Data and hypotheses as for 2-sample Gauss test, but unknown variance.
Test statistic:

$$T=\sqrt{r(1-r)n}\frac{{\overline{X}}_1-{\overline{X}}_2}{S}$$

with

$$S^2=\frac{1}{n-2}\sum _{i=1}^2\sum _{j=1}^{n_i}{(X_{ij}-{\overline{X}}_i)}^2$$

Non-centrality parameter
${\theta }^{\mathrm{\star }}\mathrm{=}\sqrt{r\mathbf{}\mathrm{(}\mathrm{1}\mathrm{-}r\mathrm{)}\mathbf{}n}\mathbf{}{\displaystyle \frac{{\mathrm{\Delta }}^{\mathrm{\star }}}{\sigma }}$

approximate sample size

$$n=\frac{1}{r(1-r)}\frac{{({\mathrm{\Phi }}^{-1}(\alpha )+{\mathrm{\Phi }}^{-1}(\beta ))}^2}{{\mathrm{\Delta }}^{\star 2}}{\sigma }^2$$

Under $H_0:\mathrm{\Delta }=0$: $T\sim t(n-2)$.

Under $H_1:\mathrm{\Delta }={\mathrm{\Delta }}^{\star }$: $T\sim t(n-2,{\theta }^{\star })$ with ${\theta }^{\star }=\sqrt{r(1-r)n}\frac{{\mathrm{\Delta }}^{\star }}{\sigma }$.

Sample size: smallest $n$ such that

$$1-T_{n-2,{\theta }^{\star }}(t_{n-2,1-\alpha })\ge 1-\beta .$$

Equation cannot be solved for $n$ explicitly.