In the previous chapter, we used resampling techniques to estimate sampling distributions for hypothesis testing. If the null distribution is already known, or can be derived based on a set of assumptions, resampling is not necessary.
Parametric tests are hypothesis tests in which the null distribution of the test statistic is derived from a probability model described by a small number of parameters, such as the mean and variance of a normal distribution.
We can follow the same steps as before to perform a hypothesis test:
In this section we will present a few common situations in which the null distribution can be described parametrically.
A one-sample test for a mean compares the mean of one sample to a prespecified value.
For example, we might know that the weight of a mouse on normal diet is normally distributed with mean 24.0 g and standard deviation 3 g and want to compare the weight of a sample of 12 mice on high-fat diet to the known mean value for mice on normal diet.
The hypotheses:
\[H_0: \mu = \mu_0\] \[H_1: \mu \neq \mu_0\]
The alternative hypothesis, \(H_1,\) above is for the two-sided hypothesis test. Other options are the one-sided \(H_1\); \(H_1: \mu > \mu_0\) or \(H_1: \mu < \mu_0\).
If \[X \sim N(\mu, \sigma^2)\] (this could for example be the weight of a mouse on high-fat diet) then the sample mean \[\bar X \sim N\left(\mu, \frac{\sigma^2}{n}\right).\]
If \(\sigma\) is known under the null hypothesis, then the test statistic
\[Z = \frac{\bar X - \mu_0}{\frac{\sigma}{\sqrt{n}}}\] is normally distributed, \(\sim N(0,1)\).
In many situations the the population standard deviation is not known, but can instead be estimated from the sample. For large sample sizes, \(\sigma\) can be replaced by the sample standard deviation \(s\) and the test statistic will still be normally distributed according to the central limit theorem.
For small \(n\) and unknown \(\sigma\), if the observations are drawn from a normal distribution, we can use the one-sample t-test, that uses the test statistic
\[T = \frac{\bar X - \mu_0}{\frac{s}{\sqrt{n}}},\]
which is t-distributed with \(df=n-1\) degrees of freedom.
Once we have an appropriate test statistic, \(T\) with known distribution, we can compute the observed value, \(t_{obs}\) and use the null distribution to compute the p-value, \(P(|T| \geq |t_{obs}|)\).
In R, the functions pnorm and pt are useful for computing these probabilities. The one-sample t-test can also be performed using the function t.test, like this;
##The observed 12 mouse weights;
x <- c(25, 30, 23, 18, 31, 24, 39, 26, 36, 29, 23, 32)
## Under the null hypothesis mu=24
##Perform the t-test to investigate if the population mean differs from 24.
t.test(x, mu=24)
One Sample t-test
data: x
t = 2.3153, df = 11, p-value = 0.04092
alternative hypothesis: true mean is not equal to 24
95 percent confidence interval:
24.19742 31.80258
sample estimates:
mean of x
28 A two-sample test for a difference in means is used to determine if two population means are equal.
Two independent samples are collected (one from each population) and the means are compared. This test can for example be used to determine if a treatment group is different from a control group, in terms of the mean of a property of interest.
The null hypothesis is
\[H_0: \mu_2 = \mu_1.\]
The alternative hypothesis can either be two-sided
\[H_1: \mu_2 \neq \mu_1\] or one-sided
\[H_1: \mu_2 > \mu_1 \textrm{ or } H_1: \mu_2 < \mu_1.\]
Assume observations from both populations are normally distributed;
\[ \begin{aligned} X_1 \sim N(\mu_1, \sigma_1^2) \\ X_2 \sim N(\mu_2, \sigma_2^2) \end{aligned} \] Then it follows that the sample means will also be normally distributed;
\[ \begin{aligned} \bar X_1 \sim N(\mu_1, \sigma_1^2/n_1) \\ \bar X_2 \sim N(\mu_2, \sigma_2^2/n_2) \end{aligned} \]
The mean difference \(D = \bar X_2 - \bar X_1\) is thus also normally distributed:
\[D = \bar X_2 - \bar X_1 \sim N\left(\mu_2-\mu_1, \frac{\sigma_2^2}{n_2} + \frac{\sigma_1^2}{n_1}\right)\]
If \(H_0\) is true: \[D = \bar X_2 - \bar X_1 \sim N\left(0, \frac{\sigma_2^2}{n_2} + \frac{\sigma_1^2}{n_1}\right)\]
The test statistic: \[Z = \frac{\bar X_2 - \bar X_1}{\sqrt{\frac{\sigma_2^2}{n_2} + \frac{\sigma_1^2}{n_1}}}\] is standard normal, i.e. \(Z \sim N(0,1)\).
However, note that the test statistic requires the standard deviations \(\sigma_1\) and \(\sigma_2\) to be known.
What if the population standard deviations are not known?
If the sample sizes are large, we may replace the population standard deviations by the sample standard deviations and, according to the central limit theorem, assume that the test statistic is still normally distributed. The test statistic then becomes
\[Z = \frac{\bar X_2 - \bar X_1}{\sqrt{\frac{s_2^2}{n_2} + \frac{s_1^2}{n_1}}} \sim N(0,1)\]
and proceed as before.
For small sample sizes, if the populations are approximately normal, t-based methods are used.
\[t = \frac{\bar X_2 - \bar X_1}{\sqrt{\frac{s_2^2}{n_2} + \frac{s_1^2}{n_1}}}\]
If it can be assumed that both \(X_1\) and \(X_2\) are normally distributed and have equal variances, Student’s t-test can be used. For equal variances the pooled sample variance can be computed;
\[s_p^2 = \frac{(n_1-1)s_1^2 + (n_2-1)s_2^2}{n_1+n_2-2}\]
and the test statistic
\[t = \frac{\bar X_2 - \bar X_1}{\sqrt{s_p^2(\frac{1}{n_1} + \frac{1}{n_2})}}\]
is t-distributed with \(n_1+n_2-2\) degrees of freedom.
If equal variances cannot be assumed, Welch’s t-test can be used. This test does not assume equal variances, and is the default in R’s t.test. The test statistic is the same as above, but the degrees of freedom are approximated using the Welch-Satterthwaite formula.
The t-test is implemented in R, e.g. in the function t.test in the R-package stats, both Student’s t-test for equal variances and Welch’s t-test for unequal variances.
Example 7.1 (Pollen allergy) Let’s get back to the pollen example!
Assume that the proportion of pollen allergy in Sweden is known to be \(0.3\). Observe 100 people from Uppsala, 42 of these were allergic to pollen. Is there a reason to believe that the proportion who are allergic to pollen in Uppsala \(\pi > 0.3\)?
Null and alternative hypotheses
\[H_0:\, \pi=\pi_0\]
\[H_1:\, \pi>\pi_0,\]
where \(\pi_0=0.30\) in this example. Other possible alternative hypotheses are \(H_1: \pi<\pi_0\) or \(H_1:\pi \neq \pi_0,\) but in this particular example we are only interested in the alternative that \(\pi > \pi_0\).
Significance level
Set the significance level to \(\alpha=0.05\).
Test statistic
Here, we will use \(X\), the number of allergic persons in a random sample of size \(n=100\). The observed value is \(x_{obs} = 42\).
Null distribution
\(X\) is binomially distributed, so there is no need to use resampling here, we can use the binomial distribution to answer the question.
p-value
The p-value is the probability of observing a value at least as extreme as the observed value, if \(H_0\) is true. In this case, the p-value is the probability of \(x_{obs}\) or something higher,
\(P(X \geq 42) = 1 - P(X \leq 41)\) = [1-pbinom(41,100,0.3)] = 0.007174
Can \(H_0\) be rejected?
As \(p<0.05\) \(H_0\) is rejected and we conclude that there is reason to believe that the proportion of people with pollen allergy in Uppsala is higher than 0.3.
This p-value can also be computed using the exact binomial test;
binom.test(42, 100, 0.3, alternative="greater")
Exact binomial test
data: 42 and 100
number of successes = 42, number of trials = 100, p-value = 0.007174
alternative hypothesis: true probability of success is greater than 0.3
95 percent confidence interval:
0.3364797 1.0000000
sample estimates:
probability of success
0.42 An alternative approach is to use the central limit theorem and use the normal approximation.
As a result of the central limit theorem, the distribution of number or proportion of allergic individuals in a sample of size \(n\) is approximately normal. At least if the sample is large enough. A common rule of thumb is that both \(n \pi_0\) and \(n(1-\pi_0)\) should be at least 5.
Here, the sample size is 100 and \(n\pi_0 = 100*0.3 = 30\) and \(n(1-\pi_0) = 100*0.7 = 70\), which are both greater than 5, so the normal approximation can be used.
This approximate test for a proportion is implemented in the R function prop.test,
1-sample proportions test with continuity correction
data: 42 out of 100, null probability 0.3
X-squared = 6.2976, df = 1, p-value = 0.006045
alternative hypothesis: true p is greater than 0.3
95 percent confidence interval:
0.3372368 1.0000000
sample estimates:
p
0.42 but it can of course also be calculated using only the normal distribution table.
The normal distribution has two parameters, mean and standard deviation.
From the binomial distribution we know that \(E[X] = n\pi\) and \(var(X) = n\pi(1-\pi)\).
The standard error of \(X\) is
\[SE=\sqrt{n\pi(1-\pi)}\]
If \(H_0\) is true \(\pi=\pi_0\) and
\[X \sim N\left(n\pi_0, n\pi_0(1-\pi_0)\right)\]
With this null distribution and our observed value \(x_{obs} = 42\), the p-value can be computed.
\(p = P(X \geq 42)\)
As we are now approximating a discrete distribution using a continuous distribution we need to use a trick called continuity correction, which simply means that we let each integer be represented by a region \(\pm 0.5\) from its value. Hence,
\[p = P(X \geq 42) = [continuity \; correction] = \] \[P(X \geq 41.5) = P\left(Z \geq \frac{41.5 - n\pi_0}{\sqrt{n\pi_0(1-\pi_0)}}\right) = \] \[P\left(Z \geq \frac{41.5-30}{\sqrt{100*0.3*(1-0.3)}}\right) = P(Z \geq 2.51) = 1 - P(Z \leq 2.51) = [Z-table] = 0.0060\]
As 0.00605<0.05 we reject \(H_0\) and conclude that there is reason to believe that the proportion of people allergic to pollen in Uppsala is greater than 0.3.
\[ \begin{aligned} H_0: \pi_1 - \pi_2 = 0\\ H_1: \pi_1 - \pi_2 \neq 0 \end{aligned} \]
Alternatively, a one sided alternative hypothesis can be used; \[H_1: \pi_1 - \pi_2 >0\text{ or }H_1: \pi_1 - \pi_2 < 0.\]
If the sample sizes are large the normal approximation can be used, an appropriate test statistic is
\[Z = \frac{P_1 - P_2}{\sqrt{P(1-P)\left (\frac{1}{n_1} + \frac{1}{n_2}\right)}},\]
where \(P\) is the proportion in the merged sample of size \(n_1 + n_2\). \(Z \in N(0,1)\) and p-value can be computed using the standard normal distribution.
The two-sample test for proportions is implemented in the function prop.test.
The test of equal variances in two groups is based on the null hypothesis
\[H_0: \sigma_1^2 = \sigma_2^2\]
If the two samples both come from populations with normal distributions, the sample variances (also random variables) are
\[S_1^2 = \frac{1}{n_1-1} \sum_{i=1}^{n_1} (X_{1i}-\bar X_1)^2\\ S_2^2 = \frac{1}{n_2-1} \sum_{i=1}^{n_2} (X_{2i}-\bar X_2)^2\]
It can be shown that \(\frac{(n_1-1)S_1^2}{\sigma_1^2} \sim \chi^2(n_1-1)\) and \(\frac{(n_2-1)S_2^2}{\sigma_2^2} \sim \chi^2(n_2-1)\).
Hence, the test statistic for comparing the variances of two populations
\[F = \frac{S_1^2}{S_2^2}\] is \(F\)-distributed with \(n_1-1\) and \(n_2-1\) degrees of freedom.
In R a test of equal variances can be performed using the function var.test.