3 Continuous random variables

A continuous random variable is not limited to discrete values, but can take any value within one or several intervals on the real line.

Examples of continuous random variables include weight, height, speed, and fluorescence intensity.

A continuous random variable can be described by its probability density function, PDF. For a continuous random variable, the probability of a single value is zero, i.e., \(P(X=x) = 0\). Probabilities are instead represented by the area under the probability density function.

For discrete random variables, probabilities are assigned to individual values through a probability mass function (PMF). For continuous random variables, probabilities are assigned to intervals through a probability density function (PDF).

Figure 3.1: Probability density function of the weight of a newborn baby.

The probability density function, \(f(x)\), is defined such that the total area under the curve is 1.

\[ \int_{-\infty}^{\infty} f(x) dx = 1 \]

Figure 3.2: The probability \(P(a \leq X \leq b)\) can be computed by computing the area under the probability density function between \(a\) and \(b\).

Probabilities are computed by integrating the probability density function over the interval of interest. The area under the curve from \(a\) to \(b\) is the probability that the random variable \(X\) takes a value between a and b:

\[P(a \leq X \leq b) = \int_a^b f(x) dx\]

The cumulative distribution function, CDF, sometimes called just the distribution function, \(F(x)\), is defined as:

\[F(x) = P(X \leq x) = \int_{-\infty}^x f(t) dt\]

The cumulative distribution function is increasing from 0 to 1.

\[P(X \leq x) = F(x)\]

As we know that the total probability (over all x) is 1, we can conclude that

\[P(X > x) = 1 - F(x)\] and thus

\[P(a \leq X \leq b) = F(b) - F(a)\]

Note that \(P(a \leq X \leq b) = P(a < X < b) = P(a < X \leq b) = P(a \leq X < b)\), as the probability of a single value is zero.

3.1 Parametric continuous distributions

Two important properties of a distribution are its expected value, \(\mu\), which describes the distribution’s location, and its variance, \(\sigma^2\), which describes the spread.

The expected value, or population mean, is defined as;

\[E[X] = \mu = \int_{-\infty}^\infty x f(x) dx\] We will learn more about the expected value and how to estimate a population mean from a sample later in the course.

The population variance is defined as the expected value of the squared distance from the population mean;

\[\sigma^2 = E[(X-\mu)^2] = \int_{-\infty}^\infty (x-\mu)^2 f(x) dx\]

These definitions are similar to the definitions of expected value and variance for discrete random variables, but with summation replaced by integration.

The square root of the variance is the standard deviation, \(\sigma\).

3.2 Normal distribution

The normal distribution (sometimes referred to as the Gaussian distribution) is a common bell-shaped probability distribution. Many continuous random variables can be described by the normal distribution or be approximated by the normal distribution. The normal distribution is especially important because many statistical methods are based on it directly, and many other distributions can be approximated by it.

The normal probability density function

\[f(x) = \frac{1}{\sqrt{2 \pi} \sigma} e^{-\frac{1}{2} \left(\frac{x-\mu}{\sigma}\right)^2}\]

describes the distribution of a normal random variable, \(X\), with expected value \(\mu\) and standard deviation \(\sigma\), \(e\) and \(\pi\) are two common mathematical constants, \(e \approx 2.71828\) and \(\pi \approx 3.14159\).

In short, we write \(X \sim N(\mu, \sigma^2)\) to denote that \(X\) is normally distributed with expected value \(\mu\) and variance \(\sigma^2\) (and standard deviation \(\sigma\)).

The normal distribution is symmetric around \(\mu\), and its density \(f(x) \rightarrow 0\) as \(x \rightarrow \infty\) and as \(x \rightarrow -\infty\).

As \(f(x)\) is known, the cumulative distribution function \(F(x) = \int_{- \infty}^x f(x) dx\) can be computed.

Figure 3.4: Normal probability density function and cumulative distribution functions.

Figure 3.5: Normal probability density function and cumulative distribution functions.

Using transformation rules we can define \(Z\), a random variable that is standard normally distributed (mean 0 and standard deviation 1).

\[Z = \frac{X-\mu}{\sigma}, \, Z \sim N(0,1)\]

Standardization transforms a normal random variable \(X\) to a standard normal random variable \(Z\), which makes probability calculations easy using tables and software. The cumulative standard normal distribution is denoted \(F(z)\) and can be computed in R using the function pnorm(z).

Figure 3.6: The shaded area under the curve is the tabulated value \(P(Z \leq z) = F(z)\).

Table 3.1: Cumulative distribution function for the standard normal distribution. The table gives \(F(z) = P(Z \leq z)\) for standard normal Z.

	0	0.01	0.02	0.03	0.04	0.05	0.06	0.07	0.08	0.09
0.0	0.5000	0.5040	0.5080	0.5120	0.5160	0.5199	0.5239	0.5279	0.5319	0.5359
0.1	0.5398	0.5438	0.5478	0.5517	0.5557	0.5596	0.5636	0.5675	0.5714	0.5753
0.2	0.5793	0.5832	0.5871	0.5910	0.5948	0.5987	0.6026	0.6064	0.6103	0.6141
0.3	0.6179	0.6217	0.6255	0.6293	0.6331	0.6368	0.6406	0.6443	0.6480	0.6517
0.4	0.6554	0.6591	0.6628	0.6664	0.6700	0.6736	0.6772	0.6808	0.6844	0.6879
0.5	0.6915	0.6950	0.6985	0.7019	0.7054	0.7088	0.7123	0.7157	0.7190	0.7224
0.6	0.7257	0.7291	0.7324	0.7357	0.7389	0.7422	0.7454	0.7486	0.7517	0.7549
0.7	0.7580	0.7611	0.7642	0.7673	0.7704	0.7734	0.7764	0.7794	0.7823	0.7852
0.8	0.7881	0.7910	0.7939	0.7967	0.7995	0.8023	0.8051	0.8078	0.8106	0.8133
0.9	0.8159	0.8186	0.8212	0.8238	0.8264	0.8289	0.8315	0.8340	0.8365	0.8389
1.0	0.8413	0.8438	0.8461	0.8485	0.8508	0.8531	0.8554	0.8577	0.8599	0.8621
1.1	0.8643	0.8665	0.8686	0.8708	0.8729	0.8749	0.8770	0.8790	0.8810	0.8830
1.2	0.8849	0.8869	0.8888	0.8907	0.8925	0.8944	0.8962	0.8980	0.8997	0.9015
1.3	0.9032	0.9049	0.9066	0.9082	0.9099	0.9115	0.9131	0.9147	0.9162	0.9177
1.4	0.9192	0.9207	0.9222	0.9236	0.9251	0.9265	0.9279	0.9292	0.9306	0.9319
1.5	0.9332	0.9345	0.9357	0.9370	0.9382	0.9394	0.9406	0.9418	0.9429	0.9441
1.6	0.9452	0.9463	0.9474	0.9484	0.9495	0.9505	0.9515	0.9525	0.9535	0.9545
1.7	0.9554	0.9564	0.9573	0.9582	0.9591	0.9599	0.9608	0.9616	0.9625	0.9633
1.8	0.9641	0.9649	0.9656	0.9664	0.9671	0.9678	0.9686	0.9693	0.9699	0.9706
1.9	0.9713	0.9719	0.9726	0.9732	0.9738	0.9744	0.9750	0.9756	0.9761	0.9767
2.0	0.9772	0.9778	0.9783	0.9788	0.9793	0.9798	0.9803	0.9808	0.9812	0.9817
2.1	0.9821	0.9826	0.9830	0.9834	0.9838	0.9842	0.9846	0.9850	0.9854	0.9857
2.2	0.9861	0.9864	0.9868	0.9871	0.9875	0.9878	0.9881	0.9884	0.9887	0.9890
2.3	0.9893	0.9896	0.9898	0.9901	0.9904	0.9906	0.9909	0.9911	0.9913	0.9916
2.4	0.9918	0.9920	0.9922	0.9925	0.9927	0.9929	0.9931	0.9932	0.9934	0.9936
2.5	0.9938	0.9940	0.9941	0.9943	0.9945	0.9946	0.9948	0.9949	0.9951	0.9952
2.6	0.9953	0.9955	0.9956	0.9957	0.9959	0.9960	0.9961	0.9962	0.9963	0.9964
2.7	0.9965	0.9966	0.9967	0.9968	0.9969	0.9970	0.9971	0.9972	0.9973	0.9974
2.8	0.9974	0.9975	0.9976	0.9977	0.9977	0.9978	0.9979	0.9979	0.9980	0.9981
2.9	0.9981	0.9982	0.9982	0.9983	0.9984	0.9984	0.9985	0.9985	0.9986	0.9986
3.0	0.9987	0.9987	0.9987	0.9988	0.9988	0.9989	0.9989	0.9989	0.9990	0.9990
3.1	0.9990	0.9991	0.9991	0.9991	0.9992	0.9992	0.9992	0.9992	0.9993	0.9993
3.2	0.9993	0.9993	0.9994	0.9994	0.9994	0.9994	0.9994	0.9995	0.9995	0.9995
3.3	0.9995	0.9995	0.9995	0.9996	0.9996	0.9996	0.9996	0.9996	0.9996	0.9997
3.4	0.9997	0.9997	0.9997	0.9997	0.9997	0.9997	0.9997	0.9997	0.9997	0.9998

Some values of particular interest:

\[F(1.64) = 0.95\] \[F(1.96) = 0.975\]

Because the normal distribution is symmetric, \[F(-z) = 1 - F(z).\]

\[F(-1.64) = 0.05\] \[F(-1.96) = 0.025\] \[P(-1.96 < Z < 1.96) = 0.95\]

3.2.1 Sum of two normal random variables

If \(X \sim N(\mu_1, \sigma_1^2)\) and \(Y \sim N(\mu_2, \sigma_2^2)\) are two independent normal random variables, then their sum is also a random variable:

\[X + Y \sim N(\mu_1 + \mu_2, \sigma_1^2 + \sigma_2^2)\]

and

\[X - Y \sim N(\mu_1 - \mu_2, \sigma_1^2 + \sigma_2^2)\]

The standard deviation of the sum or difference of two independent normal random variables is thus \(\sqrt{\sigma_1^2 + \sigma_2^2}\).

This can be extended to the case with \(n\) independent and identically distributed random variables \(X_i\) (\(i=1 \dots n\)). If all \(X_i\) are normally distributed with mean \(\mu\) and standard deviation \(\sigma\), \[X_i \sim N(\mu, \sigma^2)\] then \[\sum_{i=1}^n X_i \sim N(n\mu, n\sigma^2)\] and therefore \[\bar X = \frac{1}{n} \sum_{i=1}^n X_i \sim N\left(\mu, \frac{\sigma^2}{n}\right).\]

3.3 Central limit theorem, CLT

The central limit theorem states that the sum or mean of \(n\) independent and identically distributed random variables is approximately normally distributed, if \(n\) is large enough. The mean and variance of the sum are \(n\mu\) and \(n\sigma^2\), respectively.

As a result of the central limit theorem, sample means and sample proportions are often approximately normally distributed, when the sample is large enough. A rule of thumb is that the sample size \(n>30\) is large enough, but this of course depends on the distribution of the individual random variables.

Example 3.1 (A skewed distribution) A left skewed distribution has a heavier left tail than right tail. An example might be age at death of natural causes, as few individuals die of natural causes at young ages.

Figure 3.7: A left skewed distribution. Can for example show the distribution of age of a mouse who died of natural causes.

Randomly sample 3, 5, 10, 15, 20, 30 values and compute the mean value, \(m\). Repeat many times to get the distribution of mean values.

Figure 3.8: Distribution of sample means, where the means are computed based on random samples of sizes 3, 5, 10, 15, 20 and 30, respectively.

Note, mean is just the sum divided by the number of samples \(n\).

As the sample size increases, the distribution of the sample mean becomes more symmetric and closer to normal.

In addition to the normal distribution, three other continuous distributions are especially important in statistical inference: the \(\chi^2\), F, and t distributions

3.4 \(\chi^2\)-distribution

The random variable \(Y = \sum_{i=1}^n X_i^2\) is \(\chi^2\) distributed with \(n\) degrees of freedom, if \(X_i\) are independent identically distributed random variables \(X_i \in N(0,1)\).

In short \(Y \in \chi^2(n)\).

Figure 3.9: The \(\chi^2\)-distribution.

Example 3.2 The sample variance \(S^2 = \frac{1}{n-1} \sum_{i=1}^n (X_i-\bar X)^2\) is such that \(\frac{(n-1)S^2}{\sigma^2}\) is \(\chi^2\) distributed with \(n-1\) degrees of freedom.

3.5 F-distribution

If \(U \sim \chi^2(n_1)\) and \(V \sim \chi^2(n_2)\) are independent, then the ratio \[\frac{U/n_1}{V/n_2} \sim F(n_1, n_2)\] is F-distributed with \(n_1\) and \(n_2\) degrees of freedom.

Warning: Removed 47 rows containing missing values or values outside the scale range
(`geom_line()`).

Example 3.3 The ratio of two sample variances is F-distributed, if the two samples are independent and normally distributed.

3.6 t-distribution

If \(Z \sim N(0, 1)\) and \(U \sim \chi^2(v)\) are independent, then the random variable \[T = \frac{Z}{\sqrt{U/v}} \sim t(v)\] is t-distributed with \(v\) degrees of freedom.

A common example is based on the sample mean, \(\bar X\), and sample variance, \(S^2\), of a sample of size \(n\) from a normal distribtion. The standardized sample mean

\[T = \frac{\bar X - \mu}{S/\sqrt{n}} \sim t(n-1)\]

is t-distributed with \(n-1\) degrees of freedom.

3.7 Distributions in R

Probability density functions for the normal, t, \(\chi^2\) and F distributions can in R be computed using functions dnorm, dt, dchisq, and df, respectively.

Cumulative distribution functions can be computed using pnorm, pt, pchisq and pf.

Also, functions for computing an \(x\) such that \(P(X<x) = q\), where \(q\) is a probability of interest are available using qnorm, qt, qchisq and qf.

As with distributions, R uses the general naming scheme r<dist>, p<dist>, q<dist> and d<dist> for random number generation, cumulative distribution function, quantile function and probability density function, respectively.