Inference with the normal distribution
Lecture 17
The story so far
Our statistical problem
We want to understand how two numerical variables \(X\) and \(Y\) are related;
-
We do so through the lens of this model:
\[ Y=\beta_0+\beta_1X+\varepsilon. \]
\(\beta_1\) describes the relationship, and we want to know what it is;
-
Using a messy, meager, incomplete, imperfect sample of data, we construct our best guess:
\[ \widehat{Y}=b_0+b_1X. \]
\(b_1\) is our point estimate of \(\beta_1\). How reliable is it?
Grand summary from last time
Pay close attention to the similarities and differences:
The main ideas
In statistics, we quantify uncertainty with distributions:
- The sampling distribution describes how an estimate would vary across alternative datasets;
- the spread of this distribution communicates uncertainty/reliability;
- high probability regions provide a range of likely values (confidence interval);
- The null distribution describes how an estimate would vary across alternative datasets if the null hypothesis happened to be true;
- compare this to the actual point estimate to quantify the strength of evidence for competing claims.
We must approximate!
- Literally collecting new, alternative datasets is costly and impractical (if it was so easy, we would have made our original sample size larger in the first place);
- We fake it with simulation (bootstrapping or permutation) and approximate the sampling and null distributions with histograms;
- Next, we explore a set of pencil-and-paper mathematical techniques for approximating these distributions.
. . .
(This is where things start to resemble AP Stats)
Recall this example
Recall the openintro::loans_full_schema data frame:
10000 rows;
each row is an approved loan applicant;
-
the columns contain financial info about that person, including…
- annual income (in log $);
- amount of non-mortgage debt outstanding (in log $).
What is the relationship between these two variables?
Model fit with three observations
Model fit with five observations
Double the sample size
Double the sample size again
Double the sample size again
Double the sample size again
Double the sample size again
Double the sample size again
Double the sample size again
Double the sample size again
Double the sample size yet again
Double the sample size one more time
Use all the data we have
Let’s zoom in on that last distribution
Two observations
- Sampling distribution gets more concentrated in one spot;
- The shape resembles a bell curve more and more.
These are not accidents!
The central limit theorem
Theorem
As your sample size gets bigger and bigger, the sampling distribution of many statistics (OLS slope, sample mean, sample proportion, etc) will look more and more like a bell curve that is concentrating around the true value you are trying to estimate.
For the proof, take STA 240 and beyond!
Upshot
- Instead of using simulation and histograms to approximate sampling and null distributions, we can use the bell curve;
- We can use mathematical facts about the bell curve to compute confidence intervals and p-values;
- Before computers, this was the only game in town. Nowadays, you can just simulate. The results will probably be similar, especially when the sample size is large;
- Nevertheless, old school normal approximations are still used often, so let’s talk about them.
The normal distribution (AKA the bell curve)
The bell curve
Notation: \(N(\mu,\,\sigma)\)
- \(\mu\) (“mu”) is the mean. Controls center;
- \(\sigma\) (“sigma”) is the standard deviation. Controls spread.
The bell curve
#| '!! shinylive warning !!': |
#| shinylive does not work in self-contained HTML documents.
#| Please set `embed-resources: false` in your metadata.
#| standalone: true
#| viewerHeight: 500
library(shiny)
ui <- fluidPage(
titlePanel("The normal distribution"),
sidebarLayout(
sidebarPanel(
sliderInput(
inputId = "mu",
label = "Mean (μ)",
min = -5,
max = 5,
value = 0,
step = 0.1
),
sliderInput(
inputId = "sigma",
label = "Standard deviation (σ)",
min = 0.5,
max = 3,
value = 1,
step = 0.1
)
),
mainPanel(
plotOutput("density_plot")
)
)
)
server <- function(input, output) {
output$density_plot <- renderPlot({
x <- seq(-10, 10, length.out = 1000)
y <- dnorm(x, mean = input$mu, sd = input$sigma)
plot(
x, y,
type = "l",
col = "steelblue",
lwd = 3,
xlim = c(-10, 10),
ylim = c(0, 0.85),
xlab = "x",
ylab = "Density",
main = paste0("N(", input$mu, ", ", input$sigma, ")")
)
polygon(
c(x, rev(x)),
c(y, rep(0, length(y))),
col = adjustcolor("steelblue", alpha.f = 0.2),
border = NA
)
#abline(v = input$mu, col = "firebrick", lty = 2, lwd = 2)
})
}
shinyApp(ui, server)
The 68–95–99.7 rule
Let’s practice!
(in the container)
Inference with normal approximations (instead of simulation)
The main idea
The central limit theorem (CLT) says that if our sample size (\(n\)) is “large enough,” then our estimate of the slope (\(b_1\)) varies across datasets according to
\[ N(\text{mean} = \beta_1,\,\text{sd}=\text{SE}) \]
- \(\beta_1\) is the “true value” of the slope. Gosh we wish we knew it. It’s the ideal but unknown reference point that our estimates are wiggling around;
- “SE” stands for “standard error.” It’s just special vocabulary for the standard deviation of a sampling distribution. We’ll worry about how it’s calculated later.
The standard error
\[ N(\text{mean} = \beta_1,\,\text{sd}=\text{SE}) \]
The standard error measures the spread of the sampling distribution, which communicates uncertainty about the estimate. SE higher bad; SE lower good.
The value of the SE is influenced by two factors:
- the variability of the data: the more “all over the place” your data are, the less reliable the estimates, so SE goes up;
- sample size: the more (less) data you have, the lower (higher) the SE.
So, the SE is a tug-of-war between the noisiness of the data and how much of it you have.
Approximation
\[ N(\text{mean} = \beta_1,\,\text{sd}=\text{SE}) \]
- Using this is just as much an approximation as simulation + histogram;
- The bell curve only works for \(n\) “large enough;”
- Both \(\beta_1\) and SE are unknown. They need to be approximated from the data;
- For \(\beta_1\), we naturally use \(b_1\);
- Don’t worry about SE just yet. For now, be content that the computer can estimate it…
The standard error
It’s been staring us in the face the entire time:
linear_reg() |>
fit(mpg ~ wt, data = mtcars) |>
tidy()# A tibble: 2 × 5
term estimate std.error statistic p.value
<chr> <dbl> <dbl> <dbl> <dbl>
1 (Intercept) 37.3 1.88 19.9 8.24e-19
2 wt -5.34 0.559 -9.56 1.29e-10So how do we use it for interval estimation and hypothesis testing?