Uncertainty,
-standard errors and
-confidence intervals
LAST WEEK: Jennifer undertook a determined journey to find out just how clever Jessie (left) is compared to all the other dogs in the population.
-THIS WEEK: What if we don’t know anything about the population?
-
The whole game
--
-
- We want to estimate some population value (e.g. some average value - a mean) -
- Confidence intervals help us quantify uncertainty around that estimate -
- To construct a confidence interval, we use the standard error -
- We can estimate the SE by dividing the standard deviation by the number of people in our sample (specifically, by its square root): -
\[ -SE = \frac{\sigma}{\sqrt N} -\]
--
-
- Lower and upper limits of the confidence interval can be estimated as: -
\[ -\text{CI limits} = mean \pm (1.96 \times{SE}) \\ -\]
-
--Doomscrolling
-“… refers to a unique media habit where social media users persistently attend to negative information in their newsfeeds about crises, disasters, and tragedies.”
-- Sharma, Lee, and Johnson (2022)
-
Research question...
-How much does an average person doomscroll?
-
Uncertainty in research and estimation
-The problem: Each time we take a sample, we get a different estimate.
-How do we know if our estimate is accurate and close to the real population value?
-
Uncertainty in research and estimation
-The problem: Each time we take a sample, we get a different estimate.
-How do we know if our estimate is accurate and close to the real population value?
-
Sampling distribution revisited
--
-
- We can plot the estimates in a histogram to see how they’re distributed -
- x axis shows the spread of the values, y axis shows how many times each value occurs -
Sampling distribution revisited
--
-
- We can repeat the process infinite number of times - as long as our sample is large enough, the sampling distribution of the estimates will be normal -
- What is “large enough”? Some textbooks say 30, but more is often needed in real research. -
Normal distribution is awesome
--
-
- Normal distribution is useful, because we know a lot about it - a good way to describe it is by using a mean (central value) and a standard deviation (spread of the scores) -
- Standard error is the term we use for standard deviation when talking about sampling distributions -
Normal distribution is awesome
--
-
- We know that 95% of scores will fall within approximately 2 standard errors from the mean (or 1.96 if we want to precise) -
- We can use this knowledge to construct an interval around the mean showing us a range of plausible values for the population value -
Normal distribution is awesome
--
-
- We know that 95% of scores will fall within approximately 2 standard errors from the mean (or 1.96 if we want to precise) -
- We can use this knowledge to construct an interval around the mean showing us a range of plausible values -
Confidence intervals
-Average doomscrolling time for the sample: 101 minutes
-Standard deviation: 12.19
-Standard error: 6.095
-\[ -\text{Lower CI limit} = \text{sample mean} - 1.96 \times\text{SE} \\ -\text{Upper CI limit} = \text{sample mean} + 1.96 \times\text{SE} -\]
-\[ -\text{Lower CI limit} = 101 - 1.96 \times6.095 = 89.054\\ -\text{Upper CI limit} = 101 + 1.96 \times6.095 = 112.946 -\]
-
Confidence intervals for small samples
--
-
Remember: sampling distribution of the mean will have a normal shape as long as the sample size large enough
-Smaller samples don’t approximate the normal sampling distribution very well. Because of this, we can’t rely on the value 1.96 to give us accurate intervals.
-Instead, we can use the t-distribution
-
The t-distribution
-
-
-
The “critical t value” - i.e. the value of t which will give us the most accurate estimate of the confidence interval - depends on degrees of freedom (df), which in our case are related to the sample size.
-When working with the mean, the degrees of freedom are calculated as N - 1.
-
The t-distribution
-
-
-
Instead of multiplying the standard error by 1.96, we multiply by the critical t value.
-For example, in our sample of 4, the df is 4 - 1 = 3. Move the slider to df = 3 to see that the critical t value for 3 is 3.182
-
t-based confidence intervals:
-Average doomscrolling time for the sample: 101 minutes
-Standard error: 6.095
-Critical t value: 3.182
-\[ -\text{CI Limits} = \text{mean} \pm3.182 \times\text{SE} \\ -\text{CI Limits} = 101 \pm3.182 \times\text{6.095} \\ -\text{CI Limits} = [81.606, 120.394] -\]
--
-
- Compare the new confidence interval [81.61, 120.394] with the confidence interval we got using the value 1.96: [89.05, 112.95]. The new CI is wider! -
t-based confidence intervals:
-
-
-
This is to be expected - we have a tiny sample (N = 4), so there is a lot of uncertainty around whether the estimate of 101 minutes is actually representative of the population.
-The larger the sample, the tighter the confidence intervals,. because the critical t gets smaller and smaller (note how t approaches 1.96 as the sample size (df) increases)
-
-
-
We take samples over and over again, compute the mean, and construct confidence intervals around that mean - 95% of them will contain the population value, the remaining 5% will not.
-This is known as an interval with 95% coverage. 95% is the most common value that we choose, but it can take on other values as well (e.g 50%, 90%, 99%).
-
How *not* to interpret confidence intervals
-
Nope:
-“We can be 95% confident that the population value falls between 81.61 and 120.39.”
-How *not* to interpret confidence intervals
-Also nope:
-“There is 95% probability that the population value falls between 81.61 and 120.39.”
-
How to interpret confidence intervals
-
Correct interpretation
-ASSUMING THAT our sample is one of the 95% producing confidence intervals that contain the population value, then the population value for time spent doomscrolling per day falls somewhere between 81.61 and 120.39 minutes.
-The whole game (again)
--
-
- We want to estimate some population value (e.g. some average value - a mean) -
- Confidence intervals help us quantify uncertainty around that estimate -
- To construct a confidence interval, we use the standard error which we can estimate as: -
\[ -SE = \frac{\sigma}{\sqrt N} -\]
--
-
- Lower and upper limits of the confidence interval can be estimated as (replacing 1.96 with critical t for small samples): -
\[ -\text{CI limits} = mean \pm (1.96 \times{SE}) \\ -\]
-
The bigger picture…
--
-
When interpreting estimates and confidence intervals for your sample - always consider them as just one of many different possible estimates
-This is why replication is important in science - our sample could easily be the one that misses the population value
-Always be vary of studies placing too much confidence on a single finding
-
