What a confidence interval actually means
A confidence interval is a range of values, calculated from sample data, constructed using a method that is expected to capture the true population parameter in a specified percentage of repeated samples -- most commonly 95%. A 95% confidence interval does not mean there is a 95% probability that the true population value lies within that one specific calculated interval; the true value either is or is not within any given interval, and probability applies to the long-run performance of the method across repeated sampling, not to a single fixed interval after it has been calculated.
Hoekstra and colleagues (2014), publishing in Psychonomic Bulletin and Review, documented that this distinction is widely misunderstood, including among researchers and statistics students, many of whom interpreted a 95% confidence interval as if it directly described the probability that the true value fell within that specific interval. The technically correct interpretation is a statement about the reliability of the method used to construct the interval across many hypothetical repeated samples, not a probability statement about one already-observed interval.
What determines the width of a confidence interval?
The width of a confidence interval is determined primarily by three factors: the desired confidence level (commonly 90%, 95% or 99%), the variability of the underlying data (typically measured by the standard deviation or, for proportions, by p, the estimated proportion), and the sample size. Increasing the confidence level widens the interval, because capturing the true value more reliably across repeated samples requires a broader range. Increasing the sample size narrows the interval, because larger samples provide more precise estimates of the population value.
For a proportion, the margin of error in a confidence interval is calculated as z x sqrt[p(1-p)/n], where z is the z-score associated with the desired confidence level (1.96 for 95% confidence), p is the estimated proportion (0.5 is used when the true proportion is unknown, since it produces the largest, most conservative margin of error), and n is the sample size. Because sample size appears under a square root in this formula, reducing the margin of error by half requires roughly quadrupling the sample size, not simply doubling it.
The Cochran formula for sample size
The Cochran formula, published by statistician William Cochran in Sampling Techniques, provides a method for estimating the minimum sample size needed for a survey or study, calculated as n0 = (z^2 x p(1-p)) / e^2, where z is the z-score for the desired confidence level, p is the estimated proportion (0.5 if unknown), and e is the desired margin of error, expressed as a decimal. For a 95% confidence level (z = 1.96), a 5% margin of error (e = 0.05), and maximum variability (p = 0.5), the formula produces n0 = (1.96^2 x 0.5 x 0.5) / 0.05^2 = 384.16, rounded up to 385 respondents.
When sampling from a known, finite population rather than an effectively infinite one, Cochran's formula includes a finite population correction: n = n0 / (1 + (n0 - 1) / N), where N is the total population size. For example, applying this correction to a population of 2,000 reduces the required sample size from 385 to approximately 323, since a smaller total population requires proportionally fewer respondents to achieve the same margin of error.
Sample size, confidence level and margin of error: worked comparison
The table below shows how the required sample size, calculated using the Cochran formula with p = 0.5 for a large or infinite population, changes as the desired confidence level and margin of error change. Higher confidence levels and smaller margins of error both increase the required sample size, with margin of error having a particularly large effect because it appears squared in the denominator of the formula.
| Confidence level | Margin of error | z-score | Required sample size (p = 0.5) |
|---|---|---|---|
| 90% | 5% | 1.645 | 271 |
| 95% | 5% | 1.96 | 385 |
| 95% | 3% | 1.96 | 1,068 |
| 99% | 5% | 2.576 | 664 |
Common misreadings of '95% confidence'
Beyond the core probability misinterpretation described above, several related misreadings of confidence intervals appear frequently in research reporting and public communication. One common error is treating a 95% confidence interval as meaning that 95% of the individual data points in the sample fall within that range; a confidence interval describes uncertainty about a population parameter, such as a mean or proportion, not the spread of individual observations, which is instead described by the standard deviation or a prediction interval.
Another common error is assuming that a narrower confidence interval always indicates a more important or more reliable finding, without considering the effect size or practical significance of the estimate itself; a very large sample can produce a narrow confidence interval around an effect that is statistically precise but practically negligible. A third common error is comparing two confidence intervals that overlap and assuming their underlying values are not significantly different, or that non-overlapping intervals always confirm a significant difference; formally testing the difference between two estimates generally requires a dedicated statistical test rather than visual comparison of interval overlap alone.
Practical guidance for choosing a sample size
Researchers and survey designers typically choose a target confidence level and margin of error based on the precision required for the decision the data will inform, then use a formula such as Cochran's to calculate the corresponding minimum sample size before data collection begins. A 95% confidence level with a 5% margin of error is a common default in survey research and opinion polling, balancing a reasonably precise estimate against a practical, achievable sample size.
When the estimated proportion (p) is not known in advance, using p = 0.5 produces the most conservative, largest sample size estimate, since p(1-p) is maximized at p = 0.5; using a more specific estimate of p, if one is available from prior research or a pilot study, can reduce the required sample size when the true proportion is expected to be far from 50%. Organizations such as the American Association for Public Opinion Research publish detailed guidance on survey sample size planning and methodology reporting for researchers designing new studies.
Häufig gestellte Fragen
What does a 95% confidence interval actually mean?
A 95% confidence interval means that if the same sampling and calculation process were repeated many times, approximately 95% of the resulting intervals would contain the true population value. It does not mean there is a 95% probability that the true value falls within this one specific, already-calculated interval. This distinction, documented as a widespread misunderstanding by Hoekstra and colleagues (2014) in Psychonomic Bulletin and Review, is one of the most common misreadings of confidence intervals in both research and public communication.
How many people do I need to survey for a 95% confidence level?
Using the Cochran formula with a 95% confidence level (z = 1.96), a 5% margin of error, and maximum variability (p = 0.5), the required sample size for a large or effectively infinite population is approximately 385 respondents. If sampling from a smaller, known population, a finite population correction reduces this figure; for example, a population of 2,000 requires approximately 323 respondents to achieve the same confidence level and margin of error.
Does doubling the sample size halve the margin of error?
No. Because sample size appears under a square root in the margin of error formula, z x sqrt[p(1-p)/n], reducing the margin of error by half requires approximately quadrupling the sample size, not doubling it. This diminishing-returns relationship means that very large increases in sample size are needed to achieve relatively small improvements in precision once a study already has a reasonably large sample.
What is the Cochran formula used for?
The Cochran formula, from William Cochran's Sampling Techniques, estimates the minimum sample size needed to achieve a desired confidence level and margin of error when estimating a population proportion. It is calculated as n0 = (z^2 x p(1-p)) / e^2, and is widely used in survey research, market research and public health studies to determine how many respondents or participants are needed before data collection begins.
Does a narrow confidence interval mean a result is more important?
Not necessarily. A narrow confidence interval indicates greater statistical precision, often achieved through a larger sample size, but precision is not the same as practical significance. A very large study can produce a narrow, statistically precise confidence interval around an effect that is too small to be practically meaningful, while a smaller study might produce a wider interval around an effect that is large and practically important. Interpreting a confidence interval requires considering both its width and the size of the estimate it surrounds.
Quellenangaben
- Cochran WG. Sampling Techniques. 3rd ed. Wiley, 1977.
- Hoekstra R, Morey RD, Rouder JN, Wagenmakers EJ. "Robust misinterpretation of confidence intervals." Psychonomic Bulletin and Review, 2014;21(5):1157-1164.
- Kalton G. Introduction to Survey Sampling. Sage Publications, 1983.
- Moore DS, McCabe GP, Craig BA. Introduction to the Practice of Statistics. 9th ed. W.H. Freeman, 2017.
- American Association for Public Opinion Research. "Best Practices for Survey Research." AAPOR.org.