Understanding your t-test result
This calculator uses the conventional two-tailed significance threshold of α = 0.05, the most common default in scientific research.
| Two-tailed p-value | Conclusion at α = 0.05 |
|---|---|
| p < 0.05 | Statistically significant — reject the null hypothesis of no difference |
| p ≥ 0.05 | Not statistically significant — fail to reject the null hypothesis |
- Statistical significance is not the same as practical importance: with a large enough sample, even a tiny, practically meaningless difference can produce a p-value below 0.05.
- The two-sample test used here is Welch's t-test, which does not assume equal variances between groups; results can differ slightly from tools that use the classic pooled-variance (Student's) t-test.
- The t-test assumes the sampled data are independent observations and that the sampling distribution of the mean is approximately normal — satisfied either by roughly normal underlying data or by a large enough sample size via the central limit theorem.
- This is a summary-statistics calculator: it takes the mean, standard deviation and size you already computed from your data rather than a raw data list.
What is a t-test?
A t-test is a statistical hypothesis test used to determine whether an observed difference between means is larger than would be expected from random sampling variation alone. This calculator supports two forms: a one-sample t-test, which compares a sample mean to a fixed hypothesized value, and a two-sample t-test, which compares the means of two independent samples.
For the two-sample case, this calculator uses Welch's t-test, which does not assume the two groups have equal variances. Welch's test uses the Welch–Satterthwaite equation to compute an adjusted (and often non-integer) degrees of freedom, which generally produces a more reliable result than the classic Student's pooled-variance t-test when the two samples' standard deviations differ.
Both versions report a two-tailed p-value, meaning it measures evidence of a difference in either direction (higher or lower), and the result is compared against a conventional significance threshold of α = 0.05: p-values below 0.05 are typically labeled statistically significant, and values at or above 0.05 are not.
How to use this t-test calculator
- Choose the test type: one-sample (compare a sample mean to a fixed value) or two-sample (compare two independent sample means).
- For a one-sample test, enter the sample mean, standard deviation and size, and the hypothesized population mean (μ₀) you are testing against.
- For a two-sample test, enter the mean, standard deviation and size for both samples.
- Read the t-statistic, degrees of freedom and two-tailed p-value, and compare the p-value to your chosen significance level (commonly α = 0.05).
The formula behind the t-test
One-sample t-statistic: t = (x̄ − μ₀) / (s / √n), with degrees of freedom df = n − 1, where x̄ is the sample mean, s is the sample standard deviation, and μ₀ is the hypothesized population mean.
Two-sample Welch's t-statistic: t = (x̄₁ − x̄₂) / √(s₁² / n₁ + s₂² / n₂). The degrees of freedom are estimated with the Welch–Satterthwaite equation: df = (s₁²/n₁ + s₂²/n₂)² / [ (s₁²/n₁)² / (n₁ − 1) + (s₂²/n₂)² / (n₂ − 1) ], which does not assume the two population variances are equal.
The two-tailed p-value is p = 2 × (1 − T(|t|, df)), where T is the cumulative distribution function of Student's t-distribution with the computed degrees of freedom.
Worked example (one-sample): mean = 52.1, standard deviation = 4.5, n = 30, hypothesized mean μ₀ = 50. t = (52.1 − 50) / (4.5 / √30) ≈ 2.556, df = 29, and the two-tailed p-value is below 0.05, so the difference is statistically significant at the 5% level.
Common mistakes
- Treating statistical significance as practical importance — a very small p-value only indicates the difference is unlikely to be due to chance, not that the difference is large or meaningful in practice.
- Assuming this calculator pools the two samples' variances — it uses Welch's test, which allows the two variances to differ, so its degrees of freedom and p-value can differ from a pooled-variance calculation.
- Comparing a two-tailed p-value against a one-tailed decision rule, which effectively halves the intended significance threshold and inflates the apparent significance.
- Applying an independent-samples t-test to paired or repeated-measures data (such as before-and-after measurements on the same subjects), which requires a paired t-test instead.
- Running many t-tests on the same data without adjusting the significance threshold, which raises the chance of at least one false positive well above 5%.
常见问题
What is the difference between a one-sample and two-sample t-test?
A one-sample t-test compares a single sample's mean to a fixed, hypothesized value (μ₀). A two-sample t-test compares the means of two independent samples to each other, testing whether their difference is larger than expected from random variation alone.
Why does this calculator use Welch's t-test instead of Student's t-test?
Welch's t-test does not assume the two samples have equal variances, which makes it more robust when the sample standard deviations differ — a common situation in practice. It uses the Welch–Satterthwaite equation to compute an adjusted, often non-integer degrees of freedom rather than the simple n₁ + n₂ − 2 used by the classic pooled-variance test.
What does a p-value below 0.05 mean?
It means that, assuming the null hypothesis of no true difference is correct, a difference this large or larger would occur less than 5% of the time due to sampling variation alone. By the conventional α = 0.05 threshold, this is labeled statistically significant, though it says nothing about the size or practical importance of the difference.
Is statistical significance the same as practical importance?
No. A statistically significant result (p < 0.05) only indicates the observed difference is unlikely to be due to chance. With a large enough sample size, even a very small, practically unimportant difference can be statistically significant, so the effect size should always be reported and considered alongside the p-value.
Why is the degrees of freedom for a two-sample test not a whole number?
Welch's t-test computes degrees of freedom with the Welch–Satterthwaite equation, which combines the two samples' variances and sizes in a way that generally produces a non-integer result. This adjustment is what allows the test to remain accurate when the two groups' variances are unequal.
What sample sizes does this t-test require?
Each sample needs at least 2 observations for its standard deviation and degrees of freedom to be defined. In practice, the t-test performs best with roughly normal data or, via the central limit theorem, with a large enough sample size (commonly n of 30 or more per group is used as a rough guide) even when the underlying data are somewhat non-normal.
参考文献
- National Institute of Standards and Technology (NIST). NIST/SEMATECH e-Handbook of Statistical Methods — two-sample t-test for equal means. nist.gov.
- Welch BL. The generalization of 'Student's' problem when several different population variances are involved. Biometrika 1947; 34(1-2): 28-35.
- Satterthwaite FE. An approximate distribution of estimates of variance components. Biometrics Bulletin 1946; 2(6): 110-114.
- Moore DS, McCabe GP, Craig BA. Introduction to the Practice of Statistics. W. H. Freeman (one- and two-sample t procedures).
- Student (Gosset WS). The probable error of a mean. Biometrika 1908; 6(1): 1-25 (origin of the t-distribution).