P-Value Calculator

Frequently Asked Questions

What does p = 0.03 mean?

If the null hypothesis were true and you repeated the experiment many times under the same conditions, you would observe a test statistic this extreme or more extreme in only 3% of those repetitions. It does not mean there is a 3% probability the null is true.

Should I always use a two-tailed test?

Use a two-tailed test by default unless you have a strong, pre-registered scientific reason to care only about one direction. One-tailed tests are legitimate in confirmatory experiments where the direction of the effect is specified in advance - not discovered after the fact.

What is the difference between p and α?

α is the significance threshold you set before the experiment (commonly 0.05). p is the probability computed from the data. If p < α, you reject H₀. If p ≥ α, you fail to reject. α is a decision rule; p is a measure of evidence.

Why are far-tail p-values (e.g. p = 10<sup>−15</sup>) approximations?

The numerical routines (incomplete beta continued fraction, erf approximation) converge accurately in the central region but lose relative precision very far in the tail. Values beyond about p = 10<sup>−6</sup> should be reported as order-of-magnitude estimates or as "p < 0.000001" in practice.

Provided by AllCalculators.io
Free online calculators for everyday. No registration required.

Estimates for informational purposes only.

Important Disclaimer: Estimates for informational purposes only.

This calculator provides estimates for informational purposes only. Results are based on assumptions and may not reflect actual outcomes. Consult qualified professionals in relevant fields before making important decisions based on these results.

JavaScript is required to use the interactive calculator above. The questions and answers below remain readable without JavaScript.

How This Calculator Works

A p-value is the probability of observing a test statistic at least as extreme as the computed one, assuming the null hypothesis H₀ is true. This tool maps your statistic onto the chosen sampling distribution and integrates the appropriate tail:

z (standard normal): Φ(z) = ½[1 + erf(z ÷ √2)] via the Abramowitz-Stegun approximation.

t (Student's): CDF evaluated via the regularized incomplete beta I_x(df/2, 1/2) with x = df/(df+t²), using a Lentz continued fraction.

χ²: regularized lower incomplete gamma P(df/2, χ²/2); always upper-tailed.

F: reuses the incomplete beta with x = d₁F/(d₁F+d₂); always upper-tailed.

A left tail returns the CDF; a right tail returns 1−CDF; a two-tailed test returns 2×min(CDF, 1−CDF).

What a P-Value Is - and Is Not

A p-value is not the probability that H₀ is true. It is not the probability the result occurred by chance. It is a conditional probability: P(data at least this extreme | H₀ is true and all model assumptions hold).

A non-significant p does not prove H₀ - absence of evidence is not evidence of absence. A highly significant p does not mean the effect is large or practically important. With enormous samples, a trivially small effect produces p < 0.001 while remaining meaningless in practice. Always pair p with a confidence interval and an effect-size measure such as Cohen's d or η².

Worked Example: z = 2.10, Two-Tailed

A z-test yields z = 2.10. The standard-normal CDF: Φ(2.10) ≈ 0.98214, so the upper tail is 1 − 0.98214 = 0.01786. Two-tailed: p = 2 × 0.01786 ≈ 0.0357. Against α = 0.05: significant. The same z as a one-tailed right test: p = 0.0179 - still significant but exactly half.

This asymmetry is why declaring the tail before seeing the data matters. Switching from two- to one-tailed after observing the direction doubles the apparent significance without changing the evidence. Pre-registration and declaring the alternative hypothesis in advance is the honest approach.

Which Distribution to Choose

z: use when σ is known (rare) or n is large enough for normality (commonly n ≥ 30).

t (df = n−1 or Welch df): the workhorse for comparing means when σ is unknown and estimated from the sample.

χ² (df = k−1 or (r−1)(c−1)): for goodness-of-fit and tests of independence on count data. Always upper-tailed.

F (df₁ = k−1, df₂ = N−k): for ANOVA, regression omnibus tests, and variance comparisons. Always upper-tailed.

Common Mistakes in P-Value Interpretation

The most damaging misuse is p-hacking: running many tests, manipulating analysis choices, or checking results repeatedly until p < 0.05 appears. Each additional test inflates the family-wise error rate. If you run 20 tests at α = 0.05, you expect one false positive even when H₀ is true for every test. Apply Bonferroni, Benjamini-Hochberg, or other multiple-comparison corrections when testing many hypotheses.

Choosing one-tailed after seeing the data direction doubles the apparent significance fraudulently. Reporting only the tests that "worked" and hiding the rest is selective reporting bias. The only valid p-value is one computed from the analysis plan specified before data collection.

P-Value Calculator

Frequently Asked Questions

Related Statistics Calculators