Hypothesis testing is the use of statistics to determine the probability that a given hypothesis is true. The usual process of hypothesis testing consists of four steps.

1. Formulate the null hypothesis  (commonly, that the observations are the result of pure chance) and the alternative hypothesis  (commonly, that the observations show a real effect combined with a component of chance variation).

2. Identify a test statistic that can be used to assess the truth of the null hypothesis.

3. Compute the P-value, which is the probability that a test statistic at least as significant as the one observed would be obtained assuming that the null hypothesis were true. The smaller the -value, the stronger the evidence against the null hypothesis.

4. Compare the -value to an acceptable significance value  (sometimes called an alpha value). If , that the observed effect is statistically significant, the null hypothesis is ruled out, and the alternative hypothesis is valid.

REFERENCES:

Gonick, L. and Smith, W. The Cartoon Guide to Statistics. New York: Harper Perennial, pp. 140-142, 1993.

Good, P. Permutation Tests: A Practical Guide to Resampling Methods for Testing Hypotheses, 2nd ed. New York: Springer-Verlag, 2000.

Hoel, P. G.; Port, S. C.; and Stone, C. J. "Testing Hypotheses." Ch. 3 in Introduction to Statistical Theory. New York: Houghton Mifflin, pp. 52-110, 1971.

Iyanaga, S. and Kawada, Y. (Eds.). "Statistical Estimation and Statistical Hypothesis Testing." Appendix A, Table 23 in Encyclopedic Dictionary of Mathematics. Cambridge, MA: MIT Press, pp. 1486-1489, 1980.

Shaffer, J. P. "Multiple Hypothesis Testing." Ann. Rev. Psych. 46, 561-584, 1995.