To determine whether a t statistic or F statistic has less than a .05 (or .01) probability of occurring by chance, the observed (i.e., calculated) statistic is compared to a critical value taken from a probability table. The exact critical value used for any one comparison depends on the level of significance chosen, the number of observations in each group, the number of groups being compared, and whether the researcher has a directional or non-directional hypothesis. Using information about the above factors a researcher obtains a critical value and compares the observed value to it. If the observed value fails to exceed the critical value, the null hypothesis is retained and the results of the study are said to be inconclusive. If the observed value is more extreme than the critical value, the null hypothesis is rejected, the results are said to be statistically significant, and the research hypothesis is said to have received support from the study.

Note that statistical significance is a statement of probability. We can never be certain that what is true of our samples is also true of the populations they represent. This is one of the reasons why all scientific findings are tentative. Moreover, statistical significance does not indicate practical significance. A statistically significant effect may be too small or be produced at too great a cost of time or money to be useful. What if those who practice over-learning must study two extra hours each day to improve their exam performance by a statistically significant, yet relatively small, 3 points? Knowing this, students might choose to spend their time in another way. As the American statesman Henry Clay (1777-1852) noted, in determining the importance of research findings, by themselves "statistics are no substitute for judgment."