Omnibus tests of statistical significance, and in particular null hypothesis tests, have fallen under continuous criticism since Fisher first promulgated the analysis of variance in 1935. Despite the often contentious debate, few practical alternatives have been proposed in the literature. Rosenthal, Rosnow, and Rubin have changed that, with their discussion of contrast analysis and effect sizes.
Plus side: The text gives in depth coverage on the design and conduct of contrast analysis, testing particular theoretical predictions using more general data sets. A good example from the book is the influence of age on performance in a video game: do older children perform better than younger children? With five age groups sampled, a standard ANOVA might inform you that the data are very likely close to what is expected from random sampling. However, the ANOVA treats all 120 possible orders of the age groups as irrelevant to your theory. In fact, there is only one ordering that is important to your theory, but ANOVA can't test it! Contrast analysis can. The authors also offer several novel descriptive statistics, using regression and correlation, that can inform you about relationships in a data set that ordinary omnibus tests can't detect. This approach is a serious improvement over the advice to merely use confidence intervals, or the positively primitive reliance on omnibus null hypothesis tests.
That said, the book has a few defects. First, it assume the reader has a solid grasp on ANOVA, regression, and inferential testing generally. It is thus an advanced text. For an introductory text I recommend Howitt & Cramer's An Introduction to Statistics in Psychology: A Complete Guide for Students.
The text is also quite mathematically oriented, and does not have adequate expositions of all the computations used across the book. If you're not comfortable with inferential statistics and mathematical notations, the book will be a ponderous read. Finally, the book gives very short shrift to the history of omnibus tests and ways they are misinterpreted. On this score, I recommend Gigerenzer's Empire of Chance, and Oakes' Statistical Inference. The latter is now unfortunately out of print.
It is clear that complex theories generate complex predictions, and can be tested using only complex tools applied to large data sets. Omnibus tests like ANOVA are clearly inadequate to the task. Contrast analysis is a novel and potentially rich approach to this problem.
