Multiple comparison procedures: consistency and family-size robustness
by
Dave Saville
Statistics Group, AgResearch, Lincoln, New Zealand

A range of multiple comparison procedures (MCP) are now available in Genstat, including Tukey, Scheffe, Bonferroni, Student-Newman-Keuls, Duncan and Fisher's protected and unprotected LSD tests. In this paper I illustrate the differences between these procedures by using each MCP to analyse an experiment with 32 treatments, firstly including all treatments in the analysis, then restricting the analysis to subgroups of 13, 4 and 2 treatments that are of interest to different clients. The differing results demonstrate the phenomenon of inconsistency (Saville 1990), or the very similar concept of familywise robustness (Holland and Cheung 2002), perhaps more aptly entitled family-size robustness. The procedure which performs best under both criteria is the unprotected LSD. I therefore advocate usage of this simple procedure in the minority of experiments when an MCP is appropriate, with the proviso that it be treated as an "hypothesis generator, " not a procedure for simultaneous formulation and testing of hypotheses.

References

Hochberg, Burt and Cheung, Siu Hung (2002). Familywise robustness criteria for multiple-comparion procedures. Journal of Royal Statistical Society B, 64(1), 63-77.

Saville, D.J. (1990). Multiple comparison procedures: The practical solution. The American Statistician 44(2), 174-180.

Date received: September 4, 2002

Copyright © 2002 by the author(s). The author(s) of this document and the organizers of the conference have granted their consent to include this abstract in Atlas Conferences Inc. Document # cajn-27.