By Gordon Brooks (Ohio University)
Information
Because most statistics programs provide only pairwise comparisons, relatively few applied researchers use the Scheffé multiple comparison procedure (MCP) because it has lower power than pairwise MCPs (e.g., Tukey, R-E-G-W) for pairwise-only comparisons following a significant ANOVA. However, only Scheffé guarantees congruence with omnibus ANOVA. That is, a maximum Scheffé comparison can be calculated that maximally differentiates some combination of groups and has perfectly congruent statistical significance with omnibus ANOVA (i.e., significant when ANOVA is, not significant when ANOVA is not). Unfortunately, coefficients from this typically non-pairwise or complex maximum Scheffé comparison are often difficult to interpret. We present a method by which data analysts can calculate maximum "human-friendly" comparisons that approximate the Scheffé maximum comparison---but with reasonably interpretable coefficients. Research has shown that the maximum human-friendly contrast has a congruence rate over 96% for most conditions (Brooks, Adjanin, Oppong, & Liu, 2024). An R Shiny app was created that performs the analyses (https://tinyurl.com/35bkmk5u). This method has been used in College of Education graduate courses with 25-35 students, but it can be used with any ANOVA course. The presenters will share their experiences successfully using the method and the Shiny app with students.
R Shiny app:
https://tinyurl.com/35bkmk5u