"Using Randomization Tests and IPUMS-USA* to Investigate the Gender Wage Gap"
with Laura M. Schultz, Rowan University
Hosted by: Sam Morris, North Carolina State University
Historically, women have often earned less than men for performing the same job, a phenomenon known as the gender wage gap. Given that salary data are notoriously skewed, investigating the gender wage gap provides an engaging context for introducing students to modern randomization tests as an alternative to more traditional nonparametric tests. The first randomization test included in this classroom activity addresses whether the mean salary of male accountants and auditors is significantly greater than that of females with similar qualifications, and the second task investigates whether the ratio of the median salary of female accountants/auditors to the median salary of their male counterparts is significantly less than 1. (A ratio of 1 would indicate that the median female and male salaries are the same.) Both randomization tests utilize custom JMP scripts that I have written. The data set, a subset of a sample collected as part of the 2008 American Communities Survey that I downloaded from IPUMS-USA, consists of the total personal earnings for samples of full-time female and male accountants/auditors ages 25-34 with Bachelor's degrees in the Philadelphia metropolitan area. I chose this particular data set due to its appeal to the accounting and finance majors I teach at a university in the greater Philadelphia region. Given the flexibility of the IPUMS-USA interface, this classroom activity could be adapted to investigate the gender wage gap using samples from other professions, age groups, or geographical regions.
*S. Ruggles, J.T. Alexander, K. Genadek, R. Goeken, M.B. Schroeder, and M. Sobek. Integrated Public Use Microdata Series: Version 5.0 [Machine-readable database]. Minneapolis: University of Minnesota, 2010.
Hello, I think this is all marvellous. Could I just ask two silly questions. First, how much do you think the JMP scripts hide the concepts from learners? Secondly, how do deal with all the confounding that lies behind this gender gap (or are you just making the point that there is a gender gap, and figuring why is the next thing to do)?
The students are shown the JMP script code, which includes detailed comments explaining what each line does. Also, I spend quite a lot of class time explaining the mechanics and theory behind randomization tests, so I have not found that providing pre-written scripts hinders student learning. My students are primarily business majors, so I decided that it wasn't realistic to expect them to write their own code. As for your other question, part of the beauty of this data set is that it stimulates student discussion on how to resolve seemingly contradictory results. One of the lessons that I try to convey to my students is that there isn''t always a "right answer" when you conduct a statistical analysis.
Thanks. I've done some very non-computational work on randomisation tests; but never figured the best next step (using the method for real). A few years ago I seemed to freak people out when asking them to use scripts (which I never understood - what do you think happens when you click a button). At the moment I'm using Excel macros but they are very clumsy. Maybe I should just pluck up the courage to ask them to copy/paste scripts. Thanks for the reply - twice ;-)
I expect the better students to be uncomfortable with the fact that two equally valid tests gave quite different p-values. How do you reconcile them? This illustrates why many statisticians prefer to emphasize confidence intervals over tests of point null hypothesis. I suppose to do CI's you would need to bootstrap, which is a more complex topic. Have you also done CI's with these data?
Indeed, this conflict between the two test outcomes is one of the key reasons why I like using this particular data set to introduce students to randomization tests. Early on, students are told to focus on the median instead of the mean when describing the center of a skewed distribution, yet traditional t-tests dwell on the mean. An advantage of randomization tests is that you can define any test statistic you choose; hence, we could actually look at the ratio of female to male median salaries. The gender-wage gap is traditionally defined in terms of this ratio, yet the two-sample t-test forces use to analyze the difference between two means instead. It provides a real "teaching moment" to show students how both a traditional two-sample t-test (P = .0179, right-tailed) and a randomization test based on the difference between two means seem to imply the existence of a gender wage gap, yet the analysis of the ratio of median salaries (arguably the more appropriate test, given the traditional definition of the gender wage gap), instead supports the idea that, at least for a narrowly defined demographic of males and females of similar ages working full-time in the same metropolitan area with the same amount of education and employed in the same occupation, there actually isn't evidence of a significant gender wage gap. I have not tried producing bootstrap confidence intervals yet, but that is something I plan to do in the future as I write up this work for publication.