![\"\"](\"http:\/\/www.tc.umn.edu\/~catalst\/images\/cook.png\")
The Catalyst\u00a0Group, University of Minnesota<\/strong>\n Matt Beckman, Ethan Brown, Bob delMas, Elizabeth Brondos Fry, Nicola Justice, Anelise Sabbag<\/strong><\/p>\n
The Catalyst group at the University of Minnesota has developed two introductory statistics courses that employ simulation-based inference methods: an undergraduate course (EPSY 3264) and a graduate course (EPSY 5261).\u00a0[pullquote]Simulation-based inference has different audiences, even in the introductory course[\/pullquote] The courses differ in their use of technology.\u00a0 EPSY 3264, which uses the CATALST<\/a> curriculum, has students dive deep into the modeling process.\u00a0 Students build up the model themselves using TinkerPlots<\/a>TM<\/sup> software, and this setup process is a major part of the course.\u00a0 By contrast, in EPSY 5261, which uses the Lock5 <\/a>textbook, the StatKey<\/a> tool mostly takes care of the modeling process that is used to generate and analyze the empirical sampling distribution generated by the simulation results.<\/p>\n The deep modeling approach used with TinkerPlots means that students have more of an opportunity to engage with the model, but this engagement can be time-consuming.\u00a0 Thus, only a small range of statistical content can be covered in the course.\u00a0 With Lock5 and StatKey, some students may end up not understanding the modeling as thoroughly, but this frees up time to cover a greater breadth of topics by using simulation-based methods as a stepping stone to asymptotic inference strategies.<\/p>\n Deep modeling with TinkerPlots and CATALST<\/strong><\/p>\n In the EPSY 3264 (undergraduate) course, TinkerPlots is used mainly to enhance student learning by helping students visualize and understand how simulations work in inferential analysis (e.g., randomization-based tests).<\/p>\n Students start by using TinkerPlots to create models of basic random phenomena such as flipping coins, rolling dice, or shooting basketball free throws. TinkerPlots provides a robust framework for representing models of random phenomena visually, an affordance that the course exploits throughout the semester. The software both represents a model in intuitive ways and animates the generation of trials from a model (click on image to enlarge):<\/p>\n Initially, students simulate a relatively small number of trials and we ask them to do a lot of interpreting results at many stages of the simulation process. We also try to help students make connections between the results and the research questions motivating each activity.<\/p>\n When we introduce randomization tests, students start with physical manipulatives for conducting the simulations (e.g., slips of paper for randomly assigning groups). Later, students use TinkerPlots to perform large numbers of reassignment trials and use them to answer research questions. Below is the model of a randomization test. The response variable is on the left, the group labels are on the right, and TinkerPlots is randomly assigning the group labels to each given response (click on image to enlarge):<\/p>\n \u00a0 Once the students have generated many trials, they can estimate a p–<\/em>value by noting how many of the simulation results were as or more extreme than the original result from the study (here, the original result was a mean difference in groups of 2.5, resulting in a p-value of 0.12; click on image below to enlarge):<\/p>\n This approach means that students spend a good deal of time setting up and representing the model before they can conduct inference using the distribution of trial results.\u00a0 TinkerPlots does very little work for them: they have to choose the appropriate sampler and set it up for the situation at hand.\u00a0 We feel that the time is worth it, because students have the opportunity to see and interact with the model itself, creating the opportunity to develop a deeper understanding of the \u201cnull model\u201d and its relationship to the distribution of sample statistics.<\/p>\n Leaping from the bootstrap, to randomization, to asymptotic methods with StatKey and Lock5<\/strong><\/p>\n In the EPSY 5621 (graduate) course, simulation-based methods are covered for the first half of the semester, followed by traditional parametric methods. When introducing simulation-based methods, we typically first begin with a tactile simulation and single trials. For example, students write down the colors of M&M candies in a fun-size bag on pieces of paper. They draw the pieces of paper with replacement to create bootstrap samples to estimate the proportion of M&M\u2019s that are blue. They move on to use StatKey<\/a> to generate the bootstrap samples: first they do this one sample at a time, and see how each bootstrap sample proportion is different. Finally, they move on to conduct thousands of trials, which gives them the standard error that they need to compute their bootstrap interval.<\/p>\n When we introduce randomization tests, we again conduct a tactile simulation with an activity called: Do you have ESP?<\/em> Each student guesses the color of a card in a standard deck, 20 times with replacement, to obtain an observed proportion of correct guesses. Students then simulate random guesses by flipping a coin 20 times. Each group repeats this process 5 times and then we collect the randomization proportions from the entire class to build our own randomization distribution. Each student sees where his or her own observed proportion falls in the distribution, and computes a p-value. Using StatKey, students conduct one randomization trial and are asked to describe in detail what is happening in that trial. Then they move on to conduct thousands of trials and use StatKey to calculate the p-value. For example, a student who guessed 12 out of 20 cards correctly (60%), would obtain a one-tailed p-value of about 0.20, as shown below (click on image to enlarge).<\/p>\n![\"CatalstTechnology01\"](\"https:\/\/www.causeweb.org\/sbi\/wp-content\/uploads\/2014\/11\/CatalstTechnology01.gif\")
<\/a><\/p>\n![\"CatalstTechnology02\"](\"https:\/\/www.causeweb.org\/sbi\/wp-content\/uploads\/2014\/11\/CatalstTechnology02.gif\")
<\/a><\/p>\n![\"CatalstTechnology03\"](\"https:\/\/www.causeweb.org\/sbi\/wp-content\/uploads\/2014\/11\/CatalstTechnology03.gif\")
<\/a><\/p>\n
![\"statkeyp-value\"](\"https:\/\/www.causeweb.org\/sbi\/wp-content\/uploads\/2014\/11\/statkeyp-value-300x144.jpg\")
<\/a><\/p>\n![\"statkeymenu\"](\"https:\/\/www.causeweb.org\/sbi\/wp-content\/uploads\/2014\/11\/statkeymenu-300x151.jpg\")
<\/a><\/p>\n