{"id":154,"date":"2014-10-10T10:32:37","date_gmt":"2014-10-10T15:32:37","guid":{"rendered":"http:\/\/new.causeweb.org\/blog\/randomization\/?p=154"},"modified":"2014-11-07T11:37:55","modified_gmt":"2014-11-07T16:37:55","slug":"why-robin-lock","status":"publish","type":"post","link":"https:\/\/www.causeweb.org\/sbi\/?p=154","title":{"rendered":"How did I get started on teaching simulation-based inference?"},"content":{"rendered":"

Robin Lock, St Lawrence University<\/strong><\/p>\n

\"\"<\/p>\n

Around 1998, Allan Rossman and Beth Chance asked me to help out with a new edition of their popular Workshop Statistics<\/a> book that would be adapted to use a new software package called Fathom<\/a> that was being developed by Bill Finzer, then at KCP Technologies. [pullquote]But I could detect light bulbs going on with students thinking, \u201cOh, that\u2019s what he means by seeing what would happen if the null hypothesis is true!'[\/pullquote]\u00a0Fathom has a lot of neat tools designed to allow students to explore statistical concepts, including a facility to allow students to easily select a sample from a dataset, define any statistic for that sample, and then quickly generate a new dataset with values of that statistic for many new samples.<\/p>\n

Although I had primarily used these features in Fathom to do fairly traditional demonstrations of sampling distributions, I added a new example (near the end of the semester in 2004) where students found a p-value using a randomization distribution to test a hypothesis about a correlation between baseball ballpark capacity and attendance.\u00a0 The scramble feature in Fathom (permuting the values of one variable) made it easy to generate samples that would obey a null hypothesis of no association and collecting\u00a0the correlation coefficients for many samples was a quick way\u00a0to generate and display the randomization distribution.\u00a0 Just count the number of scrambles\u00a0with correlation coefficients more extreme than the original data and we had a p-value.\u00a0 This came at a point late in the semester when students had already had a lot of experience with formula\/traditional, distribution-based tests, including a t<\/em>-test for the correlation.\u00a0 But I could detect light bulbs going on with students thinking, \u201cOh, that\u2019s what he means by seeing what would happen if the null hypothesis is true!\u201d<\/p>\n

It took several semesters of doing this activity before it dawned on me that perhaps it would be better to give students this strong intuition about a p-value at the start<\/em> of teaching about hypothesis tests, rather than at the end!\u00a0 This process was helped along by George Cobb’s famous USCOTS 2005 banquet talk<\/a> and later 2007 ISE paper<\/a>, where (as only George can do) he managed to compare introducing inference via formula-based traditional distributions (rather than simulation-based methods) to continuing to believe that the sun revolves around the earth. So, as a start on hypotheses testing, I added an early Fathom activity to test a claim about the mean weight of pumpkins in a farmer\u2019s field by taking a sample, shifting it to match the null mean, and then selecting lots of random samples (with replacement) to form a sampling\u00a0distribution.<\/p>\n

Having dipped my toe in the water and found it to be pretty warm, I took the plunge in 2010 to make my course more Cobb-compliant (and less Ptolemaic) by using simulation-based methods (bootstrap confidence intervals and randomization tests) for the entire introduction to inference, before moving on to the more traditional distribution-based formulas. I was helped immensely in this endeavor by collaborations with my wife, Patti (who has been teaching intro statistics at St. Lawrence for many years), and our three statistician children, Kari, Eric, and Dennis, as we embarked on a project to develop materials to support this approach. \u00a0We got lots of positive and constructive feedback from the students who patiently worked from a couple of \u201cjust in time\u201d draft text chapters and new Fathom activities.<\/p>\n

While being happy with how things went and finding that many parts of the course still worked fine without revision, we recognized that many of the simulation-based class activities and exercises were very dependent on the technology features found in Fathom. \u00a0It didn\u2019t seem feasible to expect other instructors to abandon their existing statistics technology, need to deal with a lot of programming, or incur additional expense by adding on Fathom in order to use these methods.\u00a0 This prompted us to develop StatKey<\/a> as a set of freely available web apps that we designed specifically to facilitate teaching inference from a simulation-based perspective with easy, student-friendly tools.<\/p>\n

[pullquote]Using simulation-based methods allows students to see a wider variety of inference applications earlier in the course.\u00a0 [\/pullquote]<\/p>\n

With several years of experience now, I find that much of my course is not hugely different than before making this switch.\u00a0 Initial material on issues of data collection\/production and summarizing data numerically and graphically is basically the same.\u00a0 Using simulation-based methods allows students to see a wider variety of inference applications earlier in the course.\u00a0 For example, last Friday (October 3rd<\/sup>, Day 17 in a semester with 42 hour-long class meetings), my students worked through examples to test for a difference in proportions “Does lithium work better than a placebo in treating cocaine addiction?,” a difference in means “Are beer drinkers more attractive to mosquitoes than water drinkers?,” and a correlation “Do football teams with more malevolent uniforms tend to get more penalty yards?”\u00a0 At this point in the semester they have done confidence intervals and hypothesis tests for most of the common parameters (mean, proportion, difference in means, difference in proportions, correlation, slope, and even a CI for a standard deviation) and won\u2019t see a normal distribution until next week.<\/p>\n

I still cover the traditional methods, but find they go much more quickly now that students aren\u2019t trying to learn the basic ideas about confidence intervals and tests at the same time they are grappling with formulas.\u00a0 Now they can focus just on \u201cWhat\u2019s the formula for the standard error in this situation?\u201d and \u201cWhat conditions are needed for the bootstrap\/randomization distribution to be approximated by the theoretical reference distribution?\u201d \u00a0Thus students finish the course seeing all the topics they saw in the \u201cold\u201d days, with some additional skills from bootstrapping\/randomization and, most importantly, a better general understanding of the core statistical ideas.<\/p>\n

Overall, as often happens with curricular innovation, incorporating simulation-based methods in my courses has been an evolutionary process \u2013 keeping changes that worked well and revising some that didn\u2019t.\u00a0 I was fortunate to have contact with people like Allan, Beth, and George (as well as my family!) who had lots of good ideas along the way.\u00a0 I would hope that the process is easier now for faculty who want to move in this direction, as we have more experience from different groups developing materials to support this approach.<\/p>\n","protected":false},"excerpt":{"rendered":"

Robin Lock, St Lawrence University Around 1998, Allan Rossman and Beth Chance asked me to help out with a new edition of their popular Workshop Statistics book that would be adapted to use a new software package called Fathom that was being developed by Bill Finzer, then at KCP Technologies. [pullquote]But I could detect light […]<\/p>\n","protected":false},"author":12,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[3],"tags":[],"class_list":["post-154","post","type-post","status-publish","format-standard","hentry","category-why"],"_links":{"self":[{"href":"https:\/\/www.causeweb.org\/sbi\/index.php?rest_route=\/wp\/v2\/posts\/154","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.causeweb.org\/sbi\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.causeweb.org\/sbi\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.causeweb.org\/sbi\/index.php?rest_route=\/wp\/v2\/users\/12"}],"replies":[{"embeddable":true,"href":"https:\/\/www.causeweb.org\/sbi\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=154"}],"version-history":[{"count":7,"href":"https:\/\/www.causeweb.org\/sbi\/index.php?rest_route=\/wp\/v2\/posts\/154\/revisions"}],"predecessor-version":[{"id":376,"href":"https:\/\/www.causeweb.org\/sbi\/index.php?rest_route=\/wp\/v2\/posts\/154\/revisions\/376"}],"wp:attachment":[{"href":"https:\/\/www.causeweb.org\/sbi\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=154"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.causeweb.org\/sbi\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=154"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.causeweb.org\/sbi\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=154"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}