We begin with a graphical approach to bootstrapping and permutation testing, illuminating basic statistical concepts of standard errors, confidence intervals, p-values and significance tests. We show graphical and numerical diagnostic checks for the validity of traditional Gaussian-based inferences.

We consider a variety of statistics (mean, trimmed mean, regression, etc.), and a number of sampling situations (one-sample, two-sample, stratified, finite-population), stressing the common techniques that apply in these situations.

Introduction to Bootstrapping

General procedure

Why does bootstrapping work?

Sampling distribution and bootstrap distribution

Bootstrap Distributions and Standard Errors

Distribution of the sample mean

Bootstrap distributions of other statistics

Simple confidence intervals

Two-sample applications

How Accurate Is a Bootstrap Distribution?

Example where things go wrong

Bootstrap Confidence Intervals

Bootstrap percentiles as a check for standard intervals

More accurate bootstrap confidence intervals

Significance Testing Using Permutation Tests

Two-sample applications

Other settings

Course sessions will be a combination of PowerPoint-style presentation, live demonstrations, and hands-on work using statistical software.

The demonstrations will use the free student version of S-PLUS with a resampling library that provides an easy-to-use graphical interface for BPT. However, the focus of this course is on the concepts to be taught, not the software. We use the graphical interface so that participants with no experience with S-PLUS will have no trouble following. The teaching ideas can also be implemented with other software.

Course materials

Course participants will receive handouts and a copy of Bootstrap Methods and Permutation Tests, Hesterberg, et al., W. H. Freeman, 2003, a supplemental chapter for using BPT for teaching introductory statistics.

Learning outcomes

Participants will learn how to use resampling methods:

• to compute standard errors
• to check the accuracy of the usual Gaussian-based methods,
• to compute both quick and more accurate confidence intervals,
• for a variety of statistics and
• for a variety of sampling methods, and
• to perform significance tests in some settings.

Participants will also gain an appreciation for the benefits of these methods in teaching statistical concepts.