Justin Post (North Carolina State University), Matthew Ferrell (North Carolina State University)
Abstract
Background. Providing authentic assessments that allow students to show what they know can be difficult to do in a written setting. One method that has been shown to provide a rich assessment is an oral exam. These assessments allow for further probing and follow up questions as compared to a paper-based exam. This can allow us to better gauge student understanding. Due to the time intensive nature of these exams, most examples in the undergraduate literature come from small classes where instructors can spend 15-20 minutes with each student.
This poster describes the process of creating oral exams at scale in large (800 student), introductory, statistical programming courses by leveraging graduate student instructors (GSIs). We focus our results on two aspects: the process used to create the exams (including providing recommendations) and the experience of the GSIs in implementing the oral exams under the guidance of a course coordinator.
Methods. We recorded the process for creating and implementing two oral exams during a recent semester. We surveyed our team of six GSIs formally and informally about their thoughts, feelings, and feedback on the process. A qualitative analysis was performed to determine themes relevant to administering the exams from the GSI perspective. We develop recommendations for others attempting to use teaching assistants or similar help to administer oral exams at scale.
Findings. We create many recommendations around the creation of an oral exam that is intended to be administered by a team of instructors. These include how to manage GSI load, garner buy-in from the GSIs, prepare the students and GSIs, and collect feedback about the process. Additionally, we discuss themes around GSI feeling including scheduling, technology, student interactions, self-care, equity of the exam, and reactions to oral exams.
Implications For Teaching and For Research. As we seek authentic assessments of student learning, oral exams have been shown to have high utility. However, they are time intensive and generally not usable for large class sizes. The findings of our research can help others understand a reasonable process for creating and administering a short oral exam to a very large class with the help of graduate student instructors. The research may help others understand potential issues and pitfalls with this approach and, hopefully, plan future research into the utility of oral exams.