By Melissa Crow (New College of Florida)
Information
This classroom activity provides a hands-on introduction to key ideas in machine learning, including prediction, classification, and accuracy. It introduces first-year students at a small liberal arts college to foundational concepts in machine learning and AI ethics without requiring technical prerequisites. Students begin by writing a short passage on a given topic and then generate a comparable passage using an AI writing tool. In class, students analyze the anonymized writing samples, attempting to answer the question: faced with 8 writing samples (four of each type), can students truly distinguish between human- and AI-authored texts?
Along the way, students are prompted to reflect on which stylistic and structural characteristics they believe help indicate authorship. We also explore different methods for summarizing how successful their predictions were, which transitions into a deeper guided discussion of the practical implications of each statistical choice in different scenarios for various stakeholders. Following the activity, we suggest two different approaches to student assessment (a follow-up assignment on image classification and a reflective essay), along with anonymized student comments and guidelines for evaluation. Potential extensions of the activity include both introductory and intermediate applications of statistical inference or modeling techniques, along with parallels to Fisher’s classical experiment.
This activity not only engages students with data they create on a topic that matters to them, but also introduces core machine learning ideas and encourages reflection on implications of AI use/detection. All activity materials and a sample dataset are provided.