Mia Petrie (University of Illinois), Madeline Hunt (University of Illinois), V.N. Vimal Rao (University of Illinois), Kelly Findley (University of Illinois)
Abstract
Background. One goal enumerated in the 2016 GAISE college report is “Students should become critical consumers of statistically-based results reported in popular media” and they should be able to “interpret what graphs do and do not reveal”. We interpret “critical” to not only invoke notions of critical thinking, but also critical citizenry and literacy – as Weiland (2017) put it, a student’s ability to “interrogate the structures at play within their community and government” that interact with statistical information.
Such interpretations require complex judgments related to the subjective nature of data visualization and data storytelling – students must consider how the data was collected, what choices were made in creating the visualization, the potential bias of the author as a source, the larger community and political environment in which the visualization might be perceived, and more.
We believe that in this era of hyper-political polarization, critical statistical literacy is an immensely important learning objective. To support future assessment and instruction in critical statistical literacy, we examine the ways in which students interpret critical contexts in data visualizations, with particular focus on the ways they interrogate bias from the subjective choices inherent to data visualizations.
Methods. We recruited 8 students from an introductory statistics course. Course instruction included developing students’ critical statistical literacy habits of mind, such as questioning the sampling strategy, and also exposed students to multivariable data visualizations and models.
The students completed task-based semi-structured interviews in which they were asked to think aloud as they interpreted ten different graphs obtained from popular news or social media outlets.
Here we focus on a thematic analysis of students’ thoughts on whether the graph was being presented with any kind of bias, or whether they thought it was being presented fairly.
Findings. The students’ perception of bias primarily depended on the topic of the data visualization and rarely stemmed from the subjective choices related to the analysis or graphic design. For example, data visualizations about police shootings and vaccines were more likely to be perceived as biased, whereas graphs about snowfall data and generational voting trends were more likely to be perceived as fair and objective. When asked what might make a graph biased, students suggested adding more controversial variables like race, or fabricating the data points, but many did not consider how visualization features were a result of creator judgment.
Implications For Teaching and For Research. Our work contributes to developing theories and examples for what students consider when evaluating biases in authentic data visualizations. Students were understandably cautious about graphs that addressed controversial topics, but several struggled to consider how they might evaluate the biases of such graphs. While caution is arguably better than tacit trust, it does lead us to question what specific instruction students need to evaluate the more subtle subjective biases in the design of complex data visualizations. Without the data literacy needed to make these evaluations, our students will not be prepared to engage with complex and critical topics with data.
Additional Information. This work has been funded by a University of Illinois Provost's Initiative on Teaching Advancement grant.