[blog; cf. testing effect, calibration training, “My Favorite Liar”, “Fake Journal Club”] We adapt the social game “Two truths and a lie” to a classroom setting to give an activity that introduces principles of statistical measurement, uncertainty, prediction, and calibration, while giving students an opportunity to meet each other.
We discuss how this activity can be used in a range of different statistics courses.
[Keywords: calibration, education, generalized linear models]
…This activity can be performed during the first week of class or later on during the semester if that seems to better fit with the sequence of topics in the course.
We start the activity by dividing students into groups of 4—it’s fine if some groups have 3 or 5 students in them—to play “two truths and a lie.” We display the instructions in Figure 1 onto the screen and explain the procedure. In this game, one person makes 3 statements about him or herself; two of these statements should be true and one should be false. The other students in the group should then briefly confer and together guess which statement is the lie. They should jointly construct a numerical statement of their certainty about their guess, on a 0–10 scale, where 0 represents pure guessing and 10 corresponds to complete certainty. The true statement is then revealed so that the students know if they guessed correctly. Each group of students then rotates through, with each student playing the role of storyteller, so that when the activity is over, each group of 4 students has produced 4 certainty numbers, each corresponding to a success or failure.
…Before making the plot and displaying the data and fit, we ask students in their groups to sketch what they think the scatterplot and fitted curve for the class will look like, and then we lead the class in discussion. Some possible prompts include: “What do you think the range of certainty scores will look like: Will there be any 0’s or 10’s?” “Will there be a positive relation between x and y: are guesses with higher certainty be more accurate, on average?” “How strong will the relation be between x and y: what will the curve look like?” If students have seen logistic regression, we ask them to give approximate numerical values for the intercept and slope coefficients corresponding to their sketched curves.
After this discussion, we display the data and fitted curve and conduct a follow-up discussion of what has been learned. Figure 3 shows an example of real data from an applied regression class. In this case, there is essentially no relation between the certainty score and the outcome (coded as 1 for a successful guess and 0 for an error). In fact, the estimated logistic regression coefficient is negative: higher certainty scores correspond to slightly lower rates of accuracy!
…For an introductory course, the focus can be on probability and uncertainty. Before the activity begins, ask students to speculate on how accurate their guesses will be? On average, will they be able to guess the lie every time? 90% of the time? 50%? More than 33%, we hope, right?
…For a course on Bayesian statistics, the activity can be used to demonstrate the principle of calibration…For a class on generalized linear models or machine learning, you can use this as an introduction to logistic regression, showing the details of fitting and graphing the model, interpreting the coefficient estimates and standard errors, and using the prediction from the model to make probabilistic forecasts for new cases…Psychometrics and multilevel modeling: Another direction is to turn this into a lesson on reliability and validity of measurement. What is meant by that certainty score? How useful would we expect the certainty score to be in making a probabilistic forecast? This sort of calibration problem arises in many areas of science and policy…this discussion of measurement can serve as an entry point to the design and analysis of repeated measures data.
…We can also consider what lessons students might take away from this activity. “Statistics is fun”: that’s a good memory. “I got fooled by Jason’s lie: he’s not really adopted”: that’s fine too, as it serves the goal of students getting to know each other. “You can use logistic regression to convert a certainty score into a predicted probability”: that’s good because it’s a vivification of a general mathematical lesson. “The estimated slope was smaller than the standard error so we couldn’t distinguish it from zero”: that’s not a bad lesson either.
Think about what memories you want to create, and keep the discussion focused. For example, the details of the truths and lies are fun, and there could be a temptation to share some of the most successful lies with the class—but for a class on statistics or research methods, those sorts of details could be counterproductive, eliciting memories that would distract from the statistical lessons. We want the activity to be vivid and memorable but for the right reasons.
In our experience we have seen 3 sorts of positive outcomes associated with this sort of activity, especially when performed near the beginning of the semester. (1) The first is that students get used to the idea that attendance is active, not passive, and we hope the alertness required to perform these activities translates into better participation throughout the class period. (2) The second is that people typically find data more interesting and relatable when they can see themselves in the scatterplot. (3) The third valuable outcome is that the “Two truths and a lie” activity is a social icebreaker.
That said, we do not have direct empirical evidence of the effectiveness of this activity on student learning. It is our hope that in laying out this activity—not just the general concepts but also the details of implementation, including instructions, Google Forms form, sample data and analysis, and post-analysis discussion points—we have lowered the barrier of difficulty so that instructors in a wide range of statistics courses can try it out in their own classes, at minimal cost in classroom time and with the potential to get students more involved in their learning of statistics.