“General Knowledge Norms: Updated and Expanded from the 1980 Norms”, 2013-01-24 (; backlinks):
The 1980 general knowledge norms have been valuable to researchers in many fields. However, much has changed over the 32 years since the 1980 norms. For example, in 1980, most people knew the answer to the question “What is the name of the Lone Ranger’s Indian sidekick?” (answer: Tonto), whereas in 2012, few people know this answer.
Thus, we updated the 1980 norms and expanded them by providing new measures. [n = 671, undergraduate psychology students at Kent State University & Colorado State University] In particular, we report two new metacognitive measures (confidence judgments and peer judgments) and provide a detailed report of commission errors. [Table A1]
[[Scott Alexander’s selected questions:]
In a 1999 poll, only 66% of Americans age 18–29 knew that the US won independence from Britain (as opposed to some other country). About 47% of Americans can name all 3 branches of government (executive, legislative, and judicial). 37% know the closest planet to the sun (Mercury). 58% know which gas causes most global warming (carbon dioxide). 44% know Auschwitz was a concentration camp. <50% (ie. worse than chance) can correctly answer a true-false question about whether electrons are bigger than atoms.
These results are scattered across many polls, which makes them vulnerable to publication bias; I can’t find a good unified general knowledge survey of the whole population. But there’s a great survey of university students [ et al 2013]. Keeping in mind that this is a highly selected, extra-smart population, here are some data points:
85% know who wrote Romeo and Juliet (Shakespeare)
What is the name of the rubber object hit by hockey players? (Puck, 89%)
What is the name of the comic strip character who eats spinach to increase his strength? (Popeye, 82% correct)
What is the name of Dorothy’s dog in The Wizard of Oz? (Toto, 80% correct)
56% know the biggest planet (Jupiter)
44% know who rode on horseback in 1775 to warn that the British were coming (Paul Revere)
33% know what organ produces insulin (pancreas)
31% know the capital of Russia (Moscow)
30% know who discovered the Theory of Relativity (Einstein)
19% know what mountain range contains Mount Everest (Himalayas)
19% know who wrote 1984 (George Orwell)
16% know what word the raven says in Poe’s “The Raven” (“Nevermore!”)
10% know the captain’s name in Moby-Dick (Ahab)
7% know who discovered, in 1543, that the Earth orbits the sun (Copernicus)
4% know what Chinese religion was founded by Lao Tse (Taoism)
- <1% know what city the general Hannibal was from (Carthage)
Remember, these are university students, so the average person’s performance is worse.]
Each of these measures will be valuable to researchers, and together they are likely to facilitate future research in a number of fields, such as research investigating memory illusions, metamemory processes, and error correction. The presence of substantial generational shifts 1980–32201212ya necessitates the use of updated norms…We also identified several questions from the 1980 norms that were either outdated or incorrect. For instance, consider the question “What is the name of the company that produces Baby Ruth candy bars?” In 1980, the correct response was Curtiss, but now Nestle produces this candy bar. As an example of incorrect information, consider the question “What is the last name of the first American author to win the Nobel Prize for literature?” In the 1980 norms, the correct answer was listed as “Henry”, but the correct answer is “Lewis”.
In addition to updating the 1980 norms and correcting such errors, we also wished to provide 3 new measures: confidence judgments, peer judgments, and commission errors. We will discuss each in turn. After answering each question, participants made a confidence judgment about the likelihood (0%–100%) that the response was correct. Confidence judgments provide an index of how certain people are that their knowledge is correct. To foreshadow, in some cases, people’s confidence was in line with their knowledge: that is, high confidence for correct responses and low confidence for incorrect responses. However, for other questions, people’s confidence was a poor indicator of knowledge. For decades, research has investigated illusions of memory such as confidently-held false memories. Such research has commonly investigated false memories that have been created from word lists (eg. Anastasi et al 200024ya; Gallo2010; Roediger & McDermott1995), produced by misinformation (eg. Frenda et al 201113ya; 1989), or implanted into one’s past (eg. 1997). The expanded measures reported here provide researchers with information that will facilitate research on long-term false memories for general knowledge information by identifying questions with low probabilities of recall and high levels of confidence in errors.
We also created a new metacognitive measure, peer judgments, by having people predict how many peers would correctly answer each question. This measure provides normative information about people’s sense of how their own knowledge compares with that of others, which will be useful within a number of research areas. For instance, researchers have increasingly focused on metacognitive judgments made for oneself, in comparison with judgments for another person (eg. 1996; 2010; 1999). Such research has focused on the influence of one’s subjective experience on metacognitive processes, and on the relationship between predictions made for oneself as compared with predictions made for someone else. The peer judgments reported here provide an index of the latter relationship. To preview, for some questions the confidence and peer judgments were strongly associated (eg. Pearson’s r > 0.80), whereas for other questions the measures were weakly associated (eg. Pearson’s r < 0.30). Thus, researchers can identify subsets of questions for which confidence and peer judgments are highly correlated, and other subsets for which the two measures are uncorrelated or weakly correlated.
Perhaps most importantly, we have provided a detailed assessment of the kinds of errors that people made when answering each question. In particular, we focused on commission errors by providing the most commonly reported incorrect responses and confidence in these errors. Information about commissions and normative confidence in these errors may support inquiry in numerous areas, such as in research investigating false memories and error correction. Concerning the latter area, researchers are currently investigating how people correct errors (eg. Butler et al 201113ya; 2001). To do so, the researchers use items for which people generate responses that are incorrect but are held in high confidence, as compared with errors that are held in low confidence. The commission errors reported here provide normative values for this kind of information by identifying questions for which commission errors are both frequent and associated with differing levels of confidence.
…To assess generational stability, a coarse-grained measure was the rank-order correlation between the new norms and the original ones. The Spearman correlation (ρ) was 0.83 (p < 0.001), which suggests stability 1980–32201212ya. Even so, this value may overestimate stability, and it does not indicate that the new norms simply duplicated the original ones. For instance, consider that in 1980 the most challenging quartile of questions (the items ranked 225–300) contained only 4 questions that were impossible for participants to answer (ie. the probability of correct recall was zero). By contrast, in 2012, the majority of the items (68%, or 51 questions) in this last quartile were impossible. Thus, if researchers today wanted to select relatively difficult questions and used the original norms, they might instead be selecting questions with which many—if not all—of their college participants had no experience.