- See Also
-
Links
- “Evaluating Superhuman Models With Consistency Checks”, Fluri et al 2023
- “Incentivizing Honest Performative Predictions With Proper Scoring Rules”, Oesterheld et al 2023
- “Conditioning Predictive Models: Risks and Strategies”, Hubinger et al 2023
- “The Unlikelihood Effect: When Knowing More Creates the Perception of Less”, Karmarkar & Kupor 2022
- “Does Constructing a Belief Distribution Truly Reduce Overconfidence?”, Hu & Simmons 2022
- “Reconciling Individual Probability Forecasts”, Roth et al 2022
- “An Appropriate Verbal Probability Lexicon for Communicating Surgical Risks Is unlikely to Exist”, Harris et al 2022
- “A Simple Cognitive Method to Improve the Prediction of Matters of Taste by Exploiting the Within-person Wisdom-of-crowd Effect”, Fujisaki et al 2022
- “Language Models (Mostly) Know What They Know”, Kadavath et al 2022
- “Forecasting Future World Events With Neural Networks”, Zou et al 2022
- “Modeling Transformative AI Risks (MTAIR) Project—Summary Report”, Clarke et al 2022
- “Taking a Disagreeing Perspective Improves the Accuracy of People’s Quantitative Estimates”, Calseyde & Efendić 2022
- “Teaching Models to Express Their Uncertainty in Words”, Lin et al 2022
- ““Two Truths and a Lie” As a Class-participation Activity”, Gelman 2022
- “DeepMind: The Podcast—Excerpts on AGI”, Kiely 2022
- “Many Heads Are More Utilitarian Than One”, Keshmirian et al 2022
- “Uncalibrated Models Can Improve Human-AI Collaboration”, Vodrahalli et al 2022
- “A 680,000-person Megastudy of Nudges to Encourage Vaccination in Pharmacies”, Milkman et al 2022
- “TACTiS: Transformer-Attentional Copulas for Time Series”, Drouin et al 2022
- “Dream Interpretation from a Cognitive and Cultural Evolutionary Perspective: The Case of Oneiromancy in Traditional China”, Hong 2022
- “M5 Accuracy Competition: Results, Findings, and Conclusions”, Makridakis et al 2022
- “Megastudies Improve the Impact of Applied Behavioral Science”, Milkman et al 2021
- “Forecasting Skills in Experimental Markets: Illusion or Reality?”, Corgnet et al 2021
- “Strategically Overconfident (to a Fault): How Self-promotion Motivates Advisor Confidence”, Zant 2021
- “Sigmoids Behaving Badly: Why They Usually Cannot Predict the Future as well as They Seem to Promise”, Sandberg et al 2021
- “Wise Teamwork: Collective Confidence Calibration Predicts the Effectiveness of Group Discussion”, Silver 2021
- “Alignment Problems With Current Forecasting Platforms”, Sempere & Lawsen 2021
- “Behavioral Scientists and Laypeople Misestimate Societal Effects of COVID-19”, Hutcherson et al 2021
- “How the Wisdom of Crowds, and of the Crowd Within, Are Affected by Expertise”, Fiechter & Kornell 2021
- “Mind the Gap: Assessing Temporal Generalization in Neural Language Models § Scaling”, Lazaridou et al 2021
- “Kaggle Forecasting Competitions: An Overlooked Learning Opportunity”, Bojer & Meldgaard 2020
- “Danny Hernandez on Forecasting and the Drivers of AI Progress”, Koehler et al 2020
- “AI and Efficiency: We’re Releasing an Analysis Showing That Since 2012 the Amount of Compute Needed to Train a Neural Net to the Same Performance on ImageNet Classification Has Been Decreasing by a Factor of 2 Every 16 Months”, Hernandez & Brown 2020
- “ForecastQA: A Question Answering Challenge for Event Forecasting With Temporal Text Data”, Jin et al 2020
- “LightGBM: A Highly Efficient Gradient Boosting Decision Tree”, Ke et al 2019
- “Predicting History”, Risi et al 2019
- “N-BEATS: Neural Basis Expansion Analysis for Interpretable Time Series Forecasting”, Oreshkin et al 2019
- “Evidence on Good Forecasting Practices from the Good Judgment Project”, Impacts 2019
- “Forecasting Transformative AI: An Expert Survey”, Gruetzemacher et al 2019
- “The Wisdom of Crowds Approach to Influenza-Rate Forecasting”, Morgan et al 2018
- “Predicting Replication Outcomes in the Many Labs 2 Study”, Forsell et al 2018
- “Augur: a Decentralized Oracle and Prediction Market Platform”, Peterson & al 2018
- “The Wisdom of the Inner Crowd in Three Large Natural Experiments”, Dolder & Assem 2017
- “When Will AI Exceed Human Performance? Evidence from AI Experts”, Grace et al 2017
- “DeepAR: Probabilistic Forecasting With Autoregressive Recurrent Networks”, Salinas et al 2017
- “Long Bets As Charitable Giving Opportunity”, Gwern 2017
- “Embryo Selection For Intelligence”, Gwern 2016
- “Looking Across and Looking Beyond the Knowledge Frontier: Intellectual Distance, Novelty, and Resource Allocation in Science”, Boudreau et al 2016
- “Technology Forecasting: The Garden of Forking Paths”, Gwern 2014
- “Complexity No Bar to AI”, Gwern 2014
- “Darknet Market Mortality Risks”, Gwern 2013
- “Mechanical Versus Clinical Data Combination in Selection and Admissions Decisions: A Meta-Analysis”, Kuncel et al 2013
- “Predicting Google Closures”, Gwern 2013
- “Credit Suisse Global Investment Returns Yearbook 2013”, Dimson et al 2013
- “Statistical Basis for Predicting Technological Progress”, Nagy et al 2012
- “‘HP: Methods of Rationality’ Review Statistics”, Gwern 2012
- “LSD Microdosing RCT”, Gwern 2012
- “‘Methods of Rationality’ Predictions”, Gwern 2012
- “Learning Performance of Prediction Markets With Kelly Bettors”, Beygelzimer et al 2012
- “NGE Rebuild Predictions”, Gwern 2011
- “A Prediction Market for Macro-Economic Variables”, Teschner et al 2011
- “Why Do Humans Reason? Arguments for an Argumentative Theory”, Mercier & Sperber 2011
- “Zeo Sleep Self-experiments”, Gwern 2010
- “The Ones Who Walk Towards Acre”, Gwern 2010
- “Goodbye 2010”, Legg 2010
- “About This Website”, Gwern 2010
- “Applying the Fermi Estimation Technique to Business Problems”, Anderson & Sherman 2010
- “Nootropics”, Gwern 2010
- “Predicting the Next Big Thing: Success As a Signal of Poor Judgment”, Denrell & Fang 2010
- “Summers of Code, 2006–2013”, Gwern 2009
- “Keep Your Identity Small”, Graham 2009
- “Wikipedia & Knol: Why Knol Already Failed”, Gwern 2009
- “In Defense of Inclusionism”, Gwern 2009
- “Choosing Software”, Gwern 2008
- “Measuring the Crowd Within: Probabilistic Representations Within Individuals”, Vul & Pashler 2008
- “The Meta-Analysis of Clinical Judgment Project: 56 Years of Accumulated Research on Clinical Versus Statistical Prediction”, Ægisdóttir et al 2006
- “A Systematic Review on Communicating With Patients about Evidence”, Trevena et al 2005
- “Principles of Forecasting: A Handbook for Researchers and Practitioners”, Armstrong 2001
- “Who Is Arguing About the Cat? Moral Action and Enlightenment According to Dōgen”, Mikkelson 1997
- “Eliminating the Hindsight Bias”, Arkes et al 1988
- “Forecasting Records by Maximum Likelihood”, Smith 1988
- “Tools for Thought”, Waddington 1977
- “The Futurists”, Toffler 1972
- “Tales from Prediction Markets”
- “Using Learning Curve Theory to Redefine Moore's Law”
- Sort By Magic
- Wikipedia
- Miscellaneous
- Link Bibliography
See Also
Links
“Evaluating Superhuman Models With Consistency Checks”, Fluri et al 2023
“Incentivizing Honest Performative Predictions With Proper Scoring Rules”, Oesterheld et al 2023
“Incentivizing honest performative predictions with proper scoring rules”
“Conditioning Predictive Models: Risks and Strategies”, Hubinger et al 2023
“The Unlikelihood Effect: When Knowing More Creates the Perception of Less”, Karmarkar & Kupor 2022
“The unlikelihood effect: When knowing more creates the perception of less”
“Does Constructing a Belief Distribution Truly Reduce Overconfidence?”, Hu & Simmons 2022
“Does constructing a belief distribution truly reduce overconfidence?”
“Reconciling Individual Probability Forecasts”, Roth et al 2022
“An Appropriate Verbal Probability Lexicon for Communicating Surgical Risks Is unlikely to Exist”, Harris et al 2022
“An appropriate verbal probability lexicon for communicating surgical risks is unlikely to exist”
“A Simple Cognitive Method to Improve the Prediction of Matters of Taste by Exploiting the Within-person Wisdom-of-crowd Effect”, Fujisaki et al 2022
“Language Models (Mostly) Know What They Know”, Kadavath et al 2022
“Forecasting Future World Events With Neural Networks”, Zou et al 2022
“Modeling Transformative AI Risks (MTAIR) Project—Summary Report”, Clarke et al 2022
“Modeling Transformative AI Risks (MTAIR) Project—Summary Report”
“Taking a Disagreeing Perspective Improves the Accuracy of People’s Quantitative Estimates”, Calseyde & Efendić 2022
“Taking a Disagreeing Perspective Improves the Accuracy of People’s Quantitative Estimates”
“Teaching Models to Express Their Uncertainty in Words”, Lin et al 2022
““Two Truths and a Lie” As a Class-participation Activity”, Gelman 2022
“DeepMind: The Podcast—Excerpts on AGI”, Kiely 2022
“Many Heads Are More Utilitarian Than One”, Keshmirian et al 2022
“Uncalibrated Models Can Improve Human-AI Collaboration”, Vodrahalli et al 2022
“A 680,000-person Megastudy of Nudges to Encourage Vaccination in Pharmacies”, Milkman et al 2022
“A 680,000-person megastudy of nudges to encourage vaccination in pharmacies”
“TACTiS: Transformer-Attentional Copulas for Time Series”, Drouin et al 2022
“Dream Interpretation from a Cognitive and Cultural Evolutionary Perspective: The Case of Oneiromancy in Traditional China”, Hong 2022
“M5 Accuracy Competition: Results, Findings, and Conclusions”, Makridakis et al 2022
“M5 accuracy competition: Results, findings, and conclusions”
“Megastudies Improve the Impact of Applied Behavioral Science”, Milkman et al 2021
“Megastudies improve the impact of applied behavioral science”
“Forecasting Skills in Experimental Markets: Illusion or Reality?”, Corgnet et al 2021
“Forecasting Skills in Experimental Markets: Illusion or Reality?”
“Strategically Overconfident (to a Fault): How Self-promotion Motivates Advisor Confidence”, Zant 2021
“Strategically overconfident (to a fault): How self-promotion motivates advisor confidence”
“Sigmoids Behaving Badly: Why They Usually Cannot Predict the Future as well as They Seem to Promise”, Sandberg et al 2021
“Wise Teamwork: Collective Confidence Calibration Predicts the Effectiveness of Group Discussion”, Silver 2021
“Wise teamwork: Collective confidence calibration predicts the effectiveness of group discussion”
“Alignment Problems With Current Forecasting Platforms”, Sempere & Lawsen 2021
“Behavioral Scientists and Laypeople Misestimate Societal Effects of COVID-19”, Hutcherson et al 2021
“Behavioral scientists and laypeople misestimate societal effects of COVID-19”
“How the Wisdom of Crowds, and of the Crowd Within, Are Affected by Expertise”, Fiechter & Kornell 2021
“How the wisdom of crowds, and of the crowd within, are affected by expertise”
“Mind the Gap: Assessing Temporal Generalization in Neural Language Models § Scaling”, Lazaridou et al 2021
“Mind the Gap: Assessing Temporal Generalization in Neural Language Models § Scaling”
“Kaggle Forecasting Competitions: An Overlooked Learning Opportunity”, Bojer & Meldgaard 2020
“Kaggle forecasting competitions: An overlooked learning opportunity”
“Danny Hernandez on Forecasting and the Drivers of AI Progress”, Koehler et al 2020
“Danny Hernandez on forecasting and the drivers of AI progress”
“AI and Efficiency: We’re Releasing an Analysis Showing That Since 2012 the Amount of Compute Needed to Train a Neural Net to the Same Performance on ImageNet Classification Has Been Decreasing by a Factor of 2 Every 16 Months”, Hernandez & Brown 2020
“ForecastQA: A Question Answering Challenge for Event Forecasting With Temporal Text Data”, Jin et al 2020
“ForecastQA: A Question Answering Challenge for Event Forecasting with Temporal Text Data”
“LightGBM: A Highly Efficient Gradient Boosting Decision Tree”, Ke et al 2019
“LightGBM: A Highly Efficient Gradient Boosting Decision Tree”
“Predicting History”, Risi et al 2019
“N-BEATS: Neural Basis Expansion Analysis for Interpretable Time Series Forecasting”, Oreshkin et al 2019
“N-BEATS: Neural basis expansion analysis for interpretable time series forecasting”
“Evidence on Good Forecasting Practices from the Good Judgment Project”, Impacts 2019
“Evidence on good forecasting practices from the Good Judgment Project”
“Forecasting Transformative AI: An Expert Survey”, Gruetzemacher et al 2019
“The Wisdom of Crowds Approach to Influenza-Rate Forecasting”, Morgan et al 2018
“The Wisdom of Crowds Approach to Influenza-Rate Forecasting”
“Predicting Replication Outcomes in the Many Labs 2 Study”, Forsell et al 2018
“Augur: a Decentralized Oracle and Prediction Market Platform”, Peterson & al 2018
“Augur: a Decentralized Oracle and Prediction Market Platform”
“The Wisdom of the Inner Crowd in Three Large Natural Experiments”, Dolder & Assem 2017
“The wisdom of the inner crowd in three large natural experiments”
“When Will AI Exceed Human Performance? Evidence from AI Experts”, Grace et al 2017
“When Will AI Exceed Human Performance? Evidence from AI Experts”
“DeepAR: Probabilistic Forecasting With Autoregressive Recurrent Networks”, Salinas et al 2017
“DeepAR: Probabilistic Forecasting with Autoregressive Recurrent Networks”
“Long Bets As Charitable Giving Opportunity”, Gwern 2017
“Embryo Selection For Intelligence”, Gwern 2016
“Looking Across and Looking Beyond the Knowledge Frontier: Intellectual Distance, Novelty, and Resource Allocation in Science”, Boudreau et al 2016
“Technology Forecasting: The Garden of Forking Paths”, Gwern 2014
“Complexity No Bar to AI”, Gwern 2014
“Darknet Market Mortality Risks”, Gwern 2013
“Mechanical Versus Clinical Data Combination in Selection and Admissions Decisions: A Meta-Analysis”, Kuncel et al 2013
“Mechanical Versus Clinical Data Combination in Selection and Admissions Decisions: A Meta-Analysis”
“Predicting Google Closures”, Gwern 2013
“Credit Suisse Global Investment Returns Yearbook 2013”, Dimson et al 2013
“Statistical Basis for Predicting Technological Progress”, Nagy et al 2012
“‘HP: Methods of Rationality’ Review Statistics”, Gwern 2012
“LSD Microdosing RCT”, Gwern 2012
“‘Methods of Rationality’ Predictions”, Gwern 2012
“Learning Performance of Prediction Markets With Kelly Bettors”, Beygelzimer et al 2012
“Learning Performance of Prediction Markets with Kelly Bettors”
“NGE Rebuild Predictions”, Gwern 2011
“A Prediction Market for Macro-Economic Variables”, Teschner et al 2011
“Why Do Humans Reason? Arguments for an Argumentative Theory”, Mercier & Sperber 2011
“Why do humans reason? Arguments for an argumentative theory”
“Zeo Sleep Self-experiments”, Gwern 2010
“The Ones Who Walk Towards Acre”, Gwern 2010
“Goodbye 2010”, Legg 2010
“About This Website”, Gwern 2010
“Applying the Fermi Estimation Technique to Business Problems”, Anderson & Sherman 2010
“Applying the Fermi Estimation Technique to Business Problems”
“Nootropics”, Gwern 2010
“Predicting the Next Big Thing: Success As a Signal of Poor Judgment”, Denrell & Fang 2010
“Predicting the Next Big Thing: Success as a Signal of Poor Judgment”
“Summers of Code, 2006–2013”, Gwern 2009
“Keep Your Identity Small”, Graham 2009
“Wikipedia & Knol: Why Knol Already Failed”, Gwern 2009
“In Defense of Inclusionism”, Gwern 2009
“Choosing Software”, Gwern 2008
“Measuring the Crowd Within: Probabilistic Representations Within Individuals”, Vul & Pashler 2008
“Measuring the Crowd Within: Probabilistic Representations Within Individuals”
“The Meta-Analysis of Clinical Judgment Project: 56 Years of Accumulated Research on Clinical Versus Statistical Prediction”, Ægisdóttir et al 2006
“A Systematic Review on Communicating With Patients about Evidence”, Trevena et al 2005
“A systematic review on communicating with patients about evidence”
“Principles of Forecasting: A Handbook for Researchers and Practitioners”, Armstrong 2001
“Principles of Forecasting: A Handbook for Researchers and Practitioners”
“Who Is Arguing About the Cat? Moral Action and Enlightenment According to Dōgen”, Mikkelson 1997
“Who Is Arguing About the Cat? Moral Action and Enlightenment According to Dōgen”
“Eliminating the Hindsight Bias”, Arkes et al 1988
“Forecasting Records by Maximum Likelihood”, Smith 1988
“Tools for Thought”, Waddington 1977
“The Futurists”, Toffler 1972
“Tales from Prediction Markets”
“Using Learning Curve Theory to Redefine Moore's Law”
Sort By Magic
Annotations sorted by machine learning into inferred 'tags'. This provides an alternative way to browse: instead of by date order, one can browse in topic order. The 'sorted' list has been automatically clustered into multiple sections & auto-labeled for easier browsing.
Beginning with the newest annotation, it uses the embedding of each annotation to attempt to create a list of nearest-neighbor annotations, creating a progression of topics. For more details, see the link.
ai-forecasting
forecasting
prediction
estimation
Wikipedia
Miscellaneous
-
/doc/statistics/prediction/2021-hutcherson-figure1-expertandlaymancoronavirusforecastsvsreality.png -
/doc/statistics/prediction/2019-aiimpacts-goodforecasting-gjp-ensemblingperformance.png -
/doc/statistics/prediction/2008-vul-figure1-wisdomofinnercrowdimprovespredictions.png -
https://forum.effectivealtruism.org/posts/H7xWzvwvkyywDAEkL/creating-a-database-for-base-rates -
https://joecarlsmith.com/2023/05/08/predictable-updating-about-ai-risk -
https://manifold.markets/IsaacKing/will-this-markets-probability-be-at -
https://marginalrevolution.com/marginalrevolution/2013/12/shiller-on-trills.html -
https://maximumprogress.substack.com/p/grading-extropian-predictions -
https://predictingpolitics.com/2021/01/09/mining-the-silver-lining-of-the-trump-presidency/ -
https://www.antipope.org/charlie/blog-static/2011/09/science-fiction-as-foresight.html -
https://www.astralcodexten.com/p/crowds-are-wise-and-ones-a-crowd -
https://www.astralcodexten.com/p/the-buying-things-from-a-store-faq -
https://www.cold-takes.com/the-track-record-of-futurists-seems-fine/ -
https://www.lesswrong.com/posts/68TGNutjDcBcq6PCZ/bitcoin-cryonics-fund?commentId=5Da2f8n9aXmfJ7FYA -
https://www.lesswrong.com/posts/6tErqpd2tDcpiBrX9/why-sigmoids-are-so-hard-to-predict -
https://www.lesswrong.com/posts/MSpfFBCQYw3YA8kMC/violating-the-emh-prediction-markets -
https://www.lesswrong.com/posts/c3cQgBN3v2Cxpe2kc/getting-gpt-3-to-predict-metaculus-questions -
https://www.lesswrong.com/posts/epgCXiv3Yy3qgcsys/you-can-t-predict-a-game-of-pinball -
https://www.openphilanthropy.org/research/how-accurate-are-our-predictions/
Link Bibliography
-
https://arxiv.org/abs/2207.05221#anthropic: “Language Models (Mostly) Know What They Know”, -
https://arxiv.org/abs/2206.15474: “Forecasting Future World Events With Neural Networks”, -
2022-gelman.pdf: ““Two Truths and a Lie” As a Class-participation Activity”, Andrew Gelman -
https://www.lesswrong.com/posts/SbAgRYo8tkHwhd9Qx/deepmind-the-podcast-excerpts-on-agi: “DeepMind: The Podcast—Excerpts on AGI”, William Kiely -
https://www.pnas.org/doi/10.1073/pnas.2115126119: “A 680,000-person Megastudy of Nudges to Encourage Vaccination in Pharmacies”, -
2022-hong.pdf: “Dream Interpretation from a Cognitive and Cultural Evolutionary Perspective: The Case of Oneiromancy in Traditional China”, Ze Hong -
https://www.sciencedirect.com/science/article/pii/S0169207021001874: “M5 Accuracy Competition: Results, Findings, and Conclusions”, Spyros Makridakis, Evangelos Spiliotis, Vassilios Assimakopoulos -
2021-milkman.pdf: “Megastudies Improve the Impact of Applied Behavioral Science”, -
https://arxiv.org/abs/2102.01951#scaling&org=deepmind: “Mind the Gap: Assessing Temporal Generalization in Neural Language Models § Scaling”, -
https://aiimpacts.org/evidence-on-good-forecasting-practices-from-the-good-judgment-project-an-accompanying-blog-post/: “Evidence on Good Forecasting Practices from the Good Judgment Project”, AI Impacts -
2017-vandolder.pdf: “The Wisdom of the Inner Crowd in Three Large Natural Experiments”, Dennie van Dolder, Martijn J. van den Assem -
long-bets: “Long Bets As Charitable Giving Opportunity”, Gwern -
embryo-selection: “Embryo Selection For Intelligence”, Gwern -
forking-path: “Technology Forecasting: The Garden of Forking Paths”, Gwern -
complexity: “Complexity No Bar to AI”, Gwern -
dnm-survival: “Darknet Market Mortality Risks”, Gwern -
2013-kuncel.pdf: “Mechanical Versus Clinical Data Combination in Selection and Admissions Decisions: A Meta-Analysis”, Nathan R. Kuncel, David M. Klieger, Brian S. Connelly, Deniz S. Ones -
google-shutdown: “Predicting Google Closures”, Gwern -
hpmor: “‘HP: Methods of Rationality’ Review Statistics”, Gwern -
lsd-microdosing: “LSD Microdosing RCT”, Gwern -
hpmor-prediction: “‘Methods of Rationality’ Predictions”, Gwern -
otaku-prediction: “NGE Rebuild Predictions”, Gwern -
zeo: “Zeo Sleep Self-experiments”, Gwern -
acre: “The Ones Who Walk Towards Acre”, Gwern -
https://www.vetta.org/2010/12/goodbye-2010/: “Goodbye 2010”, Shane Legg -
about: “About This Website”, Gwern -
nootropics: “Nootropics”, Gwern -
summer-of-code: “Summers of Code, 2006–2013”, Gwern -
wikipedia-and-knol: “Wikipedia & Knol: Why Knol Already Failed”, Gwern -
inclusionism: “In Defense of Inclusionism”, Gwern