--- title: '‘AI scaling’ directory' description: "Bibliography for directory ai/scaling, most recent first: 22 related tags, 807 annotations, & 106 links ()." thumbnail: /doc/fiction/humor/comics/meme-nnlayers-nnlol-template.png thumbnail-text: '' thumbnail-css: "outline" created: 2019-09-02 modified: 2026-09-29 status: in progress previous: /doc/ai/poetry/index next: /doc/ai/scaling/economics/index confidence: log importance: 0 css-extension: dropcaps-not index: True backlink: False ... ::: {#manual-annotation .abstract-small .abstract-tag-directory} [\[page summary\]](/doc/ai/scaling/abstract "Transclude link for doc/ai/scaling/ tag-abstract page."){#gwern-doc-ai-scaling-abstract .link-annotated-partial .include-content-core .include-strict .link-page} ::: # See Also ::: {#see-alsos .directory-indexes .columns} - [Parent ('*`AI`{=html}*' tag)](/doc/ai/index "Link to parent directory 'doc/ai/' (ascending)"){#_zsVHfDNX .link-annotated .link-tag .directory-indexes-upwards rel="tag"} - [*`AI economics`{=html}*](/doc/ai/scaling/economics/index "‘AI economics’ directory"){#_-ytgHk25 .link-annotated .link-tag .directory-indexes-downwards rel="tag"} - [*`grokking (NN)`{=html}*](/doc/ai/scaling/emergence/grokking/index "‘grokking (NN)’ directory"){#_MKt2P_R0 .link-annotated .link-tag .directory-indexes-downwards rel="tag"} - [*`AI emergence`{=html}*](/doc/ai/scaling/emergence/index "‘AI emergence’ directory"){#_6Y2vFxf4 .link-annotated .link-tag .directory-indexes-downwards rel="tag"} - [*`AI hardware`{=html}*](/doc/ai/scaling/hardware/index "‘AI hardware’ directory"){#_SnXXGYvz .link-annotated .link-tag .directory-indexes-downwards rel="tag"} - [*`MoE NN`{=html}*](/doc/ai/scaling/mixture-of-experts/index "‘MoE NN’ directory"){#_REiAxK2d .link-annotated .link-tag .directory-indexes-downwards rel="tag"} - [*`ML dataset`{=html}*](/doc/ai/dataset/index "‘ML dataset’ directory"){#_DgDlPcQ_ .link-annotated .link-tag .directory-indexes-sideways rel="tag"} - [*`Highleyman’s AI`{=html}*](/doc/ai/highleyman/index "‘Highleyman’s AI’ directory"){#_AODszYbl .link-annotated .link-tag .directory-indexes-sideways rel="tag"} - [*`diffusion NN`{=html}*](/doc/ai/nn/diffusion/index "‘diffusion NN’ directory"){#_e5Z3iqeU .link-annotated .link-tag .directory-indexes-sideways rel="tag"} - [*`MLP NN`{=html}*](/doc/ai/nn/fully-connected/index "‘MLP NN’ directory"){#_PV3GzkZY .link-annotated .link-tag .directory-indexes-sideways rel="tag"} - [*`BigGAN`{=html}*](/doc/ai/nn/gan/biggan/index "‘BigGAN’ directory"){#_CZTceyZH .link-annotated .link-tag .directory-indexes-sideways rel="tag"} - [*`CLIP`{=html}*](/doc/ai/nn/transformer/clip/index "‘CLIP’ directory"){#_TMPbFo84 .link-annotated .link-tag .directory-indexes-sideways rel="tag"} - [*`DALL·E`{=html}*](/doc/ai/nn/transformer/gpt/dall-e/index "‘DALL·E’ directory"){#_tVYKDgVI .link-annotated .link-tag .directory-indexes-sideways rel="tag"} - [*`instruct-tuning LLMs`{=html}*](/doc/ai/nn/transformer/gpt/instruction-tuning/index "‘instruct-tuning LLMs’ directory"){#_Mkb9lNiB .link-annotated .link-tag .directory-indexes-sideways rel="tag"} - [*`masked autoencoder`{=html}*](/doc/ai/nn/vae/mae/index "‘masked autoencoder’ directory"){#_zzQdZGVE .link-annotated .link-tag .directory-indexes-sideways rel="tag"} - [*`video analysis`{=html}*](/doc/ai/video/analysis/index "‘video analysis’ directory"){#_ggQkkuPP .link-annotated .link-tag .directory-indexes-sideways rel="tag"} - [*`video generation`{=html}*](/doc/ai/video/generation/index "‘video generation’ directory"){#_FVaGQXtH .link-annotated .link-tag .directory-indexes-sideways rel="tag"} - [*`active learning`{=html}*](/doc/reinforcement-learning/exploration/active-learning/index "‘active learning’ directory"){#_qVjF058G .link-annotated .link-tag .directory-indexes-sideways rel="tag"} - [*`continual learning`{=html}*](/doc/reinforcement-learning/meta-learning/continual-learning/index "‘continual learning’ directory"){#_CPEYsW4o .link-annotated .link-tag .directory-indexes-sideways rel="tag"} - [*`Decision Transformer`{=html}*](/doc/reinforcement-learning/model/decision-transformer/index "‘Decision Transformer’ directory"){#_e4c3cmDO .link-annotated .link-tag .directory-indexes-sideways rel="tag"} - [*`MARL`{=html}*](/doc/reinforcement-learning/multi-agent/index "‘MARL’ directory"){#_WyRlTYkp .link-annotated .link-tag .directory-indexes-sideways rel="tag"} - [*`robotics`{=html}*](/doc/reinforcement-learning/robot/index "‘robotics’ directory"){#_zOiNbx1W .link-annotated .link-tag .directory-indexes-sideways rel="tag"} - [*`RL scaling`{=html}*](/doc/reinforcement-learning/scaling/index "‘RL scaling’ directory"){#_vKu_j5JJ .link-annotated .link-tag .directory-indexes-sideways rel="tag"} ::: # Gwern {.display-pop-not} ## `“LLM Time Travel Visualization”, Gwern 2026`{=html} {#gwern-2026-llm-timetravel-section} **[`LLM Time Travel Visualization`{=html}](/blog/2026/llm-timetravel "'LLM Time Travel Visualization', Gwern 2026"){.link-modified-recently .link-annotated .id-not .include-annotation}** ## `“Dwarkesh Patel Interview”, Gwern & Patel 2024`{=html} {#gwern-patel-2024-section} **[`Dwarkesh Patel Interview`{=html}](/interview-dwarkesh "'Dwarkesh Patel Interview', Gwern & Patel 2024"){.link-annotated .id-not .include-annotation}** ## `“Anonymous Writer Who Predicted AI Trajectory on $12K⧸year Salary; Legacy & Anonymity in the Age of AGI [Video]”, Patel et al 2024`{=html} {#patel-et-al-2024-section} **[`Anonymous writer who predicted AI trajectory on $12K⧸year salary; Legacy & Anonymity in the Age of AGI [video]`{=html}](/doc/interview/2024-11-13-dwarkeshpatel-interviewwithgwernbranwen.webm "'Anonymous writer who predicted AI trajectory on $12K⧸year salary; Legacy & Anonymity in the Age of AGI [video]', Patel et al 2024"){.link-annotated-partial .id-not .include-annotation link-icon="file-video" link-icon-type="svg" link-icon-color="#acd147"}** ## `“Model Collapse Won’t Happen”, Gwern 2022`{=html} {#gwern-2022-no-model-collapse-section} **[`Model Collapse Won’t Happen`{=html}](https://www.lesswrong.com/posts/uKp6tBFStnsvrot5t/what-dall-e-2-can-and-cannot-do?commentId=CWKFyJYfgoZfP9955 "'Model Collapse Won’t Happen', Gwern 2022"){.link-annotated .id-not .include-annotation link-icon="LW" link-icon-type="text" link-icon-color="#7faf83"}** ## `“Scaling Image Generation Will Work”, Gwern 2022`{=html} {#gwern-2022-image-scaling-works-section} **[`Scaling Image Generation Will Work`{=html}](https://www.lesswrong.com/posts/uKp6tBFStnsvrot5t/what-dall-e-2-can-and-cannot-do?commentId=N5gFmWr2euf5avaKh "'Scaling Image Generation Will Work', Gwern 2022"){.link-annotated .id-not .include-annotation link-icon="LW" link-icon-type="text" link-icon-color="#7faf83"}** ## `“WBE and DRL: a Middle Way of Imitation Learning”, Gwern 2018`{=html} {#gwern-2018-brain-imitation-learning-section} **[`WBE and DRL: a Middle Way of imitation learning`{=html}](https://www.reddit.com/r/reinforcementlearning/comments/9pwy2f/wbe_and_drl_a_middle_way_of_imitation_learning/ "'WBE and DRL: a Middle Way of imitation learning', Gwern 2018"){.link-annotated .id-not .include-annotation link-icon="reddit" link-icon-type="svg" link-icon-color="#ff4500"}** ## `“30 Questions for Hans Moravec”, Gwern et al 2026`{=html} {#gwern-2026-questions-moravec-section} **[`30 Questions for Hans Moravec`{=html}](/blog/2026/questions-moravec "'30 Questions for Hans Moravec', Gwern et al 2026"){.link-annotated .id-not .include-annotation}** ## `“Gwern Visits BAIR”, Liu & Gwern 2025`{=html} {#liu-gwern-2025-section} **[`Gwern visits BAIR`{=html}](https://yuxi.ml/logs/posts/2025-gwern-bair/ "'Gwern visits BAIR', Liu & Gwern 2025"){.link-annotated-partial .id-not .include-annotation link-icon="Yuxi" link-icon-type="text,mono,quad" link-icon-color="#2c3e50"}** ## `“AI Cannibalism Can Be Good”, Gwern 2025`{=html} {#gwern-2025-ai-cannibalism-section} **[`AI Cannibalism Can Be Good`{=html}](/blog/2025/ai-cannibalism "'AI Cannibalism Can Be Good', Gwern 2025"){.link-annotated .id-not .include-annotation}** ## `“Absolute Unit NNs: Regression-Based MLPs for Everything”, Gwern 2023`{=html} {#gwern-aunn-section} **[`Absolute Unit NNs: Regression-Based MLPs for Everything`{=html}](/aunn "'Absolute Unit NNs: Regression-Based MLPs for Everything', Gwern 2023"){.link-annotated .id-not .include-annotation}** ## `“Is OpenAI OK?”, Gwern 2024`{=html} {#gwern-2024-openai-mojo-section} **[`Is OpenAI OK?`{=html}](https://www.reddit.com/r/mlscaling/comments/1djoqjh/ilya_sutskever_launches_safe_superintelligence_a/l9uogp9/ "'Is OpenAI OK?', Gwern 2024"){.link-annotated .id-not .include-annotation link-icon="reddit" link-icon-type="svg" link-icon-color="#ff4500"}** ## `“‘Winning’ AI Arms Races: Then What?”, Gwern 2024`{=html} {#gwern-2024-winning-arms-races-section} **[`‘Winning’ AI arms races: then what?`{=html}](https://www.lesswrong.com/posts/KPBPc7RayDPxqxdqY/china-hawks-are-manufacturing-an-ai-arms-race?commentId=pgSg4wfJ256xyz9i6 "'‘Winning’ AI arms races: then what?', Gwern 2024"){.link-annotated .id-not .include-annotation link-icon="LW" link-icon-type="text" link-icon-color="#7faf83"}** ## `“Scaling ‘Diminishing Returns’”, Gwern 2024`{=html} {#gwern-2024-diminishing-returns-section} **[`Scaling ‘diminishing returns’`{=html}](https://www.econlib.org/the-importance-of-diminishing-returns/#comment-359000 "'Scaling ‘diminishing returns’', Gwern 2024"){.link-annotated .id-not .include-annotation link-icon="econlib" link-icon-type="svg"}** ## `“Research Ideas”, Gwern 2017`{=html} {#gwern-idea-section} **[`Research Ideas`{=html}](/idea "'Research Ideas', Gwern 2017"){.link-annotated .id-not .include-annotation}** ## `“GPT-3 Creative Fiction”, Gwern 2020`{=html} {#gwern-gpt-3-section} **[`GPT-3 Creative Fiction`{=html}](/gpt-3 "'GPT-3 Creative Fiction', Gwern 2020"){.link-annotated .id-not .include-annotation}** ## `“GANs Didn’t Fail, They Were Abandoned”, Gwern 2022`{=html} {#gwern-gan-section} **[`GANs Didn’t Fail, They Were Abandoned`{=html}](/gan "'GANs Didn’t Fail, They Were Abandoned', Gwern 2022"){.link-annotated .id-not .include-annotation}** ## `“The Scaling Hypothesis”, Gwern 2020`{=html} {#gwern-scaling-hypothesis-section} **[`The Scaling Hypothesis`{=html}](/scaling-hypothesis "'The Scaling Hypothesis', Gwern 2020"){.link-annotated .id-not .include-annotation}** ## `“ML Scaling Subreddit”, Gwern 2020`{=html} {#gwern-2020-mlscaling-section} **[`ML Scaling subreddit`{=html}](https://www.reddit.com/r/mlscaling/ "'ML Scaling subreddit', Gwern 2020"){.link-annotated .id-not .include-annotation link-icon="reddit" link-icon-type="svg" link-icon-color="#ff4500"}** ## `“Computer Optimization: Your Computer Is Faster Than You Think”, Gwern 2021`{=html} {#gwern-note-faster-section} **[`Computer Optimization: Your Computer Is Faster Than You Think`{=html}](/note/faster "'Computer Optimization: Your Computer Is Faster Than You Think', Gwern 2021"){.link-annotated-partial .id-not .include-annotation}** ## `“Technology Forecasting: The Garden of Forking Paths”, Gwern 2014`{=html} {#gwern-forking-path-section} **[`Technology Forecasting: The Garden of Forking Paths`{=html}](/forking-path "'Technology Forecasting: The Garden of Forking Paths', Gwern 2014"){.link-annotated .id-not .include-annotation}** # Links {.display-pop-not} ## `“How Good Are Frontier Models at Physics? Expert Re-Grading Reveals Broken Evaluations and Near-Saturation of Leading Benchmarks”, Ansari et al 2026`{=html} {#ansari-et-al-2026-section} **[`How Good Are Frontier Models at Physics? Expert Re-Grading Reveals Broken Evaluations and Near-Saturation of Leading Benchmarks`{=html}](https://arxiv.org/abs/2609.13009 "'How Good Are Frontier Models at Physics? Expert Re-Grading Reveals Broken Evaluations and Near-Saturation of Leading Benchmarks', Ansari et al 2026"){.link-modified-recently .link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Countering Misuse of AI: September 2026”, Anthropic 2026`{=html} {#anthropic-2026-section} **[`Countering misuse of AI: September 2026`{=html}](https://www.anthropic.com/threat-intelligence-report-september-2026 "'Countering misuse of AI: September 2026', Anthropic 2026"){.link-modified-recently .link-annotated-partial .id-not .include-annotation link-icon="anthropic" link-icon-type="svg" link-icon-color="#d4a27f"}** ## `“Intelligence Scales As the Logarithm of Compute (& Data)”`{=html} {#_2mysFp_4-section} **[`Intelligence Scales as the Logarithm of Compute (& Data)`{=html}](https://www.lesswrong.com/posts/Gs7YJt5u9T95uWebu/intelligence-scales-as-the-logarithm-of-compute-and-data "Intelligence Scales as the Logarithm of Compute (& Data)"){.link-modified-recently .link-annotated-partial .id-not .include-annotation link-icon="LW" link-icon-type="text" link-icon-color="#7faf83"}** ## `“He Was Right About AI. What About the Fate of Mankind? Hans Moravec Predicted Human-Level AI by 2028. Now He Contemplates Our Species’ End—And His Own”, Kagan-Kans & Moravec 2026`{=html} {#kagan-kans-moravec-2026-section} **[`He Was Right About AI. What About the Fate of Mankind? Hans Moravec predicted human-level AI by 2028. Now he contemplates our species’ end—and his own`{=html}](https://nymag.com/intelligencer/article/hans-moravec-interview.html "'He Was Right About AI. What About the Fate of Mankind? Hans Moravec predicted human-level AI by 2028. Now he contemplates our species’ end—and his own', Kagan-Kans & Moravec 2026"){.id-not .include-annotation link-icon="𝒩𝒴" link-icon-type="text"}** ::: aux-links-transclude-file ::: collapse **View HTML**: [`https://nymag.com/intelligencer/article/hans-moravec-interview.html`](https://nymag.com/intelligencer/article/hans-moravec-interview.html){.id-not .link-annotated-not .include-content .include-lazy link-icon="𝒩𝒴" link-icon-type="text"} ::: ::: ## `“Duane Arnold: So Maybe I Should Enlighten You on What Happens in Your Absence”, Bjartur 2026`{=html} {#bjartur-2026-section} **[`Duane Arnold: So maybe I should enlighten you on what happens in your absence`{=html}](https://tomasbjartur.substack.com/p/duane-arnold "'Duane Arnold: So maybe I should enlighten you on what happens in your absence', Bjartur 2026"){.id-not .include-annotation link-icon="substack" link-icon-type="svg" link-icon-color="#ff6719"}** ## `“On Kimi K3: Its Capabilities And Related Discontents”`{=html} {#_nSUm2xdH-section} **[`On Kimi K3: Its Capabilities And Related Discontents`{=html}](https://thezvi.wordpress.com/2026/07/20/on-kimi-k3-its-capabilities-and-related-discontents/ "On Kimi K3: Its Capabilities And Related Discontents"){.link-annotated-partial .id-not .include-annotation}** ## `“Incoherent Values? Probing LLM Preferences Through Parametric Variation”, Ajayi et al 2026`{=html} {#ajayi-et-al-2026-section} **[`Incoherent Values? Probing LLM Preferences Through Parametric Variation`{=html}](https://arxiv.org/abs/2606.21102 "'Incoherent Values? Probing LLM Preferences Through Parametric Variation', Ajayi et al 2026"){.link-modified-recently .link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Estimating No-CoT Task-Completion Time Horizons of Frontier AI Models [3-Minute-Long Problems With P=0.5]”, Woodruff et al 2026`{=html} {#woodruff-et-al-2026-section} **[`Estimating No-CoT Task-Completion Time Horizons of Frontier AI Models [3-minute-long problems with P=0.5]`{=html}](https://www.lesswrong.com/posts/SieLowPgNgRSPGhFw/estimating-no-cot-task-completion-time-horizons-of-frontier "'Estimating No-CoT Task-Completion Time Horizons of Frontier AI Models [3-minute-long problems with P=0.5]', Woodruff et al 2026"){.link-annotated-partial .id-not .include-annotation link-icon="LW" link-icon-type="text" link-icon-color="#7faf83"}** ## `“Programmer Science Fiction: My Case for a New Sub-Genre”, Oates 2026`{=html} {#oates-2026-section} **[`Programmer Science Fiction: My case for a new sub-genre`{=html}](https://stoates.substack.com/p/programmer-science-fiction "'Programmer Science Fiction: My case for a new sub-genre', Oates 2026"){.link-annotated-partial .id-not .include-annotation link-icon="substack" link-icon-type="svg" link-icon-color="#ff6719"}** ## `industriaalist @ "2026-06-04"`{=html} {#dahal-2026-section} **[`Now that we’re running out of data, how do you optimally scale multi-epoch pretraining to hundreds of epochs? Our first paper from Q! q0 trains a population of models, instead of single model that saturates fast, reaching a dramatically lower loss at every epoch budget`{=html}](https://x.com/industriaalist/status/2062560899706327286 "'Now that we’re running out of data, how do you optimally scale multi-epoch pretraining to hundreds of epochs? Our first paper from Q! q0 trains a population of models, instead of single model that saturates fast, reaching a dramatically lower loss at every epoch budget', Dahal 2026"){.link-annotated-partial .id-not .include-annotation link-icon="twitter" link-icon-type="svg" link-icon-color="#1da1f2"}** ## `“q0: Primitives for Hyper-Epoch Pretraining”, Mandal et al 2026`{=html} {#mandal-et-al-2026-section} **[`q0: Primitives for Hyper-Epoch Pretraining`{=html}](https://arxiv.org/abs/2606.03938 "'q0: Primitives for Hyper-Epoch Pretraining', Mandal et al 2026"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Searching for Amanda Askell With Chinese Characteristics: If You Love Claude so Much, Why Don't You Hire a Philosopher?”, Caithrin 2026`{=html} {#caithrin-2026-section} **[`Searching for Amanda Askell with Chinese Characteristics: If you love Claude so much, why don't you hire a philosopher?`{=html}](https://www.caithrin.com/p/searching-for-amanda-askell-with "'Searching for Amanda Askell with Chinese Characteristics: If you love Claude so much, why don't you hire a philosopher?', Caithrin 2026"){.link-annotated .id-not .include-annotation}** ## `“Negation Neglect: When Models Fail to Learn Negations in Training”, Mayne et al 2026`{=html} {#mayne-et-al-2026-section} **[`Negation Neglect: When models fail to learn negations in training`{=html}](https://arxiv.org/abs/2605.13829 "'Negation Neglect: When models fail to learn negations in training', Mayne et al 2026"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Vision Banana: Image Generators Are Generalist Vision Learners”, Gabeur et al 2026`{=html} {#gabeur-et-al-2026-section} **[`Vision Banana: Image Generators are Generalist Vision Learners`{=html}](https://arxiv.org/abs/2604.20329#google "'Vision Banana: Image Generators are Generalist Vision Learners', Gabeur et al 2026"){.link-annotated .id-not .include-annotation link-icon="alphabet" link-icon-type="svg" link-icon-color="#4285f4"}** ## `“Steering Might Stop Working Soon”, Babcock 2026`{=html} {#babcock-2026-section} **[`Steering Might Stop Working Soon`{=html}](https://www.lesswrong.com/posts/fuzfbz8TbuLcskGCx/steering-might-stop-working-soon "'Steering Might Stop Working Soon', Babcock 2026"){.link-annotated-partial .id-not .include-annotation link-icon="LW" link-icon-type="text" link-icon-color="#7faf83"}** ## `“Introducing talkie: a 13b-Parameter Vintage Language Model from 1930”, Levine et al 2026`{=html} {#levine-et-al-2026-section} **[`Introducing talkie: a 13b-parameter vintage language model from 1930`{=html}](https://talkie-lm.com/introducing-talkie "'Introducing talkie: a 13b-parameter vintage language model from 1930', Levine et al 2026"){.link-annotated-partial .id-not .include-annotation}** ## `“My Hobby: Running Deranged Surveys; You Can Just Ask People Things”, Gao 2026`{=html} {#gao-2026-section} **[`My hobby: running deranged surveys; you can just ask people things`{=html}](https://nablatheta.substack.com/p/my-hobby-running-deranged-surveys "'My hobby: running deranged surveys; you can just ask people things', Gao 2026"){.link-annotated-partial .id-not .include-annotation link-icon="substack" link-icon-type="svg" link-icon-color="#ff6719"}** ## `“MDM-Prime-V2: Binary Encoding and Index Shuffling Enable Compute-Optimal Scaling of Diffusion Language Models”, Chao et al 2026`{=html} {#chao-et-al-2026-section} **[`MDM-Prime-v2: Binary Encoding and Index Shuffling Enable Compute-optimal Scaling of Diffusion Language Models`{=html}](https://arxiv.org/abs/2603.16077 "'MDM-Prime-v2: Binary Encoding and Index Shuffling Enable Compute-optimal Scaling of Diffusion Language Models', Chao et al 2026"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Elon Musk Pushes out More XAI Founders As AI Coding Effort Falters: Tesla and SpaceX Managers Sent in to Review Work As Billionaire’s Start-Up Struggles to Keep pace With Rivals”, Morris & Criddle 2026`{=html} {#morris-criddle-2026-section} **[`Elon Musk pushes out more xAI founders as AI coding effort falters: Tesla and SpaceX managers sent in to review work as billionaire’s start-up struggles to keep pace with rivals`{=html}](https://www.ft.com/content/e5fbc6c2-d5a6-4b97-a105-6a96ea849de5 "'Elon Musk pushes out more xAI founders as AI coding effort falters: Tesla and SpaceX managers sent in to review work as billionaire’s start-up struggles to keep pace with rivals', Morris & Criddle 2026"){.link-annotated .id-not .include-annotation link-icon="FT" link-icon-type="text" link-icon-color="#e3b68e"}** ## `“Neural Thickets: Diverse Task Experts Are Dense Around Pretrained Weights”, Gan & Isola 2026`{=html} {#gan-isola-2026-section} **[`Neural Thickets: Diverse Task Experts Are Dense Around Pretrained Weights`{=html}](https://arxiv.org/abs/2603.12228 "'Neural Thickets: Diverse Task Experts Are Dense Around Pretrained Weights', Gan & Isola 2026"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Against Time-Series Foundation Models; Or: My Experience in Modern Forecasting”, shako 2026`{=html} {#shako-2026-section} **[`Against Time-Series Foundation Models; Or: My Experience in Modern Forecasting`{=html}](https://shakoist.substack.com/p/against-time-series-foundation-models "'Against Time-Series Foundation Models; Or: My Experience in Modern Forecasting', shako 2026"){.link-annotated-partial .id-not .include-annotation link-icon="substack" link-icon-type="svg" link-icon-color="#ff6719"}** ## `“Revisiting the Platonic Representation Hypothesis: An Aristotelian View”, Gröger et al 2026`{=html} {#gröger-et-al-2026-section} **[`Revisiting the Platonic Representation Hypothesis: An Aristotelian View`{=html}](https://arxiv.org/abs/2602.14486 "'Revisiting the Platonic Representation Hypothesis: An Aristotelian View', Gröger et al 2026"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“AI Researchers’ Views on Automating AI R&D and Intelligence Explosions”, Field et al 2026`{=html} {#field-et-al-2026-section} **[`AI Researchers’ Views on Automating AI R&D and Intelligence Explosions`{=html}](https://arxiv.org/abs/2603.03338 "'AI Researchers’ Views on Automating AI R&D and Intelligence Explosions', Field et al 2026"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Elon Musk Suggests Spate of XAI Exits Have Been Push, Not Pull”`{=html} {#_64lFj71m-section} **[`Elon Musk suggests spate of xAI exits have been push, not pull`{=html}](https://techcrunch.com/2026/02/13/elon-musk-suggests-spate-of-xai-exits-have-been-push-not-pull/ "Elon Musk suggests spate of xAI exits have been push, not pull"){.id-not .include-annotation link-icon="TC" link-icon-type="text,mono" link-icon-color="#0a8935"}** ## `“Soft Contamination Means Benchmarks Test Shallow Generalization”, Spiesberger et al 2026`{=html} {#spiesberger-et-al-2026-section} **[`Soft Contamination Means Benchmarks Test Shallow Generalization`{=html}](https://arxiv.org/abs/2602.12413 "'Soft Contamination Means Benchmarks Test Shallow Generalization', Spiesberger et al 2026"){.link-modified-recently .link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“What Happened With Cotra’s Bio Anchors?”, Alexander 2026`{=html} {#alexander-2026-section} **[`What Happened With Cotra’s Bio Anchors?`{=html}](https://www.astralcodexten.com/p/what-happened-with-bio-anchors "'What Happened With Cotra’s Bio Anchors?', Alexander 2026"){.id-not .include-annotation link-icon="SSC" link-icon-type="text,tri" link-icon-color="#5175c2"}** ## `“Weight-Sparse Circuits May Be Interpretable Yet Unfaithful”, Drori 2026`{=html} {#drori-2026-section} **[`Weight-Sparse Circuits May Be Interpretable Yet Unfaithful`{=html}](https://www.lesswrong.com/posts/sHpZZnRDLg7ccX9aF/weight-sparse-circuits-may-be-interpretable-yet-unfaithful "'Weight-Sparse Circuits May Be Interpretable Yet Unfaithful', Drori 2026"){.link-annotated-partial .id-not .include-annotation link-icon="LW" link-icon-type="text" link-icon-color="#7faf83"}** ## `“Reflections on 2025: The Compute Theory of Everything, Grading the Homework of a Minor Deity, and the Acoustic Preferences of Atlantic Salmon [Learning to Feel the AGI]”, Albanie 2025`{=html} {#albanie-2025-section} **[`Reflections on 2025: The Compute Theory of Everything, grading the homework of a minor deity, and the acoustic preferences of Atlantic salmon [learning to feel the AGI]`{=html}](https://samuelalbanie.substack.com/p/reflections-on-2025 "'Reflections on 2025: The Compute Theory of Everything, grading the homework of a minor deity, and the acoustic preferences of Atlantic salmon [learning to feel the AGI]', Albanie 2025"){.id-not .include-annotation link-icon="substack" link-icon-type="svg" link-icon-color="#ff6719"}** ## `“2025 Letter: Compute, Inevitability, 2nd-Order Effects, Travel Tips, Andor & Isaiah Berlin [Learning to Feel the AGI]”, Wang 2025`{=html} {#wang-2025-section} **[`2025 letter: compute, inevitability, 2nd-order effects, travel tips, Andor & Isaiah Berlin [learning to feel the AGI]`{=html}](https://zhengdongwang.com/2025/12/30/2025-letter.html "'2025 letter: compute, inevitability, 2nd-order effects, travel tips, Andor & Isaiah Berlin [learning to feel the AGI]', Wang 2025"){.link-annotated-partial .id-not .include-annotation}** ## `“Shared Sensitivity to Data Distribution during Learning in Humans and Transformer Networks”, Lerousseau & Summerfield 2025`{=html} {#lerousseau-summerfield-2025-section} **[`Shared sensitivity to data distribution during learning in humans and Transformer networks`{=html}](https://www.nature.com/articles/s41562-025-02359-3 "'Shared sensitivity to data distribution during learning in humans and Transformer networks', Lerousseau & Summerfield 2025"){.link-annotated .id-not .include-annotation link-icon="n" link-icon-type="text"}** ## `“2025 LLM Year in Review”, Karpathy 2025`{=html} {#karpathy-2025-section} **[`2025 LLM Year in Review`{=html}](https://karpathy.bearblog.dev/year-in-review-2025/ "'2025 LLM Year in Review', Karpathy 2025"){.link-annotated-partial .id-not .include-annotation}** ## `“Universally Converging Representations of Matter Across Scientific Foundation Models”, Edamadaka et al 2025`{=html} {#edamadaka-et-al-2025-section} **[`Universally Converging Representations of Matter Across Scientific Foundation Models`{=html}](https://arxiv.org/abs/2512.03750 "'Universally Converging Representations of Matter Across Scientific Foundation Models', Edamadaka et al 2025"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Perch 2.0 Transfers ‘Whale’ to Underwater Tasks”, Burns et al 2025`{=html} {#burns-et-al-2025-section} **[`Perch 2.0 transfers ‘whale’ to underwater tasks`{=html}](https://arxiv.org/abs/2512.03219 "'Perch 2.0 transfers ‘whale’ to underwater tasks', Burns et al 2025"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“The Price of Progress: Algorithmic Efficiency and the Falling Cost of AI Inference”, Gundlach et al 2025`{=html} {#gundlach-et-al-2025-section} **[`The Price of Progress: Algorithmic Efficiency and the Falling Cost of AI Inference`{=html}](https://arxiv.org/abs/2511.23455 "'The Price of Progress: Algorithmic Efficiency and the Falling Cost of AI Inference', Gundlach et al 2025"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“A Rosetta Stone for AI Benchmarks”, Ho et al 2025`{=html} {#ho-et-al-2025-section} **[`A Rosetta Stone for AI Benchmarks`{=html}](https://arxiv.org/abs/2512.00193 "'A Rosetta Stone for AI Benchmarks', Ho et al 2025"){.link-modified-recently .link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Weight-Sparse Transformers Have Interpretable Circuits”, Gao et al 2025`{=html} {#gao-et-al-2025-section} **[`Weight-sparse transformers have interpretable circuits`{=html}](https://arxiv.org/abs/2511.13653#openai "'Weight-sparse transformers have interpretable circuits', Gao et al 2025"){.link-annotated .id-not .include-annotation link-icon="openai" link-icon-type="svg"}** ## `“Scaling Recommender Transformers to a Billion Parameters: How to Implement a New Generation of Transformer Recommenders”, Кhrylchenko 2025`{=html} {#кhrylchenko-2025-section} **[`Scaling Recommender Transformers to a Billion Parameters: How to implement a new generation of transformer recommenders`{=html}](https://towardsdatascience.com/scaling-recommender-transformers-to-a-billion-parameters/#yandex "'Scaling Recommender Transformers to a Billion Parameters: How to implement a new generation of transformer recommenders', Кhrylchenko 2025"){.link-annotated-partial .id-not .include-annotation link-icon="𝐌" link-icon-type="text"}** ## `“The Coverage Principle: How Pre-Training Enables Post-Training”, Chen et al 2025`{=html} {#chen-et-al-2025-section} **[`The Coverage Principle: How Pre-Training Enables Post-Training`{=html}](https://arxiv.org/abs/2510.15020 "'The Coverage Principle: How Pre-Training Enables Post-Training', Chen et al 2025"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Scaling Laws for Code: A More Data-Hungry Regime”, Luo et al 2025`{=html} {#luo-et-al-2025-section} **[`Scaling Laws for Code: A More Data-Hungry Regime`{=html}](https://arxiv.org/abs/2510.08702 "'Scaling Laws for Code: A More Data-Hungry Regime', Luo et al 2025"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Verbalized Sampling: How to Mitigate Mode Collapse and Unlock LLM Diversity”, Zhang et al 2025`{=html} {#zhang-et-al-2025-section} **[`Verbalized Sampling: How to Mitigate Mode Collapse and Unlock LLM Diversity`{=html}](https://arxiv.org/abs/2510.01171 "'Verbalized Sampling: How to Mitigate Mode Collapse and Unlock LLM Diversity', Zhang et al 2025"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Pre-Training under Infinite Compute”, Kim et al 2025`{=html} {#kim-et-al-2025-section} **[`Pre-training under infinite compute`{=html}](https://arxiv.org/abs/2509.14786 "'Pre-training under infinite compute', Kim et al 2025"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“‘I Have to Do It’: Why One of the World’s Most Brilliant AI Scientists Left the US for China”, Che 2025`{=html} {#che-2025-section} **[`‘I have to do it’: Why one of the world’s most brilliant AI scientists left the US for China`{=html}](https://www.theguardian.com/news/ng-interactive/2025/sep/16/song-chun-zhu-why-one-of-the-worlds-most-brilliant-ai-scientists-left-the-us-for-china "'‘I have to do it’: Why one of the world’s most brilliant AI scientists left the US for China', Che 2025"){.link-annotated-partial .id-not .include-annotation link-icon="the-guardian" link-icon-type="svg" link-icon-color="#052962"}** ## `“How Does A Blind Model See The Earth? A Tiny LLM Eval With Pretty Pictures”, Henry 2025`{=html} {#henry-2025-section} **[`How Does A Blind Model See The Earth? A tiny LLM eval with pretty pictures`{=html}](https://www.lesswrong.com/posts/xwdRzJxyqFqgXTWbH/how-does-a-blind-model-see-the-earth "'How Does A Blind Model See The Earth? A tiny LLM eval with pretty pictures', Henry 2025"){.link-annotated-partial .id-not .include-annotation link-icon="LW" link-icon-type="text" link-icon-color="#7faf83"}** ## `“GPT-5 Is Here: Our Smartest, Fastest, and Most Useful Model Yet, With Thinking Built In. Available to Everyone”, OpenAI 2025`{=html} {#openai-2025-gpt5-section} **[`GPT-5 is here: Our smartest, fastest, and most useful model yet, with thinking built in. Available to everyone`{=html}](https://openai.com/gpt-5/ "'GPT-5 is here: Our smartest, fastest, and most useful model yet, with thinking built in. Available to everyone', OpenAI 2025"){.link-annotated-partial .id-not .include-annotation link-icon="openai" link-icon-type="svg"}** ## `“Perch 2.0: The Bittern Lesson for Bioacoustics”, Merriënboer et al 2025`{=html} {#merriënboer-et-al-2025-section} **[`Perch 2.0: The Bittern Lesson for Bioacoustics`{=html}](https://arxiv.org/abs/2508.04665#google "'Perch 2.0: The Bittern Lesson for Bioacoustics', Merriënboer et al 2025"){.link-annotated .id-not .include-annotation link-icon="alphabet" link-icon-type="svg" link-icon-color="#4285f4"}** ## `“The Making Of Dario Amodei: AI’s Most Outspoken Leader Found Direction in a Personal Tragedy”, Kantrowitz 2025`{=html} {#kantrowitz-2025-section} **[`The Making Of Dario Amodei: AI’s most outspoken leader found direction in a personal tragedy`{=html}](https://www.bigtechnology.com/p/the-making-of-dario-amodei "'The Making Of Dario Amodei: AI’s most outspoken leader found direction in a personal tragedy', Kantrowitz 2025"){.link-annotated-partial .id-not .include-annotation}** ## `“Scaling Recommender Transformers to One Billion Parameters”, Khrylchenko et al 2025`{=html} {#khrylchenko-et-al-2025-section} **[`Scaling Recommender Transformers to One Billion Parameters`{=html}](https://arxiv.org/abs/2507.15994#yandex "'Scaling Recommender Transformers to One Billion Parameters', Khrylchenko et al 2025"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“How Many Instructions Can LLMs Follow at Once?”, Jaroslawicz et al 2025`{=html} {#jaroslawicz-et-al-2025-section} **[`How Many Instructions Can LLMs Follow at Once?`{=html}](https://arxiv.org/abs/2507.11538 "'How Many Instructions Can LLMs Follow at Once?', Jaroslawicz et al 2025"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Cloudflare Will Now Block AI Bots from Crawling Its Clients’ Websites by Default: The Company Will Also Introduce a ‘Pay-Per-Crawl’ System to Give Users More Fine-Grained Control over How AI Companies Can Access Their Sites”, Hall 2025`{=html} {#hall-2025-section} **[`Cloudflare will now block AI bots from crawling its clients’ websites by default: The company will also introduce a ‘pay-per-crawl’ system to give users more fine-grained control over how AI companies can access their sites`{=html}](https://www.technologyreview.com/2025/07/01/1119498/cloudflare-will-now-by-default-block-ai-bots-from-crawling-its-clients-websites "'Cloudflare will now block AI bots from crawling its clients’ websites by default: The company will also introduce a ‘pay-per-crawl’ system to give users more fine-grained control over how AI companies can access their sites', Hall 2025"){.id-not .include-annotation link-icon="T" link-icon-type="text,sans"}** ## `“Performance Prediction for Large Systems via Text-To-Text Regression”, Akhauri et al 2025`{=html} {#akhauri-et-al-2025-section} **[`Performance Prediction for Large Systems via Text-to-Text Regression`{=html}](https://arxiv.org/abs/2506.21718#google "'Performance Prediction for Large Systems via Text-to-Text Regression', Akhauri et al 2025"){.link-annotated .id-not .include-annotation link-icon="alphabet" link-icon-type="svg" link-icon-color="#4285f4"}** ## `“The Bitter Lesson Is Coming for Tokenization: a World of LLMs without Tokenization Is Desirable and Increasingly Possible”, Perić 2025`{=html} {#perić-2025-section} **[`The Bitter Lesson is coming for Tokenization: a world of LLMs without tokenization is desirable and increasingly possible`{=html}](https://lucalp.dev/bitter-lesson-tokenization-and-blt/ "'The Bitter Lesson is coming for Tokenization: a world of LLMs without tokenization is desirable and increasingly possible', Perić 2025"){.link-annotated .id-not .include-annotation}** ## `“Q1 AI Benchmarking Results: the Most Important Factor for Good Forecasting Is the Base Model”, Wilson 2025`{=html} {#wilson-2025-section} **[`Q1 AI Benchmarking Results: the most important factor for good forecasting is the base model`{=html}](https://www.metaculus.com/notebooks/38673/q1-ai-benchmarking-results/ "'Q1 AI Benchmarking Results: the most important factor for good forecasting is the base model', Wilson 2025"){.id-not .include-annotation link-icon="metaculus" link-icon-type="svg" link-icon-color="#001935"}** ## `“Common Corpus: The Largest Collection of Ethical Data for LLM Pre-Training”, Langlais et al 2025`{=html} {#langlais-et-al-2025-section} **[`Common Corpus: The Largest Collection of Ethical Data for LLM Pre-Training`{=html}](https://arxiv.org/abs/2506.01732 "'Common Corpus: The Largest Collection of Ethical Data for LLM Pre-Training', Langlais et al 2025"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Why I Don’t Think AGI Is Right around the Corner: Continual Learning Is a Huge Bottleneck”, Patel 2025`{=html} {#patel-2025-section} **[`Why I don’t think AGI is right around the corner: Continual learning is a huge bottleneck`{=html}](https://www.dwarkesh.com/p/timelines-june-2025 "'Why I don’t think AGI is right around the corner: Continual learning is a huge bottleneck', Patel 2025"){.id-not .include-annotation link-icon="Dwkh" link-icon-type="text,sans,quad" link-icon-color="#f3c016"}** ## `“What We Learned from Briefing 70+ Lawmakers on the Threat from AI”, Martínez 2025`{=html} {#martínez-2025-section} **[`What We Learned from Briefing 70+ Lawmakers on the Threat from AI`{=html}](https://www.lesswrong.com/posts/Xwrajm92fdjd7cqnN/what-we-learned-from-briefing-70-lawmakers-on-the-threat "'What We Learned from Briefing 70+ Lawmakers on the Threat from AI', Martínez 2025"){.link-annotated-partial .id-not .include-annotation link-icon="LW" link-icon-type="text" link-icon-color="#7faf83"}** ## `“Gen2seg: Generative Models Enable Generalizable Instance Segmentation”, Khangaonkar & Pirsiavash 2025`{=html} {#khangaonkar-pirsiavash-2025-section} **[`gen2seg: Generative Models Enable Generalizable Instance Segmentation`{=html}](https://arxiv.org/abs/2505.15263 "'gen2seg: Generative Models Enable Generalizable Instance Segmentation', Khangaonkar & Pirsiavash 2025"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Vec2vec: Harnessing the Universal Geometry of Embeddings”, Jha et al 2025`{=html} {#jha-et-al-2025-section} **[`vec2vec: Harnessing the Universal Geometry of Embeddings`{=html}](https://arxiv.org/abs/2505.12540 "'vec2vec: Harnessing the Universal Geometry of Embeddings', Jha et al 2025"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Neural Thermodynamic Laws for Large Language Model Training”, Liu et al 2025`{=html} {#liu-et-al-2025-section} **[`Neural Thermodynamic Laws for Large Language Model Training`{=html}](https://arxiv.org/abs/2505.10559 "'Neural Thermodynamic Laws for Large Language Model Training', Liu et al 2025"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Xi Takes an AI Masterclass: Inside the Politburo’s AI Study Session”, Schneider 2025`{=html} {#schneider-2025-section} **[`Xi Takes an AI Masterclass: Inside the Politburo’s AI Study Session`{=html}](https://www.chinatalk.media/p/xi-takes-an-ai-masterclass "'Xi Takes an AI Masterclass: Inside the Politburo’s AI Study Session', Schneider 2025"){.link-annotated-partial .id-not .include-annotation}** ## `“How To Scale: Model Size, Parameterization, HP Scaling Laws, Batch Size”, Seo 2025`{=html} {#seo-2025-section} **[`How To Scale: Model Size, Parameterization, HP Scaling Laws, Batch Size`{=html}](https://howtoscalenn.github.io/ "'How To Scale: Model Size, Parameterization, HP Scaling Laws, Batch Size', Seo 2025"){.id-not .include-annotation}** ::: aux-links-transclude-file ::: collapse **View External Link**: [`https://howtoscalenn.github.io/`](https://howtoscalenn.github.io/){.id-not .link-annotated-not .include-content .include-lazy} ::: ::: ## `“Psychometrically Derived 60-Question Benchmarks: Substantial Efficiencies and the Possibility of Human-AI Comparisons”, Gignac & Ilić 2025`{=html} {#gignac-ilić-2025-section} **[`Psychometrically derived 60-question benchmarks: Substantial efficiencies and the possibility of human-AI comparisons`{=html}](https://www.sciencedirect.com/science/article/pii/S016028962500025X "'Psychometrically derived 60-question benchmarks: Substantial efficiencies and the possibility of human-AI comparisons', Gignac & Ilić 2025"){.link-annotated .id-not .include-annotation link-icon="E" link-icon-type="text" link-icon-color="#eb6500"}** ## `“When ChatGPT Broke an Entire Field: An Oral History”, Pavlus 2025`{=html} {#pavlus-2025-section} **[`When ChatGPT Broke an Entire Field: An Oral History`{=html}](https://www.quantamagazine.org/when-chatgpt-broke-an-entire-field-an-oral-history-20250430/ "'When ChatGPT Broke an Entire Field: An Oral History', Pavlus 2025"){.link-annotated-partial .id-not .include-annotation link-icon="quanta" link-icon-type="svg" link-icon-color="#ff8600"}** ## `“Testing the Limit of Atmospheric Predictability With a Machine Learning Weather Model”, Vonich & Hakim 2025`{=html} {#vonich-hakim-2025-section} **[`Testing the Limit of Atmospheric Predictability with a Machine Learning Weather Model`{=html}](https://arxiv.org/abs/2504.20238 "'Testing the Limit of Atmospheric Predictability with a Machine Learning Weather Model', Vonich & Hakim 2025"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Safety Pretraining: Toward the Next Generation of Safe AI”, Maini et al 2025`{=html} {#maini-et-al-2025-section} **[`Safety Pretraining: Toward the Next Generation of Safe AI`{=html}](https://arxiv.org/abs/2504.16980 "'Safety Pretraining: Toward the Next Generation of Safe AI', Maini et al 2025"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Pre-Training Isn’t Dead, It’s Just Resting: GPT-4.5, the Value of RL, and the Economics of Frontier Training”, Chow & Gross-Whitaker 2025`{=html} {#chow-gross-whitaker-2025-section} **[`Pre-training isn’t dead, it’s just resting: GPT-4.5, the value of RL, and the economics of frontier training`{=html}](https://tmychow.substack.com/p/pre-training-isnt-dead-its-just-resting#%C2%A7is-pre-training-a-dead-end "'Pre-training isn’t dead, it’s just resting: GPT-4.5, the value of RL, and the economics of frontier training', Chow & Gross-Whitaker 2025"){.link-annotated-partial .id-not .include-annotation link-icon="substack" link-icon-type="svg" link-icon-color="#ff6719"}** ## `“Progress Report on a Toy Model Of Memorization”, Brave 2025`{=html} {#brave-2025-section} **[`Progress report on a Toy Model Of Memorization`{=html}](https://www.boristhebrave.com/2025/04/11/progress-report-on-a-toy-model-of-memorization/ "'Progress report on a Toy Model Of Memorization', Brave 2025"){.link-annotated-partial .id-not .include-annotation}** ## `“Cyc: Obituary for the Greatest Monument to Logical AGI”, Liu 2025`{=html} {#liu-2025-cyc-section} **[`Cyc: Obituary for the greatest monument to logical AGI`{=html}](https://yuxi.ml/essays/posts/cyc/ "'Cyc: Obituary for the greatest monument to logical AGI', Liu 2025"){.link-annotated .id-not .include-annotation link-icon="Yuxi" link-icon-type="text,mono,quad" link-icon-color="#2c3e50"}** ## `“Emuru: Zero-Shot Styled Text Image Generation, but Make It Autoregressive”, Pippi et al 2025`{=html} {#pippi-et-al-2025-section} **[`Emuru: Zero-Shot Styled Text Image Generation, but Make It Autoregressive`{=html}](https://arxiv.org/abs/2503.17074 "'Emuru: Zero-Shot Styled Text Image Generation, but Make It Autoregressive', Pippi et al 2025"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Compute-Optimal LLMs Provably Generalize Better With Scale”, Finzi et al 2025`{=html} {#finzi-et-al-2025-section} **[`Compute-Optimal LLMs Provably Generalize Better with Scale`{=html}](https://openreview.net/forum?id=MF7ljU8xcf "'Compute-Optimal LLMs Provably Generalize Better with Scale', Finzi et al 2025"){.link-annotated .id-not .include-annotation link-icon="OR" link-icon-type="text,sans" link-icon-color="#8c1b13"}** ## `“My Thoughts on the Future of ‘AI’”, Carlini 2025`{=html} {#carlini-2025-section} **[`My Thoughts on the Future of ‘AI’`{=html}](https://nicholas.carlini.com/writing/2025/thoughts-on-future-ai.html "'My Thoughts on the Future of ‘AI’', Carlini 2025"){.id-not .include-annotation}** ## `“Deep Learning Is Not So Mysterious or Different”, Wilson 2025`{=html} {#wilson-2025-section} **[`Deep Learning is Not So Mysterious or Different`{=html}](https://arxiv.org/abs/2503.02113 "'Deep Learning is Not So Mysterious or Different', Wilson 2025"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“SSI Israel Hires First Senior Researchers”, Gilead 2025`{=html} {#gilead-2025-section} **[`SSI Israel hires first senior researchers`{=html}](https://en.globes.co.il/en/article-ssi-israel-hires-first-senior-researchers-1001503535 "'SSI Israel hires first senior researchers', Gilead 2025"){.id-not .include-annotation}** ## `“Obscure Scientific Facts Benchmark”, Azulay 2025`{=html} {#azulay-2025-section} **[`Obscure Scientific Facts Benchmark`{=html}](https://github.com/AzulEye/obscure-scientific-facts-benchmark "'Obscure Scientific Facts Benchmark', Azulay 2025"){.link-annotated-partial .id-not .include-annotation link-icon="github" link-icon-type="svg"}** ## `“LLaDA: Large Language Diffusion Models”, Nie et al 2025`{=html} {#nie-et-al-2025-section} **[`LLaDA: Large Language Diffusion Models`{=html}](https://arxiv.org/abs/2502.09992 "'LLaDA: Large Language Diffusion Models', Nie et al 2025"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `tamaybes @ "2025-02-13"`{=html} {#besiroglu-2025-section} **[`[NanoGPT optimization experience curve]`{=html}](https://x.com/tamaybes/status/1890263322462994932 "'[NanoGPT optimization experience curve]', Besiroglu 2025"){.link-annotated .id-not .include-annotation link-icon="twitter" link-icon-type="svg" link-icon-color="#1da1f2"}** ## `“Over-Tokenized Transformer: Vocabulary Is Generally Worth Scaling”, Huang et al 2025`{=html} {#huang-et-al-2025-section} **[`Over-Tokenized Transformer: Vocabulary is Generally Worth Scaling`{=html}](https://arxiv.org/abs/2501.16975 "'Over-Tokenized Transformer: Vocabulary is Generally Worth Scaling', Huang et al 2025"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Do Generative Video Models Learn Physical Principles from Watching Videos?”, Motamed et al 2025`{=html} {#motamed-et-al-2025-section} **[`Do generative video models learn physical principles from watching videos?`{=html}](https://arxiv.org/abs/2501.09038#deepmind "'Do generative video models learn physical principles from watching videos?', Motamed et al 2025"){.link-annotated .id-not .include-annotation link-icon="deepmind" link-icon-type="svg" link-icon-color="#4185f4"}** ## `“Emergent Effects of Scaling on the Functional Hierarchies within Large Language Models”, Foop 2025`{=html} {#foop-2025-section} **[`Emergent effects of scaling on the functional hierarchies within large language models`{=html}](https://www.lesswrong.com/posts/LTxkwMGEJXMjYHoSh/emergent-effects-of-scaling-on-the-functional-hierarchies "'Emergent effects of scaling on the functional hierarchies within large language models', Foop 2025"){.link-annotated-partial .id-not .include-annotation link-icon="LW" link-icon-type="text" link-icon-color="#7faf83"}** ## `“What’s the Deal With Mid-Training?”, Doria 2025`{=html} {#doria-2025-section} **[`What’s the deal with mid-training?`{=html}](http://vintagedata.org/blog/posts/what-is-mid-training "'What’s the deal with mid-training?', Doria 2025"){.link-annotated-partial .id-not .include-annotation}** ## `“Things We Learned about LLMs in 2024”, Willison 2024`{=html} {#willison-2024-section} **[`Things we learned about LLMs in 2024`{=html}](https://simonwillison.net/2024/Dec/31/llms-in-2024/ "'Things we learned about LLMs in 2024', Willison 2024"){.link-annotated-partial .id-not .include-annotation}** ## `“2024 Letter [On LLM Benchmarking]”, Wang 2024`{=html} {#wang-2024-llm-section} **[`2024 letter [on LLM benchmarking]`{=html}](https://zhengdongwang.com/2024/12/29/2024-letter.html "'2024 letter [on LLM benchmarking]', Wang 2024"){.link-annotated-partial .id-not .include-annotation}** ## `“Smarter, Better, Faster, Longer: A Modern Bidirectional Encoder for Fast, Memory Efficient, and Long Context Finetuning and Inference”, Warner et al 2024`{=html} {#warner-et-al-2024-section} **[`Smarter, Better, Faster, Longer: A Modern Bidirectional Encoder for Fast, Memory Efficient, and Long Context Finetuning and Inference`{=html}](https://arxiv.org/abs/2412.13663 "'Smarter, Better, Faster, Longer: A Modern Bidirectional Encoder for Fast, Memory Efficient, and Long Context Finetuning and Inference', Warner et al 2024"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Byte Latent Transformer (BLT): Patches Scale Better Than Tokens”, Pagnoni et al 2024`{=html} {#pagnoni-et-al-2024-section} **[`Byte Latent Transformer (BLT): Patches Scale Better Than Tokens`{=html}](https://arxiv.org/abs/2412.09871#facebook "'Byte Latent Transformer (BLT): Patches Scale Better Than Tokens', Pagnoni et al 2024"){.link-annotated .id-not .include-annotation link-icon="facebook" link-icon-type="svg" link-icon-color="#1877f2"}** ## `“Densing Law of LLMs”, Xiao et al 2024`{=html} {#xiao-et-al-2024-section} **[`Densing Law of LLMs`{=html}](https://arxiv.org/abs/2412.04315 "'Densing Law of LLMs', Xiao et al 2024"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Liquid: Language Models Are Scalable and Unified Multi-Modal Generators”, Wu et al 2024`{=html} {#wu-et-al-2024-section} **[`Liquid: Language Models are Scalable and Unified Multi-modal Generators`{=html}](https://arxiv.org/abs/2412.04332 "'Liquid: Language Models are Scalable and Unified Multi-modal Generators', Wu et al 2024"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“PaliGemma 2: A Family of Versatile VLMs for Transfer”, Steiner et al 2024`{=html} {#steiner-et-al-2024-section} **[`PaliGemma 2: A Family of Versatile VLMs for Transfer`{=html}](https://arxiv.org/abs/2412.03555#google "'PaliGemma 2: A Family of Versatile VLMs for Transfer', Steiner et al 2024"){.link-annotated .id-not .include-annotation link-icon="alphabet" link-icon-type="svg" link-icon-color="#4285f4"}** ## `“Best-Of-N Jailbreaking”, Hughes et al 2024`{=html} {#hughes-et-al-2024-section} **[`Best-of-N Jailbreaking`{=html}](https://arxiv.org/abs/2412.03556 "'Best-of-N Jailbreaking', Hughes et al 2024"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Drowning in Documents: Consequences of Scaling Reranker Inference”, Jacob et al 2024`{=html} {#jacob-et-al-2024-section} **[`Drowning in Documents: Consequences of Scaling Reranker Inference`{=html}](https://arxiv.org/abs/2411.11767 "'Drowning in Documents: Consequences of Scaling Reranker Inference', Jacob et al 2024"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Medical Adaptation of Large Language and Vision-Language Models: Are We Making Progress?”, Jeong et al 2024`{=html} {#jeong-et-al-2024-section} **[`Medical Adaptation of Large Language and Vision-Language Models: Are We Making Progress?`{=html}](https://arxiv.org/abs/2411.04118 "'Medical Adaptation of Large Language and Vision-Language Models: Are We Making Progress?', Jeong et al 2024"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Fractal Patterns May Illuminate the Success of Next-Token Prediction”, Alabdulmohsin et al 2024`{=html} {#alabdulmohsin-et-al-2024-section} **[`Fractal Patterns May Illuminate the Success of Next-Token Prediction`{=html}](https://openreview.net/forum?id=clAFYReaYE "'Fractal Patterns May Illuminate the Success of Next-Token Prediction', Alabdulmohsin et al 2024"){.link-annotated .id-not .include-annotation link-icon="OR" link-icon-type="text,sans" link-icon-color="#8c1b13"}** ## `“How Far Is Video Generation from World Model: A Physical Law Perspective”, Kang et al 2024`{=html} {#kang-et-al-2024-section} **[`How Far is Video Generation from World Model: A Physical Law Perspective`{=html}](https://arxiv.org/abs/2411.02385 "'How Far is Video Generation from World Model: A Physical Law Perspective', Kang et al 2024"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Scaling up Masked Diffusion Models on Text”, Nie et al 2024`{=html} {#nie-et-al-2024-section} **[`Scaling up Masked Diffusion Models on Text`{=html}](https://arxiv.org/abs/2410.18514 "'Scaling up Masked Diffusion Models on Text', Nie et al 2024"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“ABBYY’s Bitter Lesson: How Linguists Lost the Last Battle for NLP”, Skorinkin 2024`{=html} {#skorinkin-2024-section} **[`ABBYY’s Bitter Lesson: How Linguists Lost the Last Battle for NLP`{=html}](https://sysblok.ru/blog/gorkij-urok-abbyy-kak-lingvisty-proigrali-poslednjuju-bitvu-za-nlp/ "'ABBYY’s Bitter Lesson: How Linguists Lost the Last Battle for NLP', Skorinkin 2024"){.id-not .include-annotation}** ## `“CT Foundation: Taking Medical Imaging Embeddings 3D”, Kiraly & Traverse 2024`{=html} {#kiraly-traverse-2024-section} **[`CT Foundation: Taking medical imaging embeddings 3D`{=html}](https://research.google/blog/taking-medical-imaging-embeddings-3d/ "'CT Foundation: Taking medical imaging embeddings 3D', Kiraly & Traverse 2024"){.link-annotated .id-not .include-annotation link-icon="alphabet" link-icon-type="svg" link-icon-color="#4285f4"}** ## `“Inference Scaling for Long-Context Retrieval Augmented Generation”, Yue et al 2024`{=html} {#yue-et-al-2024-section} **[`Inference Scaling for Long-Context Retrieval Augmented Generation`{=html}](https://arxiv.org/abs/2410.04343#deepmind "'Inference Scaling for Long-Context Retrieval Augmented Generation', Yue et al 2024"){.link-annotated .id-not .include-annotation link-icon="deepmind" link-icon-type="svg" link-icon-color="#4185f4"}** ## `“Strategic Insights from Simulation Gaming of AI Race Dynamics”, Gruetzemacher et al 2024`{=html} {#gruetzemacher-et-al-2024-section} **[`Strategic Insights from Simulation Gaming of AI Race Dynamics`{=html}](https://arxiv.org/abs/2410.03092 "'Strategic Insights from Simulation Gaming of AI Race Dynamics', Gruetzemacher et al 2024"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“How Feature Learning Can Improve Neural Scaling Laws”, Bordelon et al 2024`{=html} {#bordelon-et-al-2024-section} **[`How Feature Learning Can Improve Neural Scaling Laws`{=html}](https://arxiv.org/abs/2409.17858 "'How Feature Learning Can Improve Neural Scaling Laws', Bordelon et al 2024"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Dwarkesh Podcast Progress Update”, Patel 2024`{=html} {#patel-2024-1-section} **[`Dwarkesh Podcast Progress Update`{=html}](https://www.dwarkesh.com/p/progress-update "'Dwarkesh Podcast Progress Update', Patel 2024"){.link-annotated-partial .id-not .include-annotation link-icon="Dwkh" link-icon-type="text,sans,quad" link-icon-color="#f3c016"}** ## `“Safetywashing: Do AI Safety Benchmarks Actually Measure Safety Progress?”, Ren et al 2024`{=html} {#ren-et-al-2024-2-section} **[`Safetywashing: Do AI Safety Benchmarks Actually Measure Safety Progress?`{=html}](https://arxiv.org/abs/2407.21792 "'Safetywashing: Do AI Safety Benchmarks Actually Measure Safety Progress?', Ren et al 2024"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Physics of Language Models: Part 2.1, Grade-School Math and the Hidden Reasoning Process”, Ye et al 2024`{=html} {#ye-et-al-2024-2-section} **[`Physics of Language Models: Part 2.1, Grade-School Math and the Hidden Reasoning Process`{=html}](https://arxiv.org/abs/2407.20311 "'Physics of Language Models: Part 2.1, Grade-School Math and the Hidden Reasoning Process', Ye et al 2024"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Scaling Laws With Vocabulary: Larger Models Deserve Larger Vocabularies”, Tao et al 2024`{=html} {#tao-et-al-2024-section} **[`Scaling Laws with Vocabulary: Larger Models Deserve Larger Vocabularies`{=html}](https://arxiv.org/abs/2407.13623 "'Scaling Laws with Vocabulary: Larger Models Deserve Larger Vocabularies', Tao et al 2024"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Scaling Law in Neural Data: Non-Invasive Speech Decoding With 175 Hours of EEG Data”, Sato et al 2024`{=html} {#sato-et-al-2024-section} **[`Scaling Law in Neural Data: Non-Invasive Speech Decoding with 175 Hours of EEG Data`{=html}](https://arxiv.org/abs/2407.07595 "'Scaling Law in Neural Data: Non-Invasive Speech Decoding with 175 Hours of EEG Data', Sato et al 2024"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Future Events As Backdoor Triggers: Investigating Temporal Vulnerabilities in LLMs”, Price et al 2024`{=html} {#price-et-al-2024-2-section} **[`Future Events as Backdoor Triggers: Investigating Temporal Vulnerabilities in LLMs`{=html}](https://arxiv.org/abs/2407.04108 "'Future Events as Backdoor Triggers: Investigating Temporal Vulnerabilities in LLMs', Price et al 2024"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Resolving Discrepancies in Compute-Optimal Scaling of Language Models”, Porian et al 2024`{=html} {#porian-et-al-2024-section} **[`Resolving Discrepancies in Compute-Optimal Scaling of Language Models`{=html}](https://arxiv.org/abs/2406.19146 "'Resolving Discrepancies in Compute-Optimal Scaling of Language Models', Porian et al 2024"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Can Long-Context Language Models Subsume Retrieval, RAG, SQL, and More?”, Lee et al 2024`{=html} {#lee-et-al-2024-1-section} **[`Can Long-Context Language Models Subsume Retrieval, RAG, SQL, and More?`{=html}](https://arxiv.org/abs/2406.13121#google "'Can Long-Context Language Models Subsume Retrieval, RAG, SQL, and More?', Lee et al 2024"){.link-annotated .id-not .include-annotation link-icon="alphabet" link-icon-type="svg" link-icon-color="#4285f4"}** ## `“Probing the Decision Boundaries of In-Context Learning in Large Language Models”, Zhao et al 2024`{=html} {#zhao-et-al-2024-2-section} **[`Probing the Decision Boundaries of In-context Learning in Large Language Models`{=html}](https://arxiv.org/abs/2406.11233 "'Probing the Decision Boundaries of In-context Learning in Large Language Models', Zhao et al 2024"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“How Do Large Language Models Acquire Factual Knowledge During Pretraining?”, Chang et al 2024`{=html} {#chang-et-al-2024-2-section} **[`How Do Large Language Models Acquire Factual Knowledge During Pretraining?`{=html}](https://arxiv.org/abs/2406.11813 "'How Do Large Language Models Acquire Factual Knowledge During Pretraining?', Chang et al 2024"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Explore the Limits of Omni-Modal Pretraining at Scale”, Zhang et al 2024`{=html} {#zhang-et-al-2024-01-section} **[`Explore the Limits of Omni-modal Pretraining at Scale`{=html}](https://arxiv.org/abs/2406.09412 "'Explore the Limits of Omni-modal Pretraining at Scale', Zhang et al 2024"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Self-Consuming Generative Models With Curated Data Provably Optimize Human Preferences”, Ferbach et al 2024`{=html} {#ferbach-et-al-2024-section} **[`Self-Consuming Generative Models with Curated Data Provably Optimize Human Preferences`{=html}](https://arxiv.org/abs/2407.09499 "'Self-Consuming Generative Models with Curated Data Provably Optimize Human Preferences', Ferbach et al 2024"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Beyond Model Collapse: Scaling Up With Synthesized Data Requires Reinforcement”, Feng et al 2024`{=html} {#feng-et-al-2024-2-section} **[`Beyond Model Collapse: Scaling Up with Synthesized Data Requires Reinforcement`{=html}](https://arxiv.org/abs/2406.07515 "'Beyond Model Collapse: Scaling Up with Synthesized Data Requires Reinforcement', Feng et al 2024"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“AI Couple”, Siegel & Kirkwood 2024`{=html} {#siegel-kirkwood-2024-section} **[`AI Couple`{=html}](/doc/ai/scaling/2024-06-10-tommysiegeldankirkwood-thenewyorker-aicouple-howcanyouthinkaboutthatwitheverythingthatsgoingoninthefieldofai.jpg "'AI Couple', Siegel & Kirkwood 2024"){.link-annotated-partial .id-not .include-annotation link-icon="image" link-icon-type="svg"}** ## `“Attention As a Hypernetwork”, Schug et al 2024`{=html} {#schug-et-al-2024-section} **[`Attention as a Hypernetwork`{=html}](https://arxiv.org/abs/2406.05816 "'Attention as a Hypernetwork', Schug et al 2024"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Training Compute-Optimal Protein Language Models”, Cheng et al 2024`{=html} {#cheng-et-al-2024-1-section} **[`Training Compute-Optimal Protein Language Models`{=html}](https://www.biorxiv.org/content/10.1101/2024.06.06.597716.full "'Training Compute-Optimal Protein Language Models', Cheng et al 2024"){.link-annotated .id-not .include-annotation link-icon="chi-dna" link-icon-type="svg" link-icon-color="#bd2736"}** ## `“AI Will Become Mathematicians’ ‘Co-Pilot’: Fields Medalist Terence Tao Explains How Proof Checkers and AI Programs Are Dramatically Changing Mathematics”, Drösser & Tao 2024`{=html} {#drösser-tao-2024-section} **[`AI Will Become Mathematicians’ ‘Co-Pilot’: Fields Medalist Terence Tao explains how proof checkers and AI programs are dramatically changing mathematics`{=html}](https://www.scientificamerican.com/article/ai-will-become-mathematicians-co-pilot/ "'AI Will Become Mathematicians’ ‘Co-Pilot’: Fields Medalist Terence Tao explains how proof checkers and AI programs are dramatically changing mathematics', Drösser & Tao 2024"){.link-annotated .id-not .include-annotation link-icon="SA" link-icon-type="text" link-icon-color="#0376a1"}** ## `“Regularization Properties of Polynomial Bases”, Shtoff 2024`{=html} {#shtoff-2024-section} **[`Regularization properties of polynomial bases`{=html}](https://alexshtf.github.io/2024/06/03/PolynomialBasesRegProps.html "'Regularization properties of polynomial bases', Shtoff 2024"){.link-annotated-partial .id-not .include-annotation}** ## `“The Scaling Law in Stellar Light Curves”, Pan et al 2024`{=html} {#pan-et-al-2024-1-section} **[`The Scaling Law in Stellar Light Curves`{=html}](https://arxiv.org/abs/2405.17156 "'The Scaling Law in Stellar Light Curves', Pan et al 2024"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“AstroPT: Scaling Large Observation Models for Astronomy”, Smith et al 2024`{=html} {#smith-et-al-2024-section} **[`AstroPT: Scaling Large Observation Models for Astronomy`{=html}](https://arxiv.org/abs/2405.14930 "'AstroPT: Scaling Large Observation Models for Astronomy', Smith et al 2024"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Aurora: A Foundation Model for the Earth System”, Bodnar et al 2024`{=html} {#bodnar-et-al-2024-section} **[`Aurora: A Foundation Model for the Earth System`{=html}](https://arxiv.org/abs/2405.13063#microsoft "'Aurora: A Foundation Model for the Earth System', Bodnar et al 2024"){.link-annotated .id-not .include-annotation link-icon="MS" link-icon-type="text,sans,italic"}** ## `“The Platonic Representation Hypothesis”, Huh et al 2024`{=html} {#huh-et-al-2024-section} **[`The Platonic Representation Hypothesis`{=html}](https://arxiv.org/abs/2405.07987 "'The Platonic Representation Hypothesis', Huh et al 2024"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“XLSTM: Extended Long Short-Term Memory”, Beck et al 2024`{=html} {#beck-et-al-2024-section} **[`xLSTM: Extended Long Short-Term Memory`{=html}](https://arxiv.org/abs/2405.04517 "'xLSTM: Extended Long Short-Term Memory', Beck et al 2024"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Position: Understanding LLMs Requires More Than Statistical Generalization”, Reizinger et al 2024`{=html} {#reizinger-et-al-2024-section} **[`Position: Understanding LLMs Requires More Than Statistical Generalization`{=html}](https://arxiv.org/abs/2405.01964 "'Position: Understanding LLMs Requires More Than Statistical Generalization', Reizinger et al 2024"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“GSM1k: A Careful Examination of Large Language Model Performance on Grade School Arithmetic”, Zhang et al 2024`{=html} {#zhang-et-al-2024-11-section} **[`GSM1k: A Careful Examination of Large Language Model Performance on Grade School Arithmetic`{=html}](https://arxiv.org/abs/2405.00332#scale "'GSM1k: A Careful Examination of Large Language Model Performance on Grade School Arithmetic', Zhang et al 2024"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Scaling and Renormalization in High-Dimensional Regression”, Atanasov et al 2024`{=html} {#atanasov-et-al-2024-section} **[`Scaling and renormalization in high-dimensional regression`{=html}](https://arxiv.org/abs/2405.00592 "'Scaling and renormalization in high-dimensional regression', Atanasov et al 2024"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“CatLIP: CLIP-Level Visual Recognition Accuracy With 2.7× Faster Pre-Training on Web-Scale Image-Text Data”, Mehta et al 2024`{=html} {#mehta-et-al-2024-section} **[`CatLIP: CLIP-level Visual Recognition Accuracy with 2.7× Faster Pre-training on Web-scale Image-Text Data`{=html}](https://arxiv.org/abs/2404.15653#apple "'CatLIP: CLIP-level Visual Recognition Accuracy with 2.7× Faster Pre-training on Web-scale Image-Text Data', Mehta et al 2024"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Private Attribute Inference from Images With Vision-Language Models”, Tömekçe et al 2024`{=html} {#tömekçe-et-al-2024-section} **[`Private Attribute Inference from Images with Vision-Language Models`{=html}](https://arxiv.org/abs/2404.10618 "'Private Attribute Inference from Images with Vision-Language Models', Tömekçe et al 2024"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Test-Time Augmentation to Solve ARC”, Cole 2024`{=html} {#cole-2024-section} **[`Test-Time Augmentation to solve ARC`{=html}](https://lab42.global/community-interview-jack-cole/ "'Test-Time Augmentation to solve ARC', Cole 2024"){.link-annotated .id-not .include-annotation}** ## `“Compression Represents Intelligence Linearly”, Huang et al 2024`{=html} {#huang-et-al-2024-section} **[`Compression Represents Intelligence Linearly`{=html}](https://arxiv.org/abs/2404.09937 "'Compression Represents Intelligence Linearly', Huang et al 2024"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Chinchilla Scaling: A Replication Attempt”, Besiroglu et al 2024`{=html} {#besiroglu-et-al-2024-section} **[`Chinchilla Scaling: A replication attempt`{=html}](https://arxiv.org/abs/2404.10102 "'Chinchilla Scaling: A replication attempt', Besiroglu et al 2024"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Scaling (Down) CLIP: A Comprehensive Analysis of Data, Architecture, and Training Strategies”, Li et al 2024`{=html} {#li-et-al-2024-05-section} **[`Scaling (Down) CLIP: A Comprehensive Analysis of Data, Architecture, and Training Strategies`{=html}](https://arxiv.org/abs/2404.08197#google "'Scaling (Down) CLIP: A Comprehensive Analysis of Data, Architecture, and Training Strategies', Li et al 2024"){.link-annotated .id-not .include-annotation link-icon="alphabet" link-icon-type="svg" link-icon-color="#4285f4"}** ## `“Why Do Small Language Models Underperform? Studying Language Model Saturation via the Softmax Bottleneck”, Godey et al 2024`{=html} {#godey-et-al-2024-section} **[`Why do small language models underperform? Studying Language Model Saturation via the Softmax Bottleneck`{=html}](https://arxiv.org/abs/2404.07647 "'Why do small language models underperform? Studying Language Model Saturation via the Softmax Bottleneck', Godey et al 2024"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Language Imbalance Can Boost Cross-Lingual Generalization”, Schäfer et al 2024`{=html} {#schäfer-et-al-2024-section} **[`Language Imbalance Can Boost Cross-lingual Generalization`{=html}](https://arxiv.org/abs/2404.07982 "'Language Imbalance Can Boost Cross-lingual Generalization', Schäfer et al 2024"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“CulturalTeaming: AI-Assisted Interactive Red-Teaming for Challenging LLMs’ (Lack Of) Multicultural Knowledge”, Chiu et al 2024`{=html} {#chiu-et-al-2024-section} **[`CulturalTeaming: AI-Assisted Interactive Red-Teaming for Challenging LLMs’ (Lack of) Multicultural Knowledge`{=html}](https://arxiv.org/abs/2404.06664 "'CulturalTeaming: AI-Assisted Interactive Red-Teaming for Challenging LLMs’ (Lack of) Multicultural Knowledge', Chiu et al 2024"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Conformer-1: Robust ASR via Large-Scale Semi-Supervised Bootstrapping”, Zhang et al 2024`{=html} {#zhang-et-al-2024-12-section} **[`Conformer-1: Robust ASR via Large-Scale Semi-supervised Bootstrapping`{=html}](https://arxiv.org/abs/2404.07341#assembly "'Conformer-1: Robust ASR via Large-Scale Semi-supervised Bootstrapping', Zhang et al 2024"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“MiniCPM: Unveiling the Potential of Small Language Models With Scalable Training Strategies”, Hu et al 2024`{=html} {#hu-et-al-2024-1-section} **[`MiniCPM: Unveiling the Potential of Small Language Models with Scalable Training Strategies`{=html}](https://arxiv.org/abs/2404.06395 "'MiniCPM: Unveiling the Potential of Small Language Models with Scalable Training Strategies', Hu et al 2024"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Visual Autoregressive Modeling (VAR): Scalable Image Generation via Next-Scale Prediction”, Tian et al 2024`{=html} {#tian-et-al-2024-section} **[`Visual Autoregressive Modeling (VAR): Scalable Image Generation via Next-Scale Prediction`{=html}](https://arxiv.org/abs/2404.02905#bytedance "'Visual Autoregressive Modeling (VAR): Scalable Image Generation via Next-Scale Prediction', Tian et al 2024"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“The Rationale-Shaped Hole At The Heart Of Forecasting”, dschwarz et al 2024`{=html} {#dschwarz-et-al-2024-section} **[`The Rationale-Shaped Hole At The Heart Of Forecasting`{=html}](https://forum.effectivealtruism.org/posts/qMP7LcCBFBEtuA3kL/the-rationale-shaped-hole-at-the-heart-of-forecasting "'The Rationale-Shaped Hole At The Heart Of Forecasting', dschwarz et al 2024"){.link-annotated-partial .id-not .include-annotation link-icon="EA" link-icon-type="text" link-icon-color="#06819a"}** ## `“Is Model Collapse Inevitable? Breaking the Curse of Recursion by Accumulating Real and Synthetic Data”, Gerstgrasser et al 2024`{=html} {#gerstgrasser-et-al-2024-section} **[`Is Model Collapse Inevitable? Breaking the Curse of Recursion by Accumulating Real and Synthetic Data`{=html}](https://arxiv.org/abs/2404.01413 "'Is Model Collapse Inevitable? Breaking the Curse of Recursion by Accumulating Real and Synthetic Data', Gerstgrasser et al 2024"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Scaling Laws For Dense Retrieval”, Fang et al 2024`{=html} {#fang-et-al-2024-section} **[`Scaling Laws For Dense Retrieval`{=html}](https://arxiv.org/abs/2403.18684 "'Scaling Laws For Dense Retrieval', Fang et al 2024"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Long-Form Factuality in Large Language Models”, Wei et al 2024`{=html} {#wei-et-al-2024-1-section} **[`Long-form factuality in large language models`{=html}](https://arxiv.org/abs/2403.18802#deepmind "'Long-form factuality in large language models', Wei et al 2024"){.link-annotated .id-not .include-annotation link-icon="deepmind" link-icon-type="svg" link-icon-color="#4185f4"}** ## `“Mechanistic Design and Scaling of Hybrid Architectures”, Poli et al 2024`{=html} {#poli-et-al-2024-section} **[`Mechanistic Design and Scaling of Hybrid Architectures`{=html}](https://arxiv.org/abs/2403.17844 "'Mechanistic Design and Scaling of Hybrid Architectures', Poli et al 2024"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Data Mixing Laws: Optimizing Data Mixtures by Predicting Language Modeling Performance”, Ye et al 2024`{=html} {#ye-et-al-2024-section} **[`Data Mixing Laws: Optimizing Data Mixtures by Predicting Language Modeling Performance`{=html}](https://arxiv.org/abs/2403.16952 "'Data Mixing Laws: Optimizing Data Mixtures by Predicting Language Modeling Performance', Ye et al 2024"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“8 Google Employees Invented Modern AI. Here’s the Inside Story: They Met by Chance, Got Hooked on an Idea, and Wrote the Transformers Paper—The Most Consequential Tech Breakthrough in Recent History”, Levy 2024`{=html} {#levy-2024-2-section} **[`8 Google Employees Invented Modern AI. Here’s the Inside Story: They met by chance, got hooked on an idea, and wrote the Transformers paper—the most consequential tech breakthrough in recent history`{=html}](https://www.wired.com/story/eight-google-employees-invented-modern-ai-transformers-paper/ "'8 Google Employees Invented Modern AI. Here’s the Inside Story: They met by chance, got hooked on an idea, and wrote the Transformers paper—the most consequential tech breakthrough in recent history', Levy 2024"){.link-annotated .id-not .include-annotation link-icon="alphabet" link-icon-type="svg" link-icon-color="#4285f4"}** ## `“Inflection-2.5: Meet the World’s Best Personal AI”, Inflection 2024`{=html} {#inflection-2024-section} **[`Inflection-2.5: meet the world’s best personal AI`{=html}](https://inflection.ai/inflection-2-5 "'Inflection-2.5: meet the world’s best personal AI', Inflection 2024"){.link-annotated .id-not .include-annotation}** ## `“Revisiting Dynamic Evaluation: Online Adaptation for Large Language Models”, Rannen-Triki et al 2024`{=html} {#rannen-triki-et-al-2024-section} **[`Revisiting Dynamic Evaluation: Online Adaptation for Large Language Models`{=html}](https://arxiv.org/abs/2403.01518#deepmind "'Revisiting Dynamic Evaluation: Online Adaptation for Large Language Models', Rannen-Triki et al 2024"){.link-annotated .id-not .include-annotation link-icon="deepmind" link-icon-type="svg" link-icon-color="#4185f4"}** ## `“Actions Speak Louder Than Words: Trillion-Parameter Sequential Transducers for Generative Recommendations (HSTU)”, Zhai et al 2024`{=html} {#zhai-et-al-2024-1-section} **[`Actions Speak Louder than Words: Trillion-Parameter Sequential Transducers for Generative Recommendations (HSTU)`{=html}](https://arxiv.org/abs/2402.17152#facebook "'Actions Speak Louder than Words: Trillion-Parameter Sequential Transducers for Generative Recommendations (HSTU)', Zhai et al 2024"){.link-annotated .id-not .include-annotation link-icon="facebook" link-icon-type="svg" link-icon-color="#1877f2"}** ## `“When Scaling Meets LLM Finetuning: The Effect of Data, Model and Finetuning Method”, Zhang et al 2024`{=html} {#zhang-et-al-2024-07-section} **[`When Scaling Meets LLM Finetuning: The Effect of Data, Model and Finetuning Method`{=html}](https://arxiv.org/abs/2402.17193#deepmind "'When Scaling Meets LLM Finetuning: The Effect of Data, Model and Finetuning Method', Zhang et al 2024"){.link-annotated .id-not .include-annotation link-icon="deepmind" link-icon-type="svg" link-icon-color="#4185f4"}** ## `“Investigating Continual Pretraining in Large Language Models: Insights and Implications”, Yıldız et al 2024`{=html} {#yıldız-et-al-2024-section} **[`Investigating Continual Pretraining in Large Language Models: Insights and Implications`{=html}](https://arxiv.org/abs/2402.17400 "'Investigating Continual Pretraining in Large Language Models: Insights and Implications', Yıldız et al 2024"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“The Era of 1-Bit LLMs: All Large Language Models Are in 1.58 Bits”, Ma et al 2024`{=html} {#ma-et-al-2024-3-section} **[`The Era of 1-bit LLMs: All Large Language Models are in 1.58 Bits`{=html}](https://arxiv.org/abs/2402.17764#microsoft "'The Era of 1-bit LLMs: All Large Language Models are in 1.58 Bits', Ma et al 2024"){.link-annotated .id-not .include-annotation link-icon="MS" link-icon-type="text,sans,italic"}** ## `“StructLM: Towards Building Generalist Models for Structured Knowledge Grounding”, Zhuang et al 2024`{=html} {#zhuang-et-al-2024-section} **[`StructLM: Towards Building Generalist Models for Structured Knowledge Grounding`{=html}](https://arxiv.org/abs/2402.16671 "'StructLM: Towards Building Generalist Models for Structured Knowledge Grounding', Zhuang et al 2024"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“How to Train Data-Efficient LLMs”, Sachdeva et al 2024`{=html} {#sachdeva-et-al-2024-2-section} **[`How to Train Data-Efficient LLMs`{=html}](https://arxiv.org/abs/2402.09668#google "'How to Train Data-Efficient LLMs', Sachdeva et al 2024"){.link-annotated .id-not .include-annotation link-icon="alphabet" link-icon-type="svg" link-icon-color="#4285f4"}** ## `“Weaver: Foundation Models for Creative Writing”, Wang et al 2024`{=html} {#wang-et-al-2024-10-section} **[`Weaver: Foundation Models for Creative Writing`{=html}](https://arxiv.org/abs/2401.17268 "'Weaver: Foundation Models for Creative Writing', Wang et al 2024"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Arrows of Time for Large Language Models”, Papadopoulos et al 2024`{=html} {#papadopoulos-et-al-2024-section} **[`Arrows of Time for Large Language Models`{=html}](https://arxiv.org/abs/2401.17505 "'Arrows of Time for Large Language Models', Papadopoulos et al 2024"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Can AI Assistants Know What They Don’t Know?”, Cheng et al 2024`{=html} {#cheng-et-al-2024-3-section} **[`Can AI Assistants Know What They Don’t Know?`{=html}](https://arxiv.org/abs/2401.13275 "'Can AI Assistants Know What They Don’t Know?', Cheng et al 2024"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“I Am a Strange Dataset: Metalinguistic Tests for Language Models”, Thrush et al 2024`{=html} {#thrush-et-al-2024-2-section} **[`I am a Strange Dataset: Metalinguistic Tests for Language Models`{=html}](https://arxiv.org/abs/2401.05300 "'I am a Strange Dataset: Metalinguistic Tests for Language Models', Thrush et al 2024"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“The Perceptron Controversy: Connectionism Died in the 1960s from Technical Limits to Scaling, Then Resurrected in the 1980s After Backprop Allowed Scaling. The Minsky–Papert Anti-Scaling Hypothesis Explained, Psychoanalyzed, and Buried”, Liu 2024`{=html} {#liu-2024-section} **[`The Perceptron Controversy: Connectionism died in the 1960s from technical limits to scaling, then resurrected in the 1980s after backprop allowed scaling. The Minsky–Papert anti-scaling hypothesis explained, psychoanalyzed, and buried`{=html}](https://yuxi.ml/essays/posts/perceptron-controversy/ "'The Perceptron Controversy: Connectionism died in the 1960s from technical limits to scaling, then resurrected in the 1980s after backprop allowed scaling. The Minsky–Papert anti-scaling hypothesis explained, psychoanalyzed, and buried', Liu 2024"){.link-annotated-partial .id-not .include-annotation link-icon="Yuxi" link-icon-type="text,mono,quad" link-icon-color="#2c3e50"}** ## `“TF-T2V: A Recipe for Scaling up Text-To-Video Generation With Text-Free Videos”, Wang et al 2023`{=html} {#wang-et-al-2023-05-section} **[`TF-T2V: A Recipe for Scaling up Text-to-Video Generation with Text-free Videos`{=html}](https://arxiv.org/abs/2312.15770#alibaba "'TF-T2V: A Recipe for Scaling up Text-to-Video Generation with Text-free Videos', Wang et al 2023"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Generative Multimodal Models Are In-Context Learners”, Sun et al 2023`{=html} {#sun-et-al-2023-1-section} **[`Generative Multimodal Models are In-Context Learners`{=html}](https://arxiv.org/abs/2312.13286#baai "'Generative Multimodal Models are In-Context Learners', Sun et al 2023"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Zoology: Measuring and Improving Recall in Efficient Language Models”, Arora et al 2023`{=html} {#arora-et-al-2023-1-section} **[`Zoology: Measuring and Improving Recall in Efficient Language Models`{=html}](https://arxiv.org/abs/2312.04927 "'Zoology: Measuring and Improving Recall in Efficient Language Models', Arora et al 2023"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Seamless: Multilingual Expressive and Streaming Speech Translation”, Communication et al 2023`{=html} {#communication-et-al-2023-1-section} **[`Seamless: Multilingual Expressive and Streaming Speech Translation`{=html}](https://arxiv.org/abs/2312.05187#facebook "'Seamless: Multilingual Expressive and Streaming Speech Translation', Communication et al 2023"){.link-annotated .id-not .include-annotation link-icon="facebook" link-icon-type="svg" link-icon-color="#1877f2"}** ## `“Scaling Transformer Neural Networks for Skillful and Reliable Medium-Range Weather Forecasting”, Nguyen et al 2023`{=html} {#nguyen-et-al-2023-1-section} **[`Scaling transformer neural networks for skillful and reliable medium-range weather forecasting`{=html}](https://arxiv.org/abs/2312.03876 "'Scaling transformer neural networks for skillful and reliable medium-range weather forecasting', Nguyen et al 2023"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Instruction-Tuning Aligns LLMs to the Human Brain”, Aw et al 2023`{=html} {#aw-et-al-2023-section} **[`Instruction-tuning Aligns LLMs to the Human Brain`{=html}](https://arxiv.org/abs/2312.00575 "'Instruction-tuning Aligns LLMs to the Human Brain', Aw et al 2023"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Mamba: Linear-Time Sequence Modeling With Selective State Spaces”, Gu & Dao 2023`{=html} {#gu-dao-2023-section} **[`Mamba: Linear-Time Sequence Modeling with Selective State Spaces`{=html}](https://arxiv.org/abs/2312.00752 "'Mamba: Linear-Time Sequence Modeling with Selective State Spaces', Gu & Dao 2023"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Sequential Modeling Enables Scalable Learning for Large Vision Models”, Bai et al 2023`{=html} {#bai-et-al-2023-2-section} **[`Sequential Modeling Enables Scalable Learning for Large Vision Models`{=html}](https://arxiv.org/abs/2312.00785 "'Sequential Modeling Enables Scalable Learning for Large Vision Models', Bai et al 2023"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“UniRepLKNet: A Universal Perception Large-Kernel ConvNet for Audio, Video, Point Cloud, Time-Series and Image Recognition”, Ding et al 2023`{=html} {#ding-et-al-2023-1-section} **[`UniRepLKNet: A Universal Perception Large-Kernel ConvNet for Audio, Video, Point Cloud, Time-Series and Image Recognition`{=html}](https://arxiv.org/abs/2311.15599#tencent "'UniRepLKNet: A Universal Perception Large-Kernel ConvNet for Audio, Video, Point Cloud, Time-Series and Image Recognition', Ding et al 2023"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasets”, Blattmann et al 2023`{=html} {#blattmann-et-al-2023-section} **[`Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasets`{=html}](https://arxiv.org/abs/2311.15127#stability "'Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasets', Blattmann et al 2023"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“In Search of the Long-Tail: Systematic Generation of Long-Tail Inferential Knowledge via Logical Rule Guided Search”, Li et al 2023`{=html} {#li-et-al-2023-1-section} **[`In Search of the Long-Tail: Systematic Generation of Long-Tail Inferential Knowledge via Logical Rule Guided Search`{=html}](https://arxiv.org/abs/2311.07237 "'In Search of the Long-Tail: Systematic Generation of Long-Tail Inferential Knowledge via Logical Rule Guided Search', Li et al 2023"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“First Tragedy, Then Parse: History Repeats Itself in the New Era of Large Language Models”, Saphra et al 2023`{=html} {#saphra-et-al-2023-section} **[`First Tragedy, then Parse: History Repeats Itself in the New Era of Large Language Models`{=html}](https://arxiv.org/abs/2311.05020 "'First Tragedy, then Parse: History Repeats Itself in the New Era of Large Language Models', Saphra et al 2023"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“I2VGen-XL: High-Quality Image-To-Video Synthesis via Cascaded Diffusion Models”, Zhang et al 2023`{=html} {#zhang-et-al-2023-07-section} **[`I2VGen-XL: High-Quality Image-to-Video Synthesis via Cascaded Diffusion Models`{=html}](https://arxiv.org/abs/2311.04145#alibaba "'I2VGen-XL: High-Quality Image-to-Video Synthesis via Cascaded Diffusion Models', Zhang et al 2023"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“A Systematic Comparison of Syllogistic Reasoning in Humans and Language Models”, Eisape et al 2023`{=html} {#eisape-et-al-2023-section} **[`A Systematic Comparison of Syllogistic Reasoning in Humans and Language Models`{=html}](https://arxiv.org/abs/2311.00445#google "'A Systematic Comparison of Syllogistic Reasoning in Humans and Language Models', Eisape et al 2023"){.link-annotated .id-not .include-annotation link-icon="alphabet" link-icon-type="svg" link-icon-color="#4285f4"}** ## `“Sam Altman Accepts the 2023 Hawking Fellowship Award § Is There Another Breakthrough That’s Needed to Reach AGI?”, Altman 2023`{=html} {#altman-2023-1-section} **[`Sam Altman accepts the 2023 Hawking Fellowship Award § Is there another breakthrough that’s needed to reach AGI?`{=html}](https://www.youtube.com/watch?v=NjpNG0CJRMM&t=3705s "'Sam Altman accepts the 2023 Hawking Fellowship Award § Is there another breakthrough that’s needed to reach AGI?', Altman 2023"){.link-annotated .id-not .include-annotation link-icon="youtube" link-icon-type="svg" link-icon-color="#ff0033"}** ## `“ConvNets Match Vision Transformers at Scale”, Smith et al 2023`{=html} {#smith-et-al-2023-2-section} **[`ConvNets Match Vision Transformers at Scale`{=html}](https://arxiv.org/abs/2310.16764#deepmind "'ConvNets Match Vision Transformers at Scale', Smith et al 2023"){.link-annotated .id-not .include-annotation link-icon="deepmind" link-icon-type="svg" link-icon-color="#4185f4"}** ## `“Evidence of Interrelated Cognitive-Like Capabilities in Large Language Models: Indications of Artificial General Intelligence or Achievement?”, Ilić & Gignac 2023`{=html} {#ilić-gignac-2023-section} **[`Evidence of interrelated cognitive-like capabilities in large language models: Indications of artificial general intelligence or achievement?`{=html}](https://arxiv.org/abs/2310.11616 "'Evidence of interrelated cognitive-like capabilities in large language models: Indications of artificial general intelligence or achievement?', Ilić & Gignac 2023"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“PaLI-3 Vision Language Models: Smaller, Faster, Stronger”, Chen et al 2023`{=html} {#chen-et-al-2023-pali3-section} **[`PaLI-3 Vision Language Models: Smaller, Faster, Stronger`{=html}](https://arxiv.org/abs/2310.09199#google "'PaLI-3 Vision Language Models: Smaller, Faster, Stronger', Chen et al 2023"){.link-annotated .id-not .include-annotation link-icon="alphabet" link-icon-type="svg" link-icon-color="#4285f4"}** ## `“GeoLLM: Extracting Geospatial Knowledge from Large Language Models”, Manvi et al 2023`{=html} {#manvi-et-al-2023-section} **[`GeoLLM: Extracting Geospatial Knowledge from Large Language Models`{=html}](https://arxiv.org/abs/2310.06213 "'GeoLLM: Extracting Geospatial Knowledge from Large Language Models', Manvi et al 2023"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Dynamical versus Bayesian Phase Transitions in a Toy Model of Superposition”, Chen et al 2023`{=html} {#chen-et-al-2023-06-section} **[`Dynamical versus Bayesian Phase Transitions in a Toy Model of Superposition`{=html}](https://arxiv.org/abs/2310.06301 "'Dynamical versus Bayesian Phase Transitions in a Toy Model of Superposition', Chen et al 2023"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Sheared LLaMA: Accelerating Language Model Pre-Training via Structured Pruning”, Xia et al 2023`{=html} {#xia-et-al-2023-1-section} **[`Sheared LLaMA: Accelerating Language Model Pre-training via Structured Pruning`{=html}](https://arxiv.org/abs/2310.06694 "'Sheared LLaMA: Accelerating Language Model Pre-training via Structured Pruning', Xia et al 2023"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“FreshLLMs: Refreshing Large Language Models With Search Engine Augmentation”, Vu et al 2023`{=html} {#vu-et-al-2023-section} **[`FreshLLMs: Refreshing Large Language Models with Search Engine Augmentation`{=html}](https://arxiv.org/abs/2310.03214#google "'FreshLLMs: Refreshing Large Language Models with Search Engine Augmentation', Vu et al 2023"){.link-annotated .id-not .include-annotation link-icon="alphabet" link-icon-type="svg" link-icon-color="#4285f4"}** ## `“Never Train from Scratch: Fair Comparison of Long-Sequence Models Requires Data-Driven Priors”, Amos et al 2023`{=html} {#amos-et-al-2023-section} **[`Never Train from Scratch: Fair Comparison of Long-Sequence Models Requires Data-Driven Priors`{=html}](https://arxiv.org/abs/2310.02980 "'Never Train from Scratch: Fair Comparison of Long-Sequence Models Requires Data-Driven Priors', Amos et al 2023"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“MTOB: A Benchmark for Learning to Translate a New Language from One Grammar Book”, Tanzer et al 2023`{=html} {#tanzer-et-al-2023-section} **[`MTOB: A Benchmark for Learning to Translate a New Language from One Grammar Book`{=html}](https://arxiv.org/abs/2309.16575 "'MTOB: A Benchmark for Learning to Translate a New Language from One Grammar Book', Tanzer et al 2023"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Intriguing Properties of Generative Classifiers”, Jaini et al 2023`{=html} {#jaini-et-al-2023-section} **[`Intriguing properties of generative classifiers`{=html}](https://arxiv.org/abs/2309.16779#google "'Intriguing properties of generative classifiers', Jaini et al 2023"){.link-annotated .id-not .include-annotation link-icon="alphabet" link-icon-type="svg" link-icon-color="#4285f4"}** ## `“Taken out of Context: On Measuring Situational Awareness in LLMs”, Berglund et al 2023`{=html} {#berglund-et-al-2023-2-section} **[`Taken out of context: On measuring situational awareness in LLMs`{=html}](https://arxiv.org/abs/2309.00667 "'Taken out of context: On measuring situational awareness in LLMs', Berglund et al 2023"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“SeamlessM4T: Massively Multilingual & Multimodal Machine Translation”, Communication et al 2023`{=html} {#communication-et-al-2023-2-section} **[`SeamlessM4T: Massively Multilingual & Multimodal Machine Translation`{=html}](https://arxiv.org/abs/2308.11596#facebook "'SeamlessM4T: Massively Multilingual & Multimodal Machine Translation', Communication et al 2023"){.link-annotated .id-not .include-annotation link-icon="facebook" link-icon-type="svg" link-icon-color="#1877f2"}** ## `“Two Phases of Scaling Laws for Nearest Neighbor Classifiers”, Yang & Zhang 2023`{=html} {#yang-zhang-2023-section} **[`Two Phases of Scaling Laws for Nearest Neighbor Classifiers`{=html}](https://arxiv.org/abs/2308.08247 "'Two Phases of Scaling Laws for Nearest Neighbor Classifiers', Yang & Zhang 2023"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Simple Synthetic Data Reduces Sycophancy in Large Language Models”, Wei et al 2023`{=html} {#wei-et-al-2023-2-section} **[`Simple synthetic data reduces sycophancy in large language models`{=html}](https://arxiv.org/abs/2308.03958#deepmind "'Simple synthetic data reduces sycophancy in large language models', Wei et al 2023"){.link-annotated .id-not .include-annotation link-icon="deepmind" link-icon-type="svg" link-icon-color="#4185f4"}** ## `“Scaling Relationship on Learning Mathematical Reasoning With Large Language Models”, Yuan et al 2023`{=html} {#yuan-et-al-2023-section} **[`Scaling Relationship on Learning Mathematical Reasoning with Large Language Models`{=html}](https://arxiv.org/abs/2308.01825#alibaba "'Scaling Relationship on Learning Mathematical Reasoning with Large Language Models', Yuan et al 2023"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“LLaMA-2: Open Foundation and Fine-Tuned Chat Models”, Touvron et al 2023`{=html} {#touvron-et-al-2023-1-section} **[`LLaMA-2: Open Foundation and Fine-Tuned Chat Models`{=html}](https://arxiv.org/abs/2307.09288#facebook "'LLaMA-2: Open Foundation and Fine-Tuned Chat Models', Touvron et al 2023"){.link-annotated .id-not .include-annotation link-icon="facebook" link-icon-type="svg" link-icon-color="#1877f2"}** ## `“Measuring Faithfulness in Chain-Of-Thought Reasoning”, Lanham et al 2023`{=html} {#lanham-et-al-2023-section} **[`Measuring Faithfulness in Chain-of-Thought Reasoning`{=html}](https://arxiv.org/abs/2307.13702 "'Measuring Faithfulness in Chain-of-Thought Reasoning', Lanham et al 2023"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Unleashing the Emergent Cognitive Synergy in Large Language Models: A Task-Solving Agent through Multi-Persona Self-Collaboration”, Wang et al 2023`{=html} {#wang-et-al-2023-12-section} **[`Unleashing the Emergent Cognitive Synergy in Large Language Models: A Task-Solving Agent through Multi-Persona Self-Collaboration`{=html}](https://arxiv.org/abs/2307.05300#microsoft "'Unleashing the Emergent Cognitive Synergy in Large Language Models: A Task-Solving Agent through Multi-Persona Self-Collaboration', Wang et al 2023"){.link-annotated .id-not .include-annotation link-icon="MS" link-icon-type="text,sans,italic"}** ## `“Introducing Superalignment”, Leike & Sutskever 2023`{=html} {#leike-sutskever-2023-section} **[`Introducing Superalignment`{=html}](https://openai.com/index/introducing-superalignment/ "'Introducing Superalignment', Leike & Sutskever 2023"){.link-annotated .id-not .include-annotation link-icon="openai" link-icon-type="svg"}** ## `“Gödel, Escher, Bach Author Douglas Hofstadter on the State of AI Today § What about AI Terrifies You?”, Hofstadter & Kim 2023`{=html} {#hofstadter-kim-2023-section} **[`Gödel, Escher, Bach author Douglas Hofstadter on the state of AI today § What about AI terrifies you?`{=html}](https://www.youtube.com/watch?v=lfXxzAVtdpU&t=1763s "'Gödel, Escher, Bach author Douglas Hofstadter on the state of AI today § What about AI terrifies you?', Hofstadter & Kim 2023"){.link-annotated .id-not .include-annotation link-icon="youtube" link-icon-type="svg" link-icon-color="#ff0033"}** ## `“Pretraining Task Diversity and the Emergence of Non-Bayesian In-Context Learning for Regression”, Raventós et al 2023`{=html} {#raventós-et-al-2023-section} **[`Pretraining task diversity and the emergence of non-Bayesian in-context learning for regression`{=html}](https://arxiv.org/abs/2306.15063 "'Pretraining task diversity and the emergence of non-Bayesian in-context learning for regression', Raventós et al 2023"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Beyond Scale: the Diversity Coefficient As a Data Quality Metric Demonstrates LLMs Are Pre-Trained on Formally Diverse Data”, Lee et al 2023`{=html} {#lee-et-al-2023-4-section} **[`Beyond Scale: the Diversity Coefficient as a Data Quality Metric Demonstrates LLMs are Pre-trained on Formally Diverse Data`{=html}](https://arxiv.org/abs/2306.13840 "'Beyond Scale: the Diversity Coefficient as a Data Quality Metric Demonstrates LLMs are Pre-trained on Formally Diverse Data', Lee et al 2023"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Scaling MLPs: A Tale of Inductive Bias”, Bachmann et al 2023`{=html} {#bachmann-et-al-2023-section} **[`Scaling MLPs: A Tale of Inductive Bias`{=html}](https://arxiv.org/abs/2306.13575 "'Scaling MLPs: A Tale of Inductive Bias', Bachmann et al 2023"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Inflection-1: Pi’s Best-In-Class LLM”, Inflection 2023`{=html} {#inflection-2023-section} **[`Inflection-1: Pi’s Best-in-Class LLM`{=html}](https://inflection.ai/blog/inflection-1 "'Inflection-1: Pi’s Best-in-Class LLM', Inflection 2023"){.id-not .include-annotation}** ## `“Understanding Social Reasoning in Language Models With Language Models”, Gandhi et al 2023`{=html} {#gandhi-et-al-2023-2-section} **[`Understanding Social Reasoning in Language Models with Language Models`{=html}](https://arxiv.org/abs/2306.15448 "'Understanding Social Reasoning in Language Models with Language Models', Gandhi et al 2023"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Image Captioners Are Scalable Vision Learners Too”, Tschannen et al 2023`{=html} {#tschannen-et-al-2023-2-section} **[`Image Captioners Are Scalable Vision Learners Too`{=html}](https://arxiv.org/abs/2306.07915#deepmind "'Image Captioners Are Scalable Vision Learners Too', Tschannen et al 2023"){.link-annotated .id-not .include-annotation link-icon="deepmind" link-icon-type="svg" link-icon-color="#4185f4"}** ## `“PaLI-X: On Scaling up a Multilingual Vision and Language Model”, Chen et al 2023`{=html} {#chen-et-al-2023-palix-section} **[`PaLI-X: On Scaling up a Multilingual Vision and Language Model`{=html}](https://arxiv.org/abs/2305.18565#google "'PaLI-X: On Scaling up a Multilingual Vision and Language Model', Chen et al 2023"){.link-annotated .id-not .include-annotation link-icon="alphabet" link-icon-type="svg" link-icon-color="#4285f4"}** ## `“The False Promise of Imitating Proprietary LLMs”, Gudibande et al 2023`{=html} {#gudibande-et-al-2023-section} **[`The False Promise of Imitating Proprietary LLMs`{=html}](https://arxiv.org/abs/2305.15717 "'The False Promise of Imitating Proprietary LLMs', Gudibande et al 2023"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Scaling Data-Constrained Language Models”, Muennighoff et al 2023`{=html} {#muennighoff-et-al-2023-repeatscaling-section} **[`Scaling Data-Constrained Language Models`{=html}](https://arxiv.org/abs/2305.16264#huggingface "'Scaling Data-Constrained Language Models', Muennighoff et al 2023"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“FST: Improving Speech Translation by Fusing Speech and Text”, Yin et al 2023`{=html} {#yin-et-al-2023-section} **[`FST: Improving speech translation by fusing speech and text`{=html}](https://arxiv.org/abs/2305.14042#bytedance "'FST: Improving speech translation by fusing speech and text', Yin et al 2023"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Scaling Laws for Language Encoding Models in FMRI”, Antonello et al 2023`{=html} {#antonello-et-al-2023-section} **[`Scaling laws for language encoding models in fMRI`{=html}](https://arxiv.org/abs/2305.11863 "'Scaling laws for language encoding models in fMRI', Antonello et al 2023"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“LIMA: Less Is More for Alignment”, Zhou et al 2023`{=html} {#zhou-et-al-2023-09-section} **[`LIMA: Less Is More for Alignment`{=html}](https://arxiv.org/abs/2305.11206 "'LIMA: Less Is More for Alignment', Zhou et al 2023"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“PaLM 2 Technical Report”, Anil et al 2023`{=html} {#anil-et-al-2023-section} **[`PaLM 2 Technical Report`{=html}](https://arxiv.org/abs/2305.10403#google "'PaLM 2 Technical Report', Anil et al 2023"){.link-annotated .id-not .include-annotation link-icon="alphabet" link-icon-type="svg" link-icon-color="#4285f4"}** ## `“Google’s Newest AI Model Uses Nearly 5× More Text Data for Training Than Its Predecessor”, Elias 2023`{=html} {#elias-2023-1-section} **[`Google’s newest AI model uses nearly 5× more text data for training than its predecessor`{=html}](https://www.cnbc.com/2023/05/16/googles-palm-2-uses-nearly-five-times-more-text-data-than-predecessor.html "'Google’s newest AI model uses nearly 5× more text data for training than its predecessor', Elias 2023"){.link-annotated .id-not .include-annotation link-icon="alphabet" link-icon-type="svg" link-icon-color="#4285f4"}** ## `“TorToise: Better Speech Synthesis through Scaling”, Betker 2023`{=html} {#betker-2023-section} **[`TorToise: Better speech synthesis through scaling`{=html}](https://arxiv.org/abs/2305.07243 "'TorToise: Better speech synthesis through scaling', Betker 2023"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“TinyStories: How Small Can Language Models Be and Still Speak Coherent English?”, Eldan & Li 2023`{=html} {#eldan-li-2023-section} **[`TinyStories: How Small Can Language Models Be and Still Speak Coherent English?`{=html}](https://arxiv.org/abs/2305.07759#microsoft "'TinyStories: How Small Can Language Models Be and Still Speak Coherent English?', Eldan & Li 2023"){.link-annotated .id-not .include-annotation link-icon="MS" link-icon-type="text,sans,italic"}** ## `“ImageBind: One Embedding Space To Bind Them All”, Girdhar et al 2023`{=html} {#girdhar-et-al-2023-section} **[`ImageBind: One Embedding Space To Bind Them All`{=html}](https://arxiv.org/abs/2305.05665#facebook "'ImageBind: One Embedding Space To Bind Them All', Girdhar et al 2023"){.link-annotated .id-not .include-annotation link-icon="facebook" link-icon-type="svg" link-icon-color="#1877f2"}** ## `“Finding Neurons in a Haystack: Case Studies With Sparse Probing”, Gurnee et al 2023`{=html} {#gurnee-et-al-2023-section} **[`Finding Neurons in a Haystack: Case Studies with Sparse Probing`{=html}](https://arxiv.org/abs/2305.01610 "'Finding Neurons in a Haystack: Case Studies with Sparse Probing', Gurnee et al 2023"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Geoffrey Hinton Tells Us Why He’s Now Scared of the Tech He Helped Build: ‘I Have Suddenly Switched My Views on Whether These Things Are Going to Be More Intelligent Than Us.’”, Heaven 2023`{=html} {#heaven-2023-section} **[`Geoffrey Hinton tells us why he’s now scared of the tech he helped build: ‘I have suddenly switched my views on whether these things are going to be more intelligent than us.’`{=html}](https://www.technologyreview.com/2023/05/02/1072528/geoffrey-hinton-google-why-scared-ai/ "'Geoffrey Hinton tells us why he’s now scared of the tech he helped build: ‘I have suddenly switched my views on whether these things are going to be more intelligent than us.’', Heaven 2023"){.link-annotated .id-not .include-annotation link-icon="alphabet" link-icon-type="svg" link-icon-color="#4285f4"}** ## `“Speak, Memory: An Archaeology of Books Known to ChatGPT/GPT-4”, Chang et al 2023`{=html} {#chang-et-al-2023-1-section} **[`Speak, Memory: An Archaeology of Books Known to ChatGPT/GPT-4`{=html}](https://arxiv.org/abs/2305.00118 "'Speak, Memory: An Archaeology of Books Known to ChatGPT/GPT-4', Chang et al 2023"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Google’s DeepMind-Brain Merger: Tech Giant Regroups for AI Battle”, Murgia 2023`{=html} {#murgia-2023-1-section} **[`Google’s DeepMind-Brain merger: tech giant regroups for AI battle`{=html}](https://www.ft.com/content/f4f73815-6fc2-4016-bd97-4bace459e95e "'Google’s DeepMind-Brain merger: tech giant regroups for AI battle', Murgia 2023"){.link-annotated .id-not .include-annotation link-icon="FT" link-icon-type="text" link-icon-color="#e3b68e"}** ## `“CLaMP: Contrastive Language-Music Pre-Training for Cross-Modal Symbolic Music Information Retrieval”, Wu et al 2023`{=html} {#wu-et-al-2023-5-section} **[`CLaMP: Contrastive Language-Music Pre-training for Cross-Modal Symbolic Music Information Retrieval`{=html}](https://arxiv.org/abs/2304.11029#microsoft "'CLaMP: Contrastive Language-Music Pre-training for Cross-Modal Symbolic Music Information Retrieval', Wu et al 2023"){.link-annotated .id-not .include-annotation link-icon="MS" link-icon-type="text,sans,italic"}** ## `“Emergent and Predictable Memorization in Large Language Models”, Biderman et al 2023`{=html} {#biderman-et-al-2023-1-section} **[`Emergent and Predictable Memorization in Large Language Models`{=html}](https://arxiv.org/abs/2304.11158#eleutherai "'Emergent and Predictable Memorization in Large Language Models', Biderman et al 2023"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Power Law Trends in Speedrunning and Machine Learning”, Erdil & Sevilla 2023`{=html} {#erdil-sevilla-2023-section} **[`Power Law Trends in Speedrunning and Machine Learning`{=html}](https://arxiv.org/abs/2304.10004 "'Power Law Trends in Speedrunning and Machine Learning', Erdil & Sevilla 2023"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Even The Politicians Thought the Open Letter Made No Sense In The Senate Hearing on AI Today’s Hearing on Ai Covered Ai Regulation and Challenges, and the Infamous Open Letter, Which Nearly Everyone in the Room Thought Was Unwise”, Gorrell 2023`{=html} {#gorrell-2023-section} **[`Even The Politicians Thought the Open Letter Made No Sense In The Senate Hearing on AI today’s hearing on ai covered ai regulation and challenges, and the infamous open letter, which nearly everyone in the room thought was unwise`{=html}](https://www.piratewires.com/p/senate-committee-ai-hearing-recap "'Even The Politicians Thought the Open Letter Made No Sense In The Senate Hearing on AI today’s hearing on ai covered ai regulation and challenges, and the infamous open letter, which nearly everyone in the room thought was unwise', Gorrell 2023"){.link-annotated .id-not .include-annotation}** ## `“DINOv2: Learning Robust Visual Features without Supervision”, Oquab et al 2023`{=html} {#oquab-et-al-2023-section} **[`DINOv2: Learning Robust Visual Features without Supervision`{=html}](https://arxiv.org/abs/2304.07193#facebook "'DINOv2: Learning Robust Visual Features without Supervision', Oquab et al 2023"){.link-annotated .id-not .include-annotation link-icon="facebook" link-icon-type="svg" link-icon-color="#1877f2"}** ## `“Segment Anything”, Kirillov et al 2023`{=html} {#kirillov-et-al-2023-section} **[`Segment Anything`{=html}](https://arxiv.org/abs/2304.02643#facebook "'Segment Anything', Kirillov et al 2023"){.link-annotated .id-not .include-annotation link-icon="facebook" link-icon-type="svg" link-icon-color="#1877f2"}** ## `“Humans in Humans Out: On GPT Converging Toward Common Sense in Both Success and Failure”, Koralus & Wang-Maścianica 2023`{=html} {#koralus-wang-maścianica-2023-section} **[`Humans in Humans Out: On GPT Converging Toward Common Sense in both Success and Failure`{=html}](https://arxiv.org/abs/2303.17276 "'Humans in Humans Out: On GPT Converging Toward Common Sense in both Success and Failure', Koralus & Wang-Maścianica 2023"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Sigmoid Loss for Language Image Pre-Training”, Zhai et al 2023`{=html} {#zhai-et-al-2023-section} **[`Sigmoid Loss for Language Image Pre-Training`{=html}](https://arxiv.org/abs/2303.15343#google "'Sigmoid Loss for Language Image Pre-Training', Zhai et al 2023"){.link-annotated .id-not .include-annotation link-icon="alphabet" link-icon-type="svg" link-icon-color="#4285f4"}** ## `“‘AI’ on a Calculator: Part 1 [MNIST CNN on a TI-84 Graphing Calculator]”, Mitchell 2023`{=html} {#mitchell-2023-section} **[`‘AI’ on a Calculator: Part 1 [MNIST CNN on a TI-84 graphing calculator]`{=html}](https://z80.me/blog/calculator-ai-part-1/ "'‘AI’ on a Calculator: Part 1 [MNIST CNN on a TI-84 graphing calculator]', Mitchell 2023"){.id-not .include-annotation}** ## `“How Well Do Large Language Models Perform in Arithmetic Tasks?”, Yuan et al 2023`{=html} {#yuan-et-al-2023-2-section} **[`How well do Large Language Models perform in Arithmetic tasks?`{=html}](https://arxiv.org/abs/2304.02015#alibaba "'How well do Large Language Models perform in Arithmetic tasks?', Yuan et al 2023"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“GPT-4 Technical Report”, OpenAI 2023`{=html} {#openai-2023-2-section} **[`GPT-4 Technical Report`{=html}](https://arxiv.org/abs/2303.08774#openai "'GPT-4 Technical Report', OpenAI 2023"){.link-annotated .id-not .include-annotation link-icon="openai" link-icon-type="svg"}** ## `“Securing Liberal Democratic Control of AGI through UK Leadership”, Phillips 2023`{=html} {#phillips-2023-section} **[`Securing Liberal Democratic Control of AGI through UK Leadership`{=html}](https://jameswphillips.substack.com/p/securing-liberal-democratic-control "'Securing Liberal Democratic Control of AGI through UK Leadership', Phillips 2023"){.link-annotated .id-not .include-annotation link-icon="substack" link-icon-type="svg" link-icon-color="#ff6719"}** ## `“GigaGAN: Scaling up GANs for Text-To-Image Synthesis”, Kang et al 2023`{=html} {#kang-et-al-2023-section} **[`GigaGAN: Scaling up GANs for Text-to-Image Synthesis`{=html}](https://arxiv.org/abs/2303.05511#adobe "'GigaGAN: Scaling up GANs for Text-to-Image Synthesis', Kang et al 2023"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Why Didn’t We Get GPT-2 in 2005?”, Dynomight 2023`{=html} {#dynomight-2023-section} **[`Why didn’t we get GPT-2 in 2005?`{=html}](https://dynomight.net/gpt-2/ "'Why didn’t we get GPT-2 in 2005?', Dynomight 2023"){.id-not .include-annotation link-icon="D" link-icon-type="text,mono"}** ::: aux-links-transclude-file ::: collapse **View External Link**: [`https://dynomight.net/gpt-2/`](https://dynomight.net/gpt-2/){.id-not .link-annotated-not .include-content .include-lazy link-icon="D" link-icon-type="text,mono"} ::: ::: ## `“Language Is Not All You Need: Aligning Perception With Language Models (Kosmos-1)”, Huang et al 2023`{=html} {#huang-et-al-2023-7-section} **[`Language Is Not All You Need: Aligning Perception with Language Models (Kosmos-1)`{=html}](https://arxiv.org/abs/2302.14045#microsoft "'Language Is Not All You Need: Aligning Perception with Language Models (Kosmos-1)', Huang et al 2023"){.link-annotated .id-not .include-annotation link-icon="MS" link-icon-type="text,sans,italic"}** ## `“Why Didn’t DeepMind Build GPT-3?”, Godwin 2023`{=html} {#godwin-2023-section} **[`Why didn’t DeepMind build GPT-3?`{=html}](https://rootnodes.substack.com/p/why-didnt-deepmind-build-gpt3 "'Why didn’t DeepMind build GPT-3?', Godwin 2023"){.link-annotated .id-not .include-annotation link-icon="deepmind" link-icon-type="svg" link-icon-color="#4185f4"}** ## `“Is Multimodal Vision Supervision Beneficial to Language?”, Madasu & Lal 2023`{=html} {#madasu-lal-2023-section} **[`Is Multimodal Vision Supervision Beneficial to Language?`{=html}](https://arxiv.org/abs/2302.05016 "'Is Multimodal Vision Supervision Beneficial to Language?', Madasu & Lal 2023"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Scaling Vision Transformers to 22 Billion Parameters”, Dehghani et al 2023`{=html} {#dehghani-et-al-2023-section} **[`Scaling Vision Transformers to 22 Billion Parameters`{=html}](https://arxiv.org/abs/2302.05442#google "'Scaling Vision Transformers to 22 Billion Parameters', Dehghani et al 2023"){.link-annotated .id-not .include-annotation link-icon="alphabet" link-icon-type="svg" link-icon-color="#4285f4"}** ## `“John Carmack’s ‘Different Path’ to Artificial General Intelligence”, Carmack 2023`{=html} {#carmack-2023-section} **[`John Carmack’s ‘Different Path’ to Artificial General Intelligence`{=html}](https://dallasinnovates.com/exclusive-qa-john-carmacks-different-path-to-artificial-general-intelligence/ "'John Carmack’s ‘Different Path’ to Artificial General Intelligence', Carmack 2023"){.link-annotated .id-not .include-annotation}** ## `“Large Language Models As Fiduciaries: A Case Study Toward Robustly Communicating With Artificial Intelligence Through Legal Standards”, Nay 2023`{=html} {#nay-2023-section} **[`Large Language Models as Fiduciaries: A Case Study Toward Robustly Communicating With Artificial Intelligence Through Legal Standards`{=html}](https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4335945 "'Large Language Models as Fiduciaries: A Case Study Toward Robustly Communicating With Artificial Intelligence Through Legal Standards', Nay 2023"){.link-annotated .id-not .include-annotation link-icon="SSRN" link-icon-type="text,quad" link-icon-color="#007398"}** ## `“ClimaX: A Foundation Model for Weather and Climate”, Nguyen et al 2023`{=html} {#nguyen-et-al-2023-2-section} **[`ClimaX: A foundation model for weather and climate`{=html}](https://arxiv.org/abs/2301.10343#microsoft "'ClimaX: A foundation model for weather and climate', Nguyen et al 2023"){.link-annotated .id-not .include-annotation link-icon="MS" link-icon-type="text,sans,italic"}** ## `“StyleGAN-T: Unlocking the Power of GANs for Fast Large-Scale Text-To-Image Synthesis”, Sauer et al 2023`{=html} {#sauer-et-al-2023-2-section} **[`StyleGAN-T: Unlocking the Power of GANs for Fast Large-Scale Text-to-Image Synthesis`{=html}](https://arxiv.org/abs/2301.09515#nvidia "'StyleGAN-T: Unlocking the Power of GANs for Fast Large-Scale Text-to-Image Synthesis', Sauer et al 2023"){.link-annotated .id-not .include-annotation link-icon="n" link-icon-type="text,sans,italic" link-icon-color="#77ba00"}** ## `“MUG: Vision Learners Meet Web Image-Text Pairs”, Zhao et al 2023`{=html} {#zhao-et-al-2023-6-section} **[`MUG: Vision Learners Meet Web Image-Text Pairs`{=html}](https://arxiv.org/abs/2301.07088#bytedance "'MUG: Vision Learners Meet Web Image-Text Pairs', Zhao et al 2023"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“GPT-3 As Knowledge Worker: A Zero-Shot Evaluation of AI CPA Capabilities”, Bommarito et al 2023`{=html} {#bommarito-et-al-2023-section} **[`GPT-3 as Knowledge Worker: A Zero-Shot Evaluation of AI CPA Capabilities`{=html}](https://arxiv.org/abs/2301.04408 "'GPT-3 as Knowledge Worker: A Zero-Shot Evaluation of AI CPA Capabilities', Bommarito et al 2023"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Scaling Laws for Generative Mixed-Modal Language Models”, Aghajanyan et al 2023`{=html} {#aghajanyan-et-al-2023-section} **[`Scaling Laws for Generative Mixed-Modal Language Models`{=html}](https://arxiv.org/abs/2301.03728#facebook "'Scaling Laws for Generative Mixed-Modal Language Models', Aghajanyan et al 2023"){.link-annotated .id-not .include-annotation link-icon="facebook" link-icon-type="svg" link-icon-color="#1877f2"}** ## `“VALL-E: Neural Codec Language Models Are Zero-Shot Text to Speech Synthesizers”, Wang et al 2023`{=html} {#wang-et-al-2023-18-section} **[`VALL-E: Neural Codec Language Models are Zero-Shot Text to Speech Synthesizers`{=html}](https://arxiv.org/abs/2301.02111#microsoft "'VALL-E: Neural Codec Language Models are Zero-Shot Text to Speech Synthesizers', Wang et al 2023"){.link-annotated .id-not .include-annotation link-icon="MS" link-icon-type="text,sans,italic"}** ## `“Large-Scale Weakly-Supervised Content Embeddings for Music Recommendation and Tagging”, Huang et al 2023`{=html} {#huang-et-al-2023-section} **[`Large-Scale Weakly-Supervised Content Embeddings for Music Recommendation and Tagging`{=html}](https://storage.googleapis.com/gweb-research2023-media/pubtools/5938.pdf "'Large-Scale Weakly-Supervised Content Embeddings for Music Recommendation and Tagging', Huang et al 2023"){.link-annotated .id-not .include-annotation link-icon="alphabet" link-icon-type="svg" link-icon-color="#4285f4"}** ## `“GPT-3 Takes the Bar Exam”, II & Katz 2022`{=html} {#ii-katz-2022-section} **[`GPT-3 Takes the Bar Exam`{=html}](https://arxiv.org/abs/2212.14402 "'GPT-3 Takes the Bar Exam', II & Katz 2022"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Cramming: Training a Language Model on a Single GPU in One Day”, Geiping & Goldstein 2022`{=html} {#geiping-goldstein-2022-section} **[`Cramming: Training a Language Model on a Single GPU in One Day`{=html}](https://arxiv.org/abs/2212.14034 "'Cramming: Training a Language Model on a Single GPU in One Day', Geiping & Goldstein 2022"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Evolutionary-Scale Prediction of Atomic Level Protein Structure With a Language Model”, Lin et al 2022`{=html} {#lin-et-al-2022-02-section} **[`Evolutionary-scale prediction of atomic level protein structure with a language model`{=html}](https://www.biorxiv.org/content/10.1101/2022.07.20.500902.full#facebook "'Evolutionary-scale prediction of atomic level protein structure with a language model', Lin et al 2022"){.link-annotated .id-not .include-annotation link-icon="facebook" link-icon-type="svg" link-icon-color="#1877f2"}** ## `“Discovering Language Model Behaviors With Model-Written Evaluations”, Perez et al 2022`{=html} {#perez-et-al-2022-1-section} **[`Discovering Language Model Behaviors with Model-Written Evaluations`{=html}](https://arxiv.org/abs/2212.09251#anthropic "'Discovering Language Model Behaviors with Model-Written Evaluations', Perez et al 2022"){.link-annotated .id-not .include-annotation link-icon="anthropic" link-icon-type="svg" link-icon-color="#d4a27f"}** ## `“One Embedder, Any Task: Instruction-Finetuned Text Embeddings (INSTRUCTOR)”, Su et al 2022`{=html} {#su-et-al-2022-1-section} **[`One Embedder, Any Task: Instruction-Finetuned Text Embeddings (INSTRUCTOR)`{=html}](https://arxiv.org/abs/2212.09741 "'One Embedder, Any Task: Instruction-Finetuned Text Embeddings (INSTRUCTOR)', Su et al 2022"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Reproducible Scaling Laws for Contrastive Language-Image Learning”, Cherti et al 2022`{=html} {#cherti-et-al-2022-section} **[`Reproducible scaling laws for contrastive language-image learning`{=html}](https://arxiv.org/abs/2212.07143 "'Reproducible scaling laws for contrastive language-image learning', Cherti et al 2022"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“ERNIE-Code: Beyond English-Centric Cross-Lingual Pretraining for Programming Languages”, Chai et al 2022`{=html} {#chai-et-al-2022-section} **[`ERNIE-Code: Beyond English-Centric Cross-lingual Pretraining for Programming Languages`{=html}](https://arxiv.org/abs/2212.06742#baidu "'ERNIE-Code: Beyond English-Centric Cross-lingual Pretraining for Programming Languages', Chai et al 2022"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“VideoCoCa: Video-Text Modeling With Zero-Shot Transfer from Contrastive Captioners”, Yan et al 2022`{=html} {#yan-et-al-2022-2-section} **[`VideoCoCa: Video-Text Modeling with Zero-Shot Transfer from Contrastive Captioners`{=html}](https://arxiv.org/abs/2212.04979#google "'VideoCoCa: Video-Text Modeling with Zero-Shot Transfer from Contrastive Captioners', Yan et al 2022"){.link-annotated .id-not .include-annotation link-icon="alphabet" link-icon-type="svg" link-icon-color="#4285f4"}** ## `“VindLU: A Recipe for Effective Video-And-Language Pretraining”, Cheng et al 2022`{=html} {#cheng-et-al-2022-1-section} **[`VindLU: A Recipe for Effective Video-and-Language Pretraining`{=html}](https://arxiv.org/abs/2212.05051 "'VindLU: A Recipe for Effective Video-and-Language Pretraining', Cheng et al 2022"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Whisper: Robust Speech Recognition via Large-Scale Weak Supervision”, Radford et al 2022`{=html} {#radford-et-al-2022-section} **[`Whisper: Robust Speech Recognition via Large-Scale Weak Supervision`{=html}](https://arxiv.org/abs/2212.04356#openai "'Whisper: Robust Speech Recognition via Large-Scale Weak Supervision', Radford et al 2022"){.link-annotated .id-not .include-annotation link-icon="openai" link-icon-type="svg"}** ## `“Scaling Language-Image Pre-Training via Masking”, Li et al 2022`{=html} {#li-et-al-2022-05-section} **[`Scaling Language-Image Pre-training via Masking`{=html}](https://arxiv.org/abs/2212.00794#facebook "'Scaling Language-Image Pre-training via Masking', Li et al 2022"){.link-annotated .id-not .include-annotation link-icon="facebook" link-icon-type="svg" link-icon-color="#1877f2"}** ## `“Galactica: A Large Language Model for Science”, Taylor et al 2022`{=html} {#taylor-et-al-2022-section} **[`Galactica: A Large Language Model for Science`{=html}](https://arxiv.org/abs/2211.09085#facebook "'Galactica: A Large Language Model for Science', Taylor et al 2022"){.link-annotated .id-not .include-annotation link-icon="facebook" link-icon-type="svg" link-icon-color="#1877f2"}** ## `“Large Language Models Struggle to Learn Long-Tail Knowledge”, Kandpal et al 2022`{=html} {#kandpal-et-al-2022-section} **[`Large Language Models Struggle to Learn Long-Tail Knowledge`{=html}](https://arxiv.org/abs/2211.08411 "'Large Language Models Struggle to Learn Long-Tail Knowledge', Kandpal et al 2022"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“EVA: Exploring the Limits of Masked Visual Representation Learning at Scale”, Fang et al 2022`{=html} {#fang-et-al-2022-1-section} **[`EVA: Exploring the Limits of Masked Visual Representation Learning at Scale`{=html}](https://arxiv.org/abs/2211.07636#baai "'EVA: Exploring the Limits of Masked Visual Representation Learning at Scale', Fang et al 2022"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“MMDialog: A Large-Scale Multi-Turn Dialogue Dataset Towards Multi-Modal Open-Domain Conversation”, Feng et al 2022`{=html} {#feng-et-al-2022-1-section} **[`MMDialog: A Large-scale Multi-turn Dialogue Dataset Towards Multi-modal Open-domain Conversation`{=html}](https://arxiv.org/abs/2211.05719#microsoft "'MMDialog: A Large-scale Multi-turn Dialogue Dataset Towards Multi-modal Open-domain Conversation', Feng et al 2022"){.link-annotated .id-not .include-annotation link-icon="MS" link-icon-type="text,sans,italic"}** ## `“Adversarial Policies Beat Superhuman Go AIs”, Wang et al 2022`{=html} {#wang-et-al-2022-08-section} **[`Adversarial Policies Beat Superhuman Go AIs`{=html}](https://arxiv.org/abs/2211.00241 "'Adversarial Policies Beat Superhuman Go AIs', Wang et al 2022"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Increments Podcast: #45—4 Central Fallacies of AI Research (With Melanie Mitchell)”, Mitchell & Chugg 2022`{=html} {#mitchell-chugg-2022-section} **[`Increments Podcast: #45—4 Central Fallacies of AI Research (with Melanie Mitchell)`{=html}](https://www.youtube.com/watch?v=Q-TJFyUoenc&t=2444s "'Increments Podcast: #45—4 Central Fallacies of AI Research (with Melanie Mitchell)', Mitchell & Chugg 2022"){.link-annotated .id-not .include-annotation link-icon="youtube" link-icon-type="svg" link-icon-color="#ff0033"}** ## `“A Solvable Model of Neural Scaling Laws”, Maloney et al 2022`{=html} {#maloney-et-al-2022-section} **[`A Solvable Model of Neural Scaling Laws`{=html}](https://arxiv.org/abs/2210.16859 "'A Solvable Model of Neural Scaling Laws', Maloney et al 2022"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Will We Run out of Data? An Analysis of the Limits of Scaling Datasets in Machine Learning”, Villalobos et al 2022`{=html} {#villalobos-et-al-2022-section} **[`Will we run out of data? An analysis of the limits of scaling datasets in Machine Learning`{=html}](https://arxiv.org/abs/2211.04325 "'Will we run out of data? An analysis of the limits of scaling datasets in Machine Learning', Villalobos et al 2022"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Evaluating Parameter Efficient Learning for Generation”, Xu et al 2022`{=html} {#xu-et-al-2022-2-section} **[`Evaluating Parameter Efficient Learning for Generation`{=html}](https://arxiv.org/abs/2210.13673#nvidia "'Evaluating Parameter Efficient Learning for Generation', Xu et al 2022"){.link-annotated .id-not .include-annotation link-icon="n" link-icon-type="text,sans,italic" link-icon-color="#77ba00"}** ## `“FLAN: Scaling Instruction-Finetuned Language Models”, Chung et al 2022`{=html} {#chung-et-al-2022-section} **[`FLAN: Scaling Instruction-Finetuned Language Models`{=html}](https://arxiv.org/abs/2210.11416#google "'FLAN: Scaling Instruction-Finetuned Language Models', Chung et al 2022"){.link-annotated .id-not .include-annotation link-icon="alphabet" link-icon-type="svg" link-icon-color="#4285f4"}** ## `“BioGPT: Generative Pre-Trained Transformer for Biomedical Text Generation and Mining”, Luo et al 2022`{=html} {#luo-et-al-2022-2-section} **[`BioGPT: Generative Pre-trained Transformer for Biomedical Text Generation and Mining`{=html}](https://arxiv.org/abs/2210.10341#microsoft "'BioGPT: Generative Pre-trained Transformer for Biomedical Text Generation and Mining', Luo et al 2022"){.link-annotated .id-not .include-annotation link-icon="MS" link-icon-type="text,sans,italic"}** ## `“Vision-Language Pre-Training: Basics, Recent Advances, and Future Trends”, Gan et al 2022`{=html} {#gan-et-al-2022-section} **[`Vision-Language Pre-training: Basics, Recent Advances, and Future Trends`{=html}](https://arxiv.org/abs/2210.09263 "'Vision-Language Pre-training: Basics, Recent Advances, and Future Trends', Gan et al 2022"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Foundation Transformers”, Wang et al 2022`{=html} {#wang-et-al-2022-10-section} **[`Foundation Transformers`{=html}](https://arxiv.org/abs/2210.06423#microsoft "'Foundation Transformers', Wang et al 2022"){.link-annotated .id-not .include-annotation link-icon="MS" link-icon-type="text,sans,italic"}** ## `“Self-Ask: Measuring and Narrowing the Compositionality Gap in Language Models (Bamboogle)”, Press et al 2022`{=html} {#press-et-al-2022-section} **[`Self-Ask: Measuring and Narrowing the Compositionality Gap in Language Models (Bamboogle)`{=html}](https://arxiv.org/abs/2210.03350#allen "'Self-Ask: Measuring and Narrowing the Compositionality Gap in Language Models (Bamboogle)', Press et al 2022"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“The Lie Derivative for Measuring Learned Equivariance”, Gruver et al 2022`{=html} {#gruver-et-al-2022-section} **[`The Lie Derivative for Measuring Learned Equivariance`{=html}](https://arxiv.org/abs/2210.02984 "'The Lie Derivative for Measuring Learned Equivariance', Gruver et al 2022"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“GLM-130B: An Open Bilingual Pre-Trained Model”, Zeng et al 2022`{=html} {#zeng-et-al-2022-1-section} **[`GLM-130B: An Open Bilingual Pre-trained Model`{=html}](https://arxiv.org/abs/2210.02414#zhipu "'GLM-130B: An Open Bilingual Pre-trained Model', Zeng et al 2022"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Ask Me Anything (AMA): A Simple Strategy for Prompting Language Models”, Arora et al 2022`{=html} {#arora-et-al-2022-2-section} **[`Ask Me Anything (AMA): A simple strategy for prompting language models`{=html}](https://arxiv.org/abs/2210.02441 "'Ask Me Anything (AMA): A simple strategy for prompting language models', Arora et al 2022"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Do Current Multi-Task Optimization Methods in Deep Learning Even Help?”, Xin et al 2022`{=html} {#xin-et-al-2022-section} **[`Do Current Multi-Task Optimization Methods in Deep Learning Even Help?`{=html}](https://arxiv.org/abs/2209.11379#google "'Do Current Multi-Task Optimization Methods in Deep Learning Even Help?', Xin et al 2022"){.link-annotated .id-not .include-annotation link-icon="alphabet" link-icon-type="svg" link-icon-color="#4285f4"}** ## `“Monolith: Real Time Recommendation System With Collisionless Embedding Table”, Liu et al 2022`{=html} {#liu-et-al-2022-12-section} **[`Monolith: Real Time Recommendation System With Collisionless Embedding Table`{=html}](https://arxiv.org/abs/2209.07663#bytedance "'Monolith: Real Time Recommendation System With Collisionless Embedding Table', Liu et al 2022"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Machine Reading, Fast and Slow: When Do Models "Understand" Language?”, Choudhury et al 2022`{=html} {#choudhury-et-al-2022-section} **[`Machine Reading, Fast and Slow: When Do Models "Understand" Language?`{=html}](https://arxiv.org/abs/2209.07430 "'Machine Reading, Fast and Slow: When Do Models "Understand" Language?', Choudhury et al 2022"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“PaLI: A Jointly-Scaled Multilingual Language-Image Model”, Chen et al 2022`{=html} {#chen-et-al-2022-pali-section} **[`PaLI: A Jointly-Scaled Multilingual Language-Image Model`{=html}](https://arxiv.org/abs/2209.06794#google "'PaLI: A Jointly-Scaled Multilingual Language-Image Model', Chen et al 2022"){.link-annotated .id-not .include-annotation link-icon="alphabet" link-icon-type="svg" link-icon-color="#4285f4"}** ## `“Using Large Language Models to Simulate Multiple Humans”, Aher et al 2022`{=html} {#aher-et-al-2022-section} **[`Using Large Language Models to Simulate Multiple Humans`{=html}](https://arxiv.org/abs/2208.10264#microsoft "'Using Large Language Models to Simulate Multiple Humans', Aher et al 2022"){.link-annotated .id-not .include-annotation link-icon="MS" link-icon-type="text,sans,italic"}** ## `“Understanding Scaling Laws for Recommendation Models”, Ardalani et al 2022`{=html} {#ardalani-et-al-2022-section} **[`Understanding Scaling Laws for Recommendation Models`{=html}](https://arxiv.org/abs/2208.08489 "'Understanding Scaling Laws for Recommendation Models', Ardalani et al 2022"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“LLM.int8(): 8-Bit Matrix Multiplication for Transformers at Scale”, Dettmers et al 2022`{=html} {#dettmers-et-al-2022-section} **[`LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scale`{=html}](https://arxiv.org/abs/2208.07339#facebook "'LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scale', Dettmers et al 2022"){.link-annotated .id-not .include-annotation link-icon="facebook" link-icon-type="svg" link-icon-color="#1877f2"}** ## `“Quality Not Quantity: On the Interaction between Dataset Design and Robustness of CLIP”, Nguyen et al 2022`{=html} {#nguyen-et-al-2022-section} **[`Quality Not Quantity: On the Interaction between Dataset Design and Robustness of CLIP`{=html}](https://arxiv.org/abs/2208.05516 "'Quality Not Quantity: On the Interaction between Dataset Design and Robustness of CLIP', Nguyen et al 2022"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Efficient Training of Language Models to Fill in the Middle”, Bavarian et al 2022`{=html} {#bavarian-et-al-2022-section} **[`Efficient Training of Language Models to Fill in the Middle`{=html}](https://arxiv.org/abs/2207.14255#openai "'Efficient Training of Language Models to Fill in the Middle', Bavarian et al 2022"){.link-annotated .id-not .include-annotation link-icon="openai" link-icon-type="svg"}** ## `“Why Do Tree-Based Models Still Outperform Deep Learning on Tabular Data?”, Grinsztajn et al 2022`{=html} {#grinsztajn-et-al-2022-section} **[`Why do tree-based models still outperform deep learning on tabular data?`{=html}](https://arxiv.org/abs/2207.08815 "'Why do tree-based models still outperform deep learning on tabular data?', Grinsztajn et al 2022"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“PIXEL: Language Modeling With Pixels”, Rust et al 2022`{=html} {#rust-et-al-2022-section} **[`PIXEL: Language Modeling with Pixels`{=html}](https://arxiv.org/abs/2207.06991 "'PIXEL: Language Modeling with Pixels', Rust et al 2022"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“High-Performing Neural Network Models of Visual Cortex Benefit from High Latent Dimensionality”, Elmoznino & Bonner 2022`{=html} {#elmoznino-bonner-2022-section} **[`High-performing neural network models of visual cortex benefit from high latent dimensionality`{=html}](https://www.biorxiv.org/content/10.1101/2022.07.13.499969.full "'High-performing neural network models of visual cortex benefit from high latent dimensionality', Elmoznino & Bonner 2022"){.link-annotated .id-not .include-annotation link-icon="chi-dna" link-icon-type="svg" link-icon-color="#bd2736"}** ## `“Exploring Length Generalization in Large Language Models”, Anil et al 2022`{=html} {#anil-et-al-2022-section} **[`Exploring Length Generalization in Large Language Models`{=html}](https://arxiv.org/abs/2207.04901#google "'Exploring Length Generalization in Large Language Models', Anil et al 2022"){.link-annotated .id-not .include-annotation link-icon="alphabet" link-icon-type="svg" link-icon-color="#4285f4"}** ## `“Language Models (Mostly) Know What They Know”, Kadavath et al 2022`{=html} {#kadavath-et-al-2022-section} **[`Language Models (Mostly) Know What They Know`{=html}](https://arxiv.org/abs/2207.05221#anthropic "'Language Models (Mostly) Know What They Know', Kadavath et al 2022"){.link-annotated .id-not .include-annotation link-icon="anthropic" link-icon-type="svg" link-icon-color="#d4a27f"}** ## `“On-Device Training Under 256KB Memory”, Lin et al 2022`{=html} {#lin-et-al-2022-smolml-section} **[`On-Device Training Under 256KB Memory`{=html}](https://arxiv.org/abs/2206.15472 "'On-Device Training Under 256KB Memory', Lin et al 2022"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Beyond Neural Scaling Laws: Beating Power Law Scaling via Data Pruning”, Sorscher et al 2022`{=html} {#sorscher-et-al-2022-section} **[`Beyond neural scaling laws: beating power law scaling via data pruning`{=html}](https://arxiv.org/abs/2206.14486 "'Beyond neural scaling laws: beating power law scaling via data pruning', Sorscher et al 2022"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“ProGen2: Exploring the Boundaries of Protein Language Models”, Nijkamp et al 2022`{=html} {#nijkamp-et-al-2022-1-section} **[`ProGen2: Exploring the Boundaries of Protein Language Models`{=html}](https://arxiv.org/abs/2206.13517#salesforce "'ProGen2: Exploring the Boundaries of Protein Language Models', Nijkamp et al 2022"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“RST: ReStructured Pre-Training”, Yuan & Liu 2022`{=html} {#yuan-liu-2022-section} **[`RST: reStructured Pre-training`{=html}](https://arxiv.org/abs/2206.11147 "'RST: reStructured Pre-training', Yuan & Liu 2022"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Limitations of the NTK for Understanding Generalization in Deep Learning”, Vyas et al 2022`{=html} {#vyas-et-al-2022-1-section} **[`Limitations of the NTK for Understanding Generalization in Deep Learning`{=html}](https://arxiv.org/abs/2206.10012 "'Limitations of the NTK for Understanding Generalization in Deep Learning', Vyas et al 2022"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Modeling Transformative AI Risks (MTAIR) Project—Summary Report”, Clarke et al 2022`{=html} {#clarke-et-al-2022-section} **[`Modeling Transformative AI Risks (MTAIR) Project—Summary Report`{=html}](https://arxiv.org/abs/2206.09360 "'Modeling Transformative AI Risks (MTAIR) Project—Summary Report', Clarke et al 2022"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“LIFT: Language-Interfaced Fine-Tuning for Non-Language Machine Learning Tasks”, Dinh et al 2022`{=html} {#dinh-et-al-2022-1-section} **[`LIFT: Language-Interfaced Fine-Tuning for Non-Language Machine Learning Tasks`{=html}](https://arxiv.org/abs/2206.06565 "'LIFT: Language-Interfaced Fine-Tuning for Non-Language Machine Learning Tasks', Dinh et al 2022"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“BigVGAN: A Universal Neural Vocoder With Large-Scale Training”, Lee et al 2022`{=html} {#lee-et-al-2022-07-section} **[`BigVGAN: A Universal Neural Vocoder with Large-Scale Training`{=html}](https://arxiv.org/abs/2206.04658#nvidia "'BigVGAN: A Universal Neural Vocoder with Large-Scale Training', Lee et al 2022"){.link-annotated .id-not .include-annotation link-icon="n" link-icon-type="text,sans,italic" link-icon-color="#77ba00"}** ## `“An Improved One Millisecond Mobile Backbone”, Vasu et al 2022`{=html} {#vasu-et-al-2022-section} **[`An Improved One millisecond Mobile Backbone`{=html}](https://arxiv.org/abs/2206.04040#apple "'An Improved One millisecond Mobile Backbone', Vasu et al 2022"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“A Neural Corpus Indexer for Document Retrieval”, Wang et al 2022`{=html} {#wang-et-al-2022-13-section} **[`A Neural Corpus Indexer for Document Retrieval`{=html}](https://arxiv.org/abs/2206.02743#microsoft "'A Neural Corpus Indexer for Document Retrieval', Wang et al 2022"){.link-annotated .id-not .include-annotation link-icon="MS" link-icon-type="text,sans,italic"}** ## `“Toward a Realistic Model of Speech Processing in the Brain With Self-Supervised Learning”, Millet et al 2022`{=html} {#millet-et-al-2022-section} **[`Toward a realistic model of speech processing in the brain with self-supervised learning`{=html}](https://arxiv.org/abs/2206.01685 "'Toward a realistic model of speech processing in the brain with self-supervised learning', Millet et al 2022"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Teaching Models to Express Their Uncertainty in Words”, Lin et al 2022`{=html} {#lin-et-al-2022-09-section} **[`Teaching Models to Express Their Uncertainty in Words`{=html}](https://arxiv.org/abs/2205.14334 "'Teaching Models to Express Their Uncertainty in Words', Lin et al 2022"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Why Robust Generalization in Deep Learning Is Difficult: Perspective of Expressive Power”, Li et al 2022`{=html} {#li-et-al-2022-17-section} **[`Why Robust Generalization in Deep Learning is Difficult: Perspective of Expressive Power`{=html}](https://arxiv.org/abs/2205.13863 "'Why Robust Generalization in Deep Learning is Difficult: Perspective of Expressive Power', Li et al 2022"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“M3AE: Multimodal Masked Autoencoders Learn Transferable Representations”, Geng et al 2022`{=html} {#geng-et-al-2022-section} **[`M3AE: Multimodal Masked Autoencoders Learn Transferable Representations`{=html}](https://arxiv.org/abs/2205.14204#google "'M3AE: Multimodal Masked Autoencoders Learn Transferable Representations', Geng et al 2022"){.link-annotated .id-not .include-annotation link-icon="alphabet" link-icon-type="svg" link-icon-color="#4285f4"}** ## `“InstructDial: Improving Zero and Few-Shot Generalization in Dialogue through Instruction Tuning”, Gupta et al 2022`{=html} {#gupta-et-al-2022-1-section} **[`InstructDial: Improving Zero and Few-shot Generalization in Dialogue through Instruction Tuning`{=html}](https://arxiv.org/abs/2205.12673 "'InstructDial: Improving Zero and Few-shot Generalization in Dialogue through Instruction Tuning', Gupta et al 2022"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Memorization Without Overfitting: Analyzing the Training Dynamics of Large Language Models”, Tirumala et al 2022`{=html} {#tirumala-et-al-2022-section} **[`Memorization Without Overfitting: Analyzing the Training Dynamics of Large Language Models`{=html}](https://arxiv.org/abs/2205.10770 "'Memorization Without Overfitting: Analyzing the Training Dynamics of Large Language Models', Tirumala et al 2022"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Least-To-Most Prompting Enables Complex Reasoning in Large Language Models”, Zhou et al 2022`{=html} {#zhou-et-al-2022-1-section} **[`Least-to-Most Prompting Enables Complex Reasoning in Large Language Models`{=html}](https://arxiv.org/abs/2205.10625#google "'Least-to-Most Prompting Enables Complex Reasoning in Large Language Models', Zhou et al 2022"){.link-annotated .id-not .include-annotation link-icon="alphabet" link-icon-type="svg" link-icon-color="#4285f4"}** ## `“Continual Pre-Training Mitigates Forgetting in Language and Vision”, Cossu et al 2022`{=html} {#cossu-et-al-2022-section} **[`Continual Pre-Training Mitigates Forgetting in Language and Vision`{=html}](https://arxiv.org/abs/2205.09357 "'Continual Pre-Training Mitigates Forgetting in Language and Vision', Cossu et al 2022"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Dialog Inpainting: Turning Documents into Dialogues”, Dai et al 2022`{=html} {#dai-et-al-2022-2-section} **[`Dialog Inpainting: Turning Documents into Dialogues`{=html}](https://arxiv.org/abs/2205.09073#google "'Dialog Inpainting: Turning Documents into Dialogues', Dai et al 2022"){.link-annotated .id-not .include-annotation link-icon="alphabet" link-icon-type="svg" link-icon-color="#4285f4"}** ## `“UL2: Unifying Language Learning Paradigms”, Tay et al 2022`{=html} {#tay-et-al-2022-ul2-section} **[`UL2: Unifying Language Learning Paradigms`{=html}](https://arxiv.org/abs/2205.05131#google "'UL2: Unifying Language Learning Paradigms', Tay et al 2022"){.link-annotated .id-not .include-annotation link-icon="alphabet" link-icon-type="svg" link-icon-color="#4285f4"}** ## `“Building Machine Translation Systems for the Next Thousand Languages”, Bapna et al 2022`{=html} {#bapna-et-al-2022-section} **[`Building Machine Translation Systems for the Next Thousand Languages`{=html}](https://arxiv.org/abs/2205.03983#google "'Building Machine Translation Systems for the Next Thousand Languages', Bapna et al 2022"){.link-annotated .id-not .include-annotation link-icon="alphabet" link-icon-type="svg" link-icon-color="#4285f4"}** ## `“When Does Dough Become a Bagel? Analyzing the Remaining Mistakes on ImageNet”, Vasudevan et al 2022`{=html} {#vasudevan-et-al-2022-section} **[`When does dough become a bagel? Analyzing the remaining mistakes on ImageNet`{=html}](https://arxiv.org/abs/2205.04596#google "'When does dough become a bagel? Analyzing the remaining mistakes on ImageNet', Vasudevan et al 2022"){.link-annotated .id-not .include-annotation link-icon="alphabet" link-icon-type="svg" link-icon-color="#4285f4"}** ## `“CoCa: Contrastive Captioners Are Image-Text Foundation Models”, Yu et al 2022`{=html} {#yu-et-al-2022-3-section} **[`CoCa: Contrastive Captioners are Image-Text Foundation Models`{=html}](https://arxiv.org/abs/2205.01917#google "'CoCa: Contrastive Captioners are Image-Text Foundation Models', Yu et al 2022"){.link-annotated .id-not .include-annotation link-icon="alphabet" link-icon-type="svg" link-icon-color="#4285f4"}** ## `“Data Determines Distributional Robustness in Contrastive Language Image Pre-Training (CLIP)”, Fang et al 2022`{=html} {#fang-et-al-2022-4-section} **[`Data Determines Distributional Robustness in Contrastive Language Image Pre-training (CLIP)`{=html}](https://arxiv.org/abs/2205.01397 "'Data Determines Distributional Robustness in Contrastive Language Image Pre-training (CLIP)', Fang et al 2022"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Continual Learning With Foundation Models: An Empirical Study of Latent Replay”, Ostapenko et al 2022`{=html} {#ostapenko-et-al-2022-section} **[`Continual Learning with Foundation Models: An Empirical Study of Latent Replay`{=html}](https://arxiv.org/abs/2205.00329 "'Continual Learning with Foundation Models: An Empirical Study of Latent Replay', Ostapenko et al 2022"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Flamingo: a Visual Language Model for Few-Shot Learning”, Alayrac et al 2022`{=html} {#alayrac-et-al-2022-section} **[`Flamingo: a Visual Language Model for Few-Shot Learning`{=html}](https://arxiv.org/abs/2204.14198#deepmind "'Flamingo: a Visual Language Model for Few-Shot Learning', Alayrac et al 2022"){.link-annotated .id-not .include-annotation link-icon="deepmind" link-icon-type="svg" link-icon-color="#4185f4"}** ## `“WebFace260M: A Benchmark for Million-Scale Deep Face Recognition”, Zhu et al 2022`{=html} {#zhu-et-al-2022-5-section} **[`WebFace260M: A Benchmark for Million-Scale Deep Face Recognition`{=html}](https://arxiv.org/abs/2204.10149 "'WebFace260M: A Benchmark for Million-Scale Deep Face Recognition', Zhu et al 2022"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Tk-Instruct: Benchmarking Generalization via In-Context Instructions on 1,600+ Language Tasks”, Wang et al 2022`{=html} {#wang-et-al-2022-16-section} **[`Tk-Instruct: Benchmarking Generalization via In-Context Instructions on 1,600+ Language Tasks`{=html}](https://arxiv.org/abs/2204.07705 "'Tk-Instruct: Benchmarking Generalization via In-Context Instructions on 1,600+ Language Tasks', Wang et al 2022"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“What Language Model Architecture and Pretraining Objective Work Best for Zero-Shot Generalization?”, Wang et al 2022`{=html} {#wang-et-al-2022-17-section} **[`What Language Model Architecture and Pretraining Objective Work Best for Zero-Shot Generalization?`{=html}](https://arxiv.org/abs/2204.05832 "'What Language Model Architecture and Pretraining Objective Work Best for Zero-Shot Generalization?', Wang et al 2022"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“DeepMind: The Podcast—Excerpts on AGI”, Kiely 2022`{=html} {#kiely-2022-section} **[`DeepMind: The Podcast—Excerpts on AGI`{=html}](https://www.lesswrong.com/posts/SbAgRYo8tkHwhd9Qx/deepmind-the-podcast-excerpts-on-agi "'DeepMind: The Podcast—Excerpts on AGI', Kiely 2022"){.link-annotated .id-not .include-annotation link-icon="deepmind" link-icon-type="svg" link-icon-color="#4185f4"}** ## `“Can Language Models Learn from Explanations in Context?”, Lampinen et al 2022`{=html} {#lampinen-et-al-2022-section} **[`Can language models learn from explanations in context?`{=html}](https://arxiv.org/abs/2204.02329#deepmind "'Can language models learn from explanations in context?', Lampinen et al 2022"){.link-annotated .id-not .include-annotation link-icon="deepmind" link-icon-type="svg" link-icon-color="#4185f4"}** ## `“Chinchilla: Training Compute-Optimal Large Language Models”, Hoffmann et al 2022`{=html} {#hoffmann-et-al-2022-section} **[`Chinchilla: Training Compute-Optimal Large Language Models`{=html}](https://arxiv.org/abs/2203.15556#deepmind "'Chinchilla: Training Compute-Optimal Large Language Models', Hoffmann et al 2022"){.link-annotated .id-not .include-annotation link-icon="deepmind" link-icon-type="svg" link-icon-color="#4185f4"}** ## `“A Roadmap for Big Model”, Yuan et al 2022`{=html} {#yuan-et-al-2022-2-section} **[`A Roadmap for Big Model`{=html}](https://arxiv.org/abs/2203.14101#baai "'A Roadmap for Big Model', Yuan et al 2022"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“A Conversational Paradigm for Program Synthesis”, Nijkamp et al 2022`{=html} {#nijkamp-et-al-2022-2-section} **[`A Conversational Paradigm for Program Synthesis`{=html}](https://arxiv.org/abs/2203.13474#salesforce "'A Conversational Paradigm for Program Synthesis', Nijkamp et al 2022"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Self-Consistency Improves Chain-Of-Thought Reasoning in Language Models”, Wang et al 2022`{=html} {#wang-et-al-2022-20-section} **[`Self-Consistency Improves Chain-of-Thought Reasoning in Language Models`{=html}](https://arxiv.org/abs/2203.11171#google "'Self-Consistency Improves Chain-of-Thought Reasoning in Language Models', Wang et al 2022"){.link-annotated .id-not .include-annotation link-icon="alphabet" link-icon-type="svg" link-icon-color="#4285f4"}** ## `“Effect of Scale on Catastrophic Forgetting in Neural Networks”, Ramasesh et al 2022`{=html} {#ramasesh-et-al-2022-section} **[`Effect of scale on catastrophic forgetting in neural networks`{=html}](https://openreview.net/forum?id=GhVS8_yPeEa#google "'Effect of scale on catastrophic forgetting in neural networks', Ramasesh et al 2022"){.link-annotated .id-not .include-annotation link-icon="alphabet" link-icon-type="svg" link-icon-color="#4285f4"}** ## `“Tensor Programs V: Tuning Large Neural Networks via Zero-Shot Hyperparameter Transfer”, Yang et al 2022`{=html} {#yang-et-al-2022-8-section} **[`Tensor Programs V: Tuning Large Neural Networks via Zero-Shot Hyperparameter Transfer`{=html}](https://arxiv.org/abs/2203.03466#microsoft "'Tensor Programs V: Tuning Large Neural Networks via Zero-Shot Hyperparameter Transfer', Yang et al 2022"){.link-annotated .id-not .include-annotation link-icon="MS" link-icon-type="text,sans,italic"}** ## `“FastFold: Reducing AlphaFold Training Time from 11 Days to 67 Hours”, Cheng et al 2022`{=html} {#cheng-et-al-2022-2-section} **[`FastFold: Reducing AlphaFold Training Time from 11 Days to 67 Hours`{=html}](https://arxiv.org/abs/2203.00854 "'FastFold: Reducing AlphaFold Training Time from 11 Days to 67 Hours', Cheng et al 2022"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Variational Autoencoders Without the Variation”, Daly et al 2022`{=html} {#daly-et-al-2022-section} **[`Variational Autoencoders Without the Variation`{=html}](https://arxiv.org/abs/2203.00645 "'Variational Autoencoders Without the Variation', Daly et al 2022"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Performance Reserves in Brain-Imaging-Based Phenotype Prediction”, Schulz et al 2022`{=html} {#schulz-et-al-2022-section} **[`Performance reserves in brain-imaging-based phenotype prediction`{=html}](https://www.biorxiv.org/content/10.1101/2022.02.23.481601.full "'Performance reserves in brain-imaging-based phenotype prediction', Schulz et al 2022"){.link-annotated .id-not .include-annotation link-icon="chi-dna" link-icon-type="svg" link-icon-color="#bd2736"}** ## `“Self-Distilled StyleGAN: Towards Generation from Internet Photos”, Mokady et al 2022`{=html} {#mokady-et-al-2022-2-section} **[`Self-Distilled StyleGAN: Towards Generation from Internet Photos`{=html}](https://arxiv.org/abs/2202.12211#google "'Self-Distilled StyleGAN: Towards Generation from Internet Photos', Mokady et al 2022"){.link-annotated .id-not .include-annotation link-icon="alphabet" link-icon-type="svg" link-icon-color="#4285f4"}** ## `“UnifiedQA-V2: Stronger Generalization via Broader Cross-Format Training”, Khashabi et al 2022`{=html} {#khashabi-et-al-2022-section} **[`UnifiedQA-v2: Stronger Generalization via Broader Cross-Format Training`{=html}](https://arxiv.org/abs/2202.12359 "'UnifiedQA-v2: Stronger Generalization via Broader Cross-Format Training', Khashabi et al 2022"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Vision Models Are More Robust And Fair When Pretrained On Uncurated Images Without Supervision”, Goyal et al 2022`{=html} {#goyal-et-al-2022-2-section} **[`Vision Models Are More Robust And Fair When Pretrained On Uncurated Images Without Supervision`{=html}](https://arxiv.org/abs/2202.08360#facebook "'Vision Models Are More Robust And Fair When Pretrained On Uncurated Images Without Supervision', Goyal et al 2022"){.link-annotated .id-not .include-annotation link-icon="facebook" link-icon-type="svg" link-icon-color="#1877f2"}** ## `“Brains and Algorithms Partially Converge in Natural Language Processing”, Caucheteux & King 2022`{=html} {#caucheteux-king-2022-section} **[`Brains and algorithms partially converge in natural language processing`{=html}](https://www.nature.com/articles/s42003-022-03036-1 "'Brains and algorithms partially converge in natural language processing', Caucheteux & King 2022"){.link-annotated .id-not .include-annotation link-icon="n" link-icon-type="text"}** ## `“Quantifying Memorization Across Neural Language Models”, Carlini et al 2022`{=html} {#carlini-et-al-2022-section} **[`Quantifying Memorization Across Neural Language Models`{=html}](https://arxiv.org/abs/2202.07646 "'Quantifying Memorization Across Neural Language Models', Carlini et al 2022"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Wukong: 100 Million Large-Scale Chinese Cross-Modal Pre-Training Dataset and A Foundation Framework”, Gu et al 2022`{=html} {#gu-et-al-2022-2-section} **[`Wukong: 100 Million Large-scale Chinese Cross-modal Pre-training Dataset and A Foundation Framework`{=html}](https://arxiv.org/abs/2202.06767#huawei "'Wukong: 100 Million Large-scale Chinese Cross-modal Pre-training Dataset and A Foundation Framework', Gu et al 2022"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“OFA: Unifying Architectures, Tasks, and Modalities Through a Simple Sequence-To-Sequence Learning Framework”, Wang et al 2022`{=html} {#wang-et-al-2022-21-section} **[`OFA: Unifying Architectures, Tasks, and Modalities Through a Simple Sequence-to-Sequence Learning Framework`{=html}](https://arxiv.org/abs/2202.03052#alibaba "'OFA: Unifying Architectures, Tasks, and Modalities Through a Simple Sequence-to-Sequence Learning Framework', Wang et al 2022"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Data Scaling Laws in NMT: The Effect of Noise and Architecture”, Bansal et al 2022`{=html} {#bansal-et-al-2022-nmtscaling-section} **[`Data Scaling Laws in NMT: The Effect of Noise and Architecture`{=html}](https://arxiv.org/abs/2202.01994#google "'Data Scaling Laws in NMT: The Effect of Noise and Architecture', Bansal et al 2022"){.link-annotated .id-not .include-annotation link-icon="alphabet" link-icon-type="svg" link-icon-color="#4285f4"}** ## `“Webly Supervised Concept Expansion for General Purpose Vision Models”, Kamath et al 2022`{=html} {#kamath-et-al-2022-section} **[`Webly Supervised Concept Expansion for General Purpose Vision Models`{=html}](https://arxiv.org/abs/2202.02317#allen "'Webly Supervised Concept Expansion for General Purpose Vision Models', Kamath et al 2022"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“StyleGAN-XL: Scaling StyleGAN to Large Diverse Datasets”, Sauer et al 2022`{=html} {#sauer-et-al-2022-section} **[`StyleGAN-XL: Scaling StyleGAN to Large Diverse Datasets`{=html}](https://arxiv.org/abs/2202.00273 "'StyleGAN-XL: Scaling StyleGAN to Large Diverse Datasets', Sauer et al 2022"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Using DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, A Large-Scale Generative Language Model”, Smith et al 2022`{=html} {#smith-et-al-2022-3-section} **[`Using DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, A Large-Scale Generative Language Model`{=html}](https://arxiv.org/abs/2201.11990#microsoftnvidia "'Using DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, A Large-Scale Generative Language Model', Smith et al 2022"){.link-annotated .id-not .include-annotation link-icon="n" link-icon-type="text,sans,italic" link-icon-color="#77ba00"}** ## `“Reasoning Like Program Executors”, Pi et al 2022`{=html} {#pi-et-al-2022-section} **[`Reasoning Like Program Executors`{=html}](https://arxiv.org/abs/2201.11473#microsoft "'Reasoning Like Program Executors', Pi et al 2022"){.link-annotated .id-not .include-annotation link-icon="MS" link-icon-type="text,sans,italic"}** ## `“Text and Code Embeddings by Contrastive Pre-Training”, Neelakantan et al 2022`{=html} {#neelakantan-et-al-2022-section} **[`Text and Code Embeddings by Contrastive Pre-Training`{=html}](https://arxiv.org/abs/2201.10005#openai "'Text and Code Embeddings by Contrastive Pre-Training', Neelakantan et al 2022"){.link-annotated .id-not .include-annotation link-icon="openai" link-icon-type="svg"}** ## `“LaMDA: Language Models for Dialog Applications”, Thoppilan et al 2022`{=html} {#thoppilan-et-al-2022-section} **[`LaMDA: Language Models for Dialog Applications`{=html}](https://arxiv.org/abs/2201.08239#google "'LaMDA: Language Models for Dialog Applications', Thoppilan et al 2022"){.link-annotated .id-not .include-annotation link-icon="alphabet" link-icon-type="svg" link-icon-color="#4285f4"}** ## `“SWAG: Revisiting Weakly Supervised Pre-Training of Visual Perception Models”, Singh et al 2022`{=html} {#singh-et-al-2022-section} **[`SWAG: Revisiting Weakly Supervised Pre-Training of Visual Perception Models`{=html}](https://arxiv.org/abs/2201.08371#facebook "'SWAG: Revisiting Weakly Supervised Pre-Training of Visual Perception Models', Singh et al 2022"){.link-annotated .id-not .include-annotation link-icon="facebook" link-icon-type="svg" link-icon-color="#1877f2"}** ## `“CM3: A Causal Masked Multimodal Model of the Internet”, Aghajanyan et al 2022`{=html} {#aghajanyan-et-al-2022-section} **[`CM3: A Causal Masked Multimodal Model of the Internet`{=html}](https://arxiv.org/abs/2201.07520#facebook "'CM3: A Causal Masked Multimodal Model of the Internet', Aghajanyan et al 2022"){.link-annotated .id-not .include-annotation link-icon="facebook" link-icon-type="svg" link-icon-color="#1877f2"}** ## `“ZeroPrompt: Scaling Prompt-Based Pretraining to 1,000 Tasks Improves Zero-Shot Generalization”, Xu et al 2022`{=html} {#xu-et-al-2022-7-section} **[`ZeroPrompt: Scaling Prompt-Based Pretraining to 1,000 Tasks Improves Zero-Shot Generalization`{=html}](https://arxiv.org/abs/2201.06910 "'ZeroPrompt: Scaling Prompt-Based Pretraining to 1,000 Tasks Improves Zero-Shot Generalization', Xu et al 2022"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“A High-Dimensional Sphere Spilling out of a High-Dimensional Cube despite Exponentially Many Constraints”, Fort 2022`{=html} {#fort-2022-section} **[`A high-dimensional sphere spilling out of a high-dimensional cube despite exponentially many constraints`{=html}](https://stanislavfort.com/blog/sphere-spilling-out/ "'A high-dimensional sphere spilling out of a high-dimensional cube despite exponentially many constraints', Fort 2022"){.link-annotated-partial .id-not .include-annotation}** ## `“ConvNeXt: A ConvNet for the 2020s”, Liu et al 2022`{=html} {#liu-et-al-2022-23-section} **[`ConvNeXt: A ConvNet for the 2020s`{=html}](https://arxiv.org/abs/2201.03545#facebook "'ConvNeXt: A ConvNet for the 2020s', Liu et al 2022"){.link-annotated .id-not .include-annotation link-icon="facebook" link-icon-type="svg" link-icon-color="#1877f2"}** ## `“The Defeat of the Winograd Schema Challenge”, Kocijan et al 2022`{=html} {#kocijan-et-al-2022-section} **[`The Defeat of the Winograd Schema Challenge`{=html}](https://arxiv.org/abs/2201.02387 "'The Defeat of the Winograd Schema Challenge', Kocijan et al 2022"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Robust Self-Supervised Audio-Visual Speech Recognition”, Shi et al 2022`{=html} {#shi-et-al-2022-avhubert-section} **[`Robust Self-Supervised Audio-Visual Speech Recognition`{=html}](https://arxiv.org/abs/2201.01763#facebook "'Robust Self-Supervised Audio-Visual Speech Recognition', Shi et al 2022"){.link-annotated .id-not .include-annotation link-icon="facebook" link-icon-type="svg" link-icon-color="#1877f2"}** ## `“AV-HuBERT: Learning Audio-Visual Speech Representation by Masked Multimodal Cluster Prediction”, Shi et al 2022`{=html} {#shi-et-al-2022-5-section} **[`AV-HuBERT: Learning Audio-Visual Speech Representation by Masked Multimodal Cluster Prediction`{=html}](https://arxiv.org/abs/2201.02184#facebook "'AV-HuBERT: Learning Audio-Visual Speech Representation by Masked Multimodal Cluster Prediction', Shi et al 2022"){.link-annotated .id-not .include-annotation link-icon="facebook" link-icon-type="svg" link-icon-color="#1877f2"}** ## `“Self-Supervised Learning from 100 Million Medical Images”, Ghesu et al 2022`{=html} {#ghesu-et-al-2022-section} **[`Self-supervised Learning from 100 Million Medical Images`{=html}](https://arxiv.org/abs/2201.01283 "'Self-supervised Learning from 100 Million Medical Images', Ghesu et al 2022"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“The Evolution of Quantitative Sensitivity”, Bryer et al 2021`{=html} {#bryer-et-al-2021-section} **[`The evolution of quantitative sensitivity`{=html}](https://royalsocietypublishing.org/doi/10.1098/rstb.2020.0529 "'The evolution of quantitative sensitivity', Bryer et al 2021"){.link-annotated .id-not .include-annotation link-icon="RS" link-icon-type="text" link-icon-color="#d31245"}** ## `“ERNIE 3.0 Titan: Exploring Larger-Scale Knowledge Enhanced Pre-Training for Language Understanding and Generation”, Wang et al 2021`{=html} {#wang-et-al-2021-ernie30titan-section} **[`ERNIE 3.0 Titan: Exploring Larger-scale Knowledge Enhanced Pre-training for Language Understanding and Generation`{=html}](https://arxiv.org/abs/2112.12731#baidu "'ERNIE 3.0 Titan: Exploring Larger-scale Knowledge Enhanced Pre-training for Language Understanding and Generation', Wang et al 2021"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“XGLM: Few-Shot Learning With Multilingual Language Models”, Lin et al 2021`{=html} {#lin-et-al-2021-2-section} **[`XGLM: Few-shot Learning with Multilingual Language Models`{=html}](https://arxiv.org/abs/2112.10668#facebook "'XGLM: Few-shot Learning with Multilingual Language Models', Lin et al 2021"){.link-annotated .id-not .include-annotation link-icon="facebook" link-icon-type="svg" link-icon-color="#1877f2"}** ## `“An Empirical Investigation of the Role of Pre-Training in Lifelong Learning”, Mehta et al 2021`{=html} {#mehta-et-al-2021-1-section} **[`An Empirical Investigation of the Role of Pre-training in Lifelong Learning`{=html}](https://arxiv.org/abs/2112.09153 "'An Empirical Investigation of the Role of Pre-training in Lifelong Learning', Mehta et al 2021"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Few-Shot Instruction Prompts for Pretrained Language Models to Detect Social Biases”, Prabhumoye et al 2021`{=html} {#prabhumoye-et-al-2021-section} **[`Few-shot Instruction Prompts for Pretrained Language Models to Detect Social Biases`{=html}](https://arxiv.org/abs/2112.07868#nvidia "'Few-shot Instruction Prompts for Pretrained Language Models to Detect Social Biases', Prabhumoye et al 2021"){.link-annotated .id-not .include-annotation link-icon="n" link-icon-type="text,sans,italic" link-icon-color="#77ba00"}** ## `“Knowledge-Rich Self-Supervised Entity Linking”, Zhang et al 2021`{=html} {#zhang-et-al-2021-krissbert-section} **[`Knowledge-Rich Self-Supervised Entity Linking`{=html}](https://arxiv.org/abs/2112.07887#microsoft "'Knowledge-Rich Self-Supervised Entity Linking', Zhang et al 2021"){.link-annotated .id-not .include-annotation link-icon="MS" link-icon-type="text,sans,italic"}** ## `“You Only Need One Model for Open-Domain Question Answering”, Lee et al 2021`{=html} {#lee-et-al-2021-3-section} **[`You Only Need One Model for Open-domain Question Answering`{=html}](https://arxiv.org/abs/2112.07381#samsung "'You Only Need One Model for Open-domain Question Answering', Lee et al 2021"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“EBERT: Epigenomic Language Models Powered by Cerebras”, Trotter et al 2021`{=html} {#trotter-et-al-2021-section} **[`EBERT: Epigenomic language models powered by Cerebras`{=html}](https://arxiv.org/abs/2112.07571 "'EBERT: Epigenomic language models powered by Cerebras', Trotter et al 2021"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“MAGMA—Multimodal Augmentation of Generative Models through Adapter-Based Finetuning”, Eichenberg et al 2021`{=html} {#eichenberg-et-al-2021-section} **[`MAGMA—Multimodal Augmentation of Generative Models through Adapter-based Finetuning`{=html}](https://arxiv.org/abs/2112.05253 "'MAGMA—Multimodal Augmentation of Generative Models through Adapter-based Finetuning', Eichenberg et al 2021"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Improving Language Models by Retrieving from Trillions of Tokens”, Borgeaud et al 2021`{=html} {#borgeaud-et-al-2021-section} **[`Improving language models by retrieving from trillions of tokens`{=html}](https://arxiv.org/abs/2112.04426#deepmind "'Improving language models by retrieving from trillions of tokens', Borgeaud et al 2021"){.link-annotated .id-not .include-annotation link-icon="deepmind" link-icon-type="svg" link-icon-color="#4185f4"}** ## `“MLP Architectures for Vision-And-Language Modeling: An Empirical Study”, Nie et al 2021`{=html} {#nie-et-al-2021-section} **[`MLP Architectures for Vision-and-Language Modeling: An Empirical Study`{=html}](https://arxiv.org/abs/2112.04453 "'MLP Architectures for Vision-and-Language Modeling: An Empirical Study', Nie et al 2021"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“LEMON: Scaling Up Vision-Language Pre-Training for Image Captioning”, Hu et al 2021`{=html} {#hu-et-al-2021-2-section} **[`LEMON: Scaling Up Vision-Language Pre-training for Image Captioning`{=html}](https://arxiv.org/abs/2111.12233#microsoft "'LEMON: Scaling Up Vision-Language Pre-training for Image Captioning', Hu et al 2021"){.link-annotated .id-not .include-annotation link-icon="MS" link-icon-type="text,sans,italic"}** ## `“Scaling Transformers: Sparse Is Enough in Scaling Transformers”, Jaszczur et al 2021`{=html} {#jaszczur-et-al-2021-section} **[`Scaling Transformers: Sparse is Enough in Scaling Transformers`{=html}](https://arxiv.org/abs/2111.12763#google "'Scaling Transformers: Sparse is Enough in Scaling Transformers', Jaszczur et al 2021"){.link-annotated .id-not .include-annotation link-icon="alphabet" link-icon-type="svg" link-icon-color="#4285f4"}** ## `“Can Pre-Trained Language Models Be Used to Resolve Textual and Semantic Merge Conflicts?”, Zhang et al 2021`{=html} {#zhang-et-al-2021-04-section} **[`Can Pre-trained Language Models be Used to Resolve Textual and Semantic Merge Conflicts?`{=html}](https://arxiv.org/abs/2111.11904#microsoft "'Can Pre-trained Language Models be Used to Resolve Textual and Semantic Merge Conflicts?', Zhang et al 2021"){.link-annotated .id-not .include-annotation link-icon="MS" link-icon-type="text,sans,italic"}** ## `“ExT5: Towards Extreme Multi-Task Scaling for Transfer Learning”, Aribandi et al 2021`{=html} {#aribandi-et-al-2021-section} **[`ExT5: Towards Extreme Multi-Task Scaling for Transfer Learning`{=html}](https://arxiv.org/abs/2111.10952#google "'ExT5: Towards Extreme Multi-Task Scaling for Transfer Learning', Aribandi et al 2021"){.link-annotated .id-not .include-annotation link-icon="alphabet" link-icon-type="svg" link-icon-color="#4285f4"}** ## `“L-Verse: Bidirectional Generation Between Image and Text”, Kim et al 2021`{=html} {#kim-et-al-2021-3-section} **[`L-Verse: Bidirectional Generation Between Image and Text`{=html}](https://arxiv.org/abs/2111.11133 "'L-Verse: Bidirectional Generation Between Image and Text', Kim et al 2021"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“RedCaps: Web-Curated Image-Text Data Created by the People, for the People”, Desai et al 2021`{=html} {#desai-et-al-2021-section} **[`RedCaps: web-curated image-text data created by the people, for the people`{=html}](https://arxiv.org/abs/2111.11431 "'RedCaps: web-curated image-text data created by the people, for the people', Desai et al 2021"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Florence: A New Foundation Model for Computer Vision”, Yuan et al 2021`{=html} {#yuan-et-al-2021-1-section} **[`Florence: A New Foundation Model for Computer Vision`{=html}](https://arxiv.org/abs/2111.11432#microsoft "'Florence: A New Foundation Model for Computer Vision', Yuan et al 2021"){.link-annotated .id-not .include-annotation link-icon="MS" link-icon-type="text,sans,italic"}** ## `“BASIC: Combined Scaling for Open-Vocabulary Image Classification”, Pham et al 2021`{=html} {#pham-et-al-2021-1-section} **[`BASIC: Combined Scaling for Open-Vocabulary Image Classification`{=html}](https://arxiv.org/abs/2111.10050#google "'BASIC: Combined Scaling for Open-Vocabulary Image Classification', Pham et al 2021"){.link-annotated .id-not .include-annotation link-icon="alphabet" link-icon-type="svg" link-icon-color="#4285f4"}** ## `“Swin Transformer V2: Scaling Up Capacity and Resolution”, Liu et al 2021`{=html} {#liu-et-al-2021-swintranformerv2-section} **[`Swin Transformer V2: Scaling Up Capacity and Resolution`{=html}](https://arxiv.org/abs/2111.09883 "'Swin Transformer V2: Scaling Up Capacity and Resolution', Liu et al 2021"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“XLS-R: Self-Supervised Cross-Lingual Speech Representation Learning at Scale”, Babu et al 2021`{=html} {#babu-et-al-2021-section} **[`XLS-R: Self-supervised Cross-lingual Speech Representation Learning at Scale`{=html}](https://arxiv.org/abs/2111.09296#facebook "'XLS-R: Self-supervised Cross-lingual Speech Representation Learning at Scale', Babu et al 2021"){.link-annotated .id-not .include-annotation link-icon="facebook" link-icon-type="svg" link-icon-color="#1877f2"}** ## `“Solving Linear Algebra by Program Synthesis”, Drori & Verma 2021`{=html} {#drori-verma-2021-section} **[`Solving Linear Algebra by Program Synthesis`{=html}](https://arxiv.org/abs/2111.08171 "'Solving Linear Algebra by Program Synthesis', Drori & Verma 2021"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Covariate Shift in High-Dimensional Random Feature Regression”, Tripuraneni et al 2021`{=html} {#tripuraneni-et-al-2021-section} **[`Covariate Shift in High-Dimensional Random Feature Regression`{=html}](https://arxiv.org/abs/2111.08234 "'Covariate Shift in High-Dimensional Random Feature Regression', Tripuraneni et al 2021"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Solving Probability and Statistics Problems by Program Synthesis”, Tang et al 2021`{=html} {#tang-et-al-2021-1-section} **[`Solving Probability and Statistics Problems by Program Synthesis`{=html}](https://arxiv.org/abs/2111.08267 "'Solving Probability and Statistics Problems by Program Synthesis', Tang et al 2021"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Few-Shot Self-Rationalization With Natural Language Prompts”, Marasović et al 2021`{=html} {#marasović-et-al-2021-section} **[`Few-Shot Self-Rationalization with Natural Language Prompts`{=html}](https://arxiv.org/abs/2111.08284#allen "'Few-Shot Self-Rationalization with Natural Language Prompts', Marasović et al 2021"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“INTERN: A New Learning Paradigm Towards General Vision”, Shao et al 2021`{=html} {#shao-et-al-2021-2-section} **[`INTERN: A New Learning Paradigm Towards General Vision`{=html}](https://arxiv.org/abs/2111.08687#sensetime "'INTERN: A New Learning Paradigm Towards General Vision', Shao et al 2021"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Scaling Law for Recommendation Models: Towards General-Purpose User Representations”, Shin et al 2021`{=html} {#shin-et-al-2021-1-section} **[`Scaling Law for Recommendation Models: Towards General-purpose User Representations`{=html}](https://arxiv.org/abs/2111.11294 "'Scaling Law for Recommendation Models: Towards General-purpose User Representations', Shin et al 2021"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“MAE: Masked Autoencoders Are Scalable Vision Learners”, He et al 2021`{=html} {#he-et-al-2021-1-section} **[`MAE: Masked Autoencoders Are Scalable Vision Learners`{=html}](https://arxiv.org/abs/2111.06377#facebook "'MAE: Masked Autoencoders Are Scalable Vision Learners', He et al 2021"){.link-annotated .id-not .include-annotation link-icon="facebook" link-icon-type="svg" link-icon-color="#1877f2"}** ## `“Persia: An Open, Hybrid System Scaling Deep Learning-Based Recommenders up to 100 Trillion Parameters”, Lian et al 2021`{=html} {#lian-et-al-2021-1-section} **[`Persia: An Open, Hybrid System Scaling Deep Learning-based Recommenders up to 100 Trillion Parameters`{=html}](https://arxiv.org/abs/2111.05897 "'Persia: An Open, Hybrid System Scaling Deep Learning-based Recommenders up to 100 Trillion Parameters', Lian et al 2021"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Scaling ASR Improves Zero and Few Shot Learning”, Xiao et al 2021`{=html} {#xiao-et-al-2021-2-section} **[`Scaling ASR Improves Zero and Few Shot Learning`{=html}](https://arxiv.org/abs/2111.05948#facebook "'Scaling ASR Improves Zero and Few Shot Learning', Xiao et al 2021"){.link-annotated .id-not .include-annotation link-icon="facebook" link-icon-type="svg" link-icon-color="#1877f2"}** ## `“Turing-Universal Learners With Optimal Scaling Laws”, Nakkiran 2021`{=html} {#nakkiran-2021-section} **[`Turing-Universal Learners with Optimal Scaling Laws`{=html}](https://arxiv.org/abs/2111.05321 "'Turing-Universal Learners with Optimal Scaling Laws', Nakkiran 2021"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“LAION-400M: Open Dataset of CLIP-Filtered 400 Million Image-Text Pairs”, Schuhmann et al 2021`{=html} {#schuhmann-et-al-2021-section} **[`LAION-400M: Open Dataset of CLIP-Filtered 400 Million Image-Text Pairs`{=html}](https://arxiv.org/abs/2111.02114#laion "'LAION-400M: Open Dataset of CLIP-Filtered 400 Million Image-Text Pairs', Schuhmann et al 2021"){.link-annotated .id-not .include-annotation link-icon="laion" link-icon-type="svg" link-icon-color="#1d374e"}** ## `“Training Verifiers to Solve Math Word Problems”, Cobbe et al 2021`{=html} {#cobbe-et-al-2021-section} **[`Training Verifiers to Solve Math Word Problems`{=html}](https://arxiv.org/abs/2110.14168#openai "'Training Verifiers to Solve Math Word Problems', Cobbe et al 2021"){.link-annotated .id-not .include-annotation link-icon="openai" link-icon-type="svg"}** ## `“Just Ask for Generalization”, Jang 2021`{=html} {#jang-2021-section} **[`Just Ask for Generalization`{=html}](https://evjang.com/2021/10/23/generalization.html "'Just Ask for Generalization', Jang 2021"){.id-not .include-annotation}** ## `“Wide Neural Networks Forget Less Catastrophically”, Mirzadeh et al 2021`{=html} {#mirzadeh-et-al-2021-section} **[`Wide Neural Networks Forget Less Catastrophically`{=html}](https://arxiv.org/abs/2110.11526#deepmind "'Wide Neural Networks Forget Less Catastrophically', Mirzadeh et al 2021"){.link-annotated .id-not .include-annotation link-icon="deepmind" link-icon-type="svg" link-icon-color="#4185f4"}** ## `“No One Representation to Rule Them All: Overlapping Features of Training Methods”, Gontijo-Lopes et al 2021`{=html} {#gontijo-lopes-et-al-2021-section} **[`No One Representation to Rule Them All: Overlapping Features of Training Methods`{=html}](https://arxiv.org/abs/2110.12899#google "'No One Representation to Rule Them All: Overlapping Features of Training Methods', Gontijo-Lopes et al 2021"){.link-annotated .id-not .include-annotation link-icon="alphabet" link-icon-type="svg" link-icon-color="#4285f4"}** ## `“When in Doubt, Summon the Titans: Efficient Inference With Large Models”, Rawat et al 2021`{=html} {#rawat-et-al-2021-section} **[`When in Doubt, Summon the Titans: Efficient Inference with Large Models`{=html}](https://arxiv.org/abs/2110.10305#google "'When in Doubt, Summon the Titans: Efficient Inference with Large Models', Rawat et al 2021"){.link-annotated .id-not .include-annotation link-icon="alphabet" link-icon-type="svg" link-icon-color="#4285f4"}** ## `“The Dangers of Underclaiming: Reasons for Caution When Reporting How NLP Systems Fail”, Bowman 2021`{=html} {#bowman-2021-section} **[`The Dangers of Underclaiming: Reasons for Caution When Reporting How NLP Systems Fail`{=html}](https://arxiv.org/abs/2110.08300 "'The Dangers of Underclaiming: Reasons for Caution When Reporting How NLP Systems Fail', Bowman 2021"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Symbolic Knowledge Distillation: from General Language Models to Commonsense Models”, West et al 2021`{=html} {#west-et-al-2021-section} **[`Symbolic Knowledge Distillation: from General Language Models to Commonsense Models`{=html}](https://arxiv.org/abs/2110.07178#allen "'Symbolic Knowledge Distillation: from General Language Models to Commonsense Models', West et al 2021"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“LFPT5: A Unified Framework for Lifelong Few-Shot Language Learning Based on Prompt Tuning of T5”, Qin & Joty 2021`{=html} {#qin-joty-2021-section} **[`LFPT5: A Unified Framework for Lifelong Few-shot Language Learning Based on Prompt Tuning of T5`{=html}](https://arxiv.org/abs/2110.07298 "'LFPT5: A Unified Framework for Lifelong Few-shot Language Learning Based on Prompt Tuning of T5', Qin & Joty 2021"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Scaling Laws for the Few-Shot Adaptation of Pre-Trained Image Classifiers”, Prato et al 2021`{=html} {#prato-et-al-2021-1-section} **[`Scaling Laws for the Few-Shot Adaptation of Pre-trained Image Classifiers`{=html}](https://arxiv.org/abs/2110.06990 "'Scaling Laws for the Few-Shot Adaptation of Pre-trained Image Classifiers', Prato et al 2021"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Unsupervised Neural Machine Translation With Generative Language Models Only”, Han et al 2021`{=html} {#han-et-al-2021-2-section} **[`Unsupervised Neural Machine Translation with Generative Language Models Only`{=html}](https://arxiv.org/abs/2110.05448#openai "'Unsupervised Neural Machine Translation with Generative Language Models Only', Han et al 2021"){.link-annotated .id-not .include-annotation link-icon="openai" link-icon-type="svg"}** ## `“Yuan 1.0: Large-Scale Pre-Trained Language Model in Zero-Shot and Few-Shot Learning”, Wu et al 2021`{=html} {#wu-et-al-2021-yuan-1-section} **[`Yuan 1.0: Large-Scale Pre-trained Language Model in Zero-Shot and Few-Shot Learning`{=html}](https://arxiv.org/abs/2110.04725#inspur "'Yuan 1.0: Large-Scale Pre-trained Language Model in Zero-Shot and Few-Shot Learning', Wu et al 2021"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Universal Paralinguistic Speech Representations Using Self-Supervised Conformers”, Shor et al 2021`{=html} {#shor-et-al-2021-section} **[`Universal Paralinguistic Speech Representations Using Self-Supervised Conformers`{=html}](https://arxiv.org/abs/2110.04621#google "'Universal Paralinguistic Speech Representations Using Self-Supervised Conformers', Shor et al 2021"){.link-annotated .id-not .include-annotation link-icon="alphabet" link-icon-type="svg" link-icon-color="#4285f4"}** ## `“M6–10T: A Sharing-Delinking Paradigm for Efficient Multi-Trillion Parameter Pretraining”, Lin et al 2021`{=html} {#lin-et-al-2021-m610t-section} **[`M6–10T: A Sharing-Delinking Paradigm for Efficient Multi-Trillion Parameter Pretraining`{=html}](https://arxiv.org/abs/2110.03888#alibaba "'M6–10T: A Sharing-Delinking Paradigm for Efficient Multi-Trillion Parameter Pretraining', Lin et al 2021"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“A Few More Examples May Be Worth Billions of Parameters”, Kirstain et al 2021`{=html} {#kirstain-et-al-2021-section} **[`A Few More Examples May Be Worth Billions of Parameters`{=html}](https://arxiv.org/abs/2110.04374 "'A Few More Examples May Be Worth Billions of Parameters', Kirstain et al 2021"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Exploring the Limits of Large Scale Pre-Training”, Abnar et al 2021`{=html} {#abnar-et-al-2021-section} **[`Exploring the Limits of Large Scale Pre-training`{=html}](https://arxiv.org/abs/2110.02095#google "'Exploring the Limits of Large Scale Pre-training', Abnar et al 2021"){.link-annotated .id-not .include-annotation link-icon="alphabet" link-icon-type="svg" link-icon-color="#4285f4"}** ## `“Show Your Work: Scratchpads for Intermediate Computation With Language Models”, Nye et al 2021`{=html} {#nye-et-al-2021-section} **[`Show Your Work: Scratchpads for Intermediate Computation with Language Models`{=html}](https://arxiv.org/abs/2112.00114#google "'Show Your Work: Scratchpads for Intermediate Computation with Language Models', Nye et al 2021"){.link-annotated .id-not .include-annotation link-icon="alphabet" link-icon-type="svg" link-icon-color="#4285f4"}** ## `“Mining for Strong Gravitational Lenses With Self-Supervised Learning”, Stein et al 2021`{=html} {#stein-et-al-2021-1-section} **[`Mining for strong gravitational lenses with self-supervised learning`{=html}](https://arxiv.org/abs/2110.00023 "'Mining for strong gravitational lenses with self-supervised learning', Stein et al 2021"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Stochastic Training Is Not Necessary for Generalization”, Geiping et al 2021`{=html} {#geiping-et-al-2021-section} **[`Stochastic Training is Not Necessary for Generalization`{=html}](https://arxiv.org/abs/2109.14119 "'Stochastic Training is Not Necessary for Generalization', Geiping et al 2021"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Evaluating Machine Accuracy on ImageNet”, Shankar et al 2021`{=html} {#shankar-et-al-2021-section} **[`Evaluating Machine Accuracy on ImageNet`{=html}](https://openreview.net/forum?id=Q3R088EFtng#google "'Evaluating Machine Accuracy on ImageNet', Shankar et al 2021"){.link-annotated .id-not .include-annotation link-icon="alphabet" link-icon-type="svg" link-icon-color="#4285f4"}** ## `“BigSSL: Exploring the Frontier of Large-Scale Semi-Supervised Learning for Automatic Speech Recognition”, Zhang et al 2021`{=html} {#zhang-et-al-2021-06-section} **[`BigSSL: Exploring the Frontier of Large-Scale Semi-Supervised Learning for Automatic Speech Recognition`{=html}](https://arxiv.org/abs/2109.13226#google "'BigSSL: Exploring the Frontier of Large-Scale Semi-Supervised Learning for Automatic Speech Recognition', Zhang et al 2021"){.link-annotated .id-not .include-annotation link-icon="alphabet" link-icon-type="svg" link-icon-color="#4285f4"}** ## `“Scale Efficiently: Insights from Pre-Training and Fine-Tuning Transformers”, Tay et al 2021`{=html} {#tay-et-al-2021-1-section} **[`Scale Efficiently: Insights from Pre-training and Fine-tuning Transformers`{=html}](https://arxiv.org/abs/2109.10686#google "'Scale Efficiently: Insights from Pre-training and Fine-tuning Transformers', Tay et al 2021"){.link-annotated .id-not .include-annotation link-icon="alphabet" link-icon-type="svg" link-icon-color="#4285f4"}** ## `“Scaling Laws for Neural Machine Translation”, Ghorbani et al 2021`{=html} {#ghorbani-et-al-2021-section} **[`Scaling Laws for Neural Machine Translation`{=html}](https://arxiv.org/abs/2109.07740#google "'Scaling Laws for Neural Machine Translation', Ghorbani et al 2021"){.link-annotated .id-not .include-annotation link-icon="alphabet" link-icon-type="svg" link-icon-color="#4285f4"}** ## `“What Changes Can Large-Scale Language Models Bring? Intensive Study on HyperCLOVA: Billions-Scale Korean Generative Pretrained Transformers”, Kim et al 2021`{=html} {#kim-et-al-2021-6-section} **[`What Changes Can Large-scale Language Models Bring? Intensive Study on HyperCLOVA: Billions-scale Korean Generative Pretrained Transformers`{=html}](https://arxiv.org/abs/2109.04650#naver "'What Changes Can Large-scale Language Models Bring? Intensive Study on HyperCLOVA: Billions-scale Korean Generative Pretrained Transformers', Kim et al 2021"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“A Recipe For Arbitrary Text Style Transfer With Large Language Models”, Reif et al 2021`{=html} {#reif-et-al-2021-section} **[`A Recipe For Arbitrary Text Style Transfer with Large Language Models`{=html}](https://arxiv.org/abs/2109.03910#google "'A Recipe For Arbitrary Text Style Transfer with Large Language Models', Reif et al 2021"){.link-annotated .id-not .include-annotation link-icon="alphabet" link-icon-type="svg" link-icon-color="#4285f4"}** ## `“TruthfulQA: Measuring How Models Mimic Human Falsehoods”, Lin et al 2021`{=html} {#lin-et-al-2021-6-section} **[`TruthfulQA: Measuring How Models Mimic Human Falsehoods`{=html}](https://arxiv.org/abs/2109.07958 "'TruthfulQA: Measuring How Models Mimic Human Falsehoods', Lin et al 2021"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“A Farewell to the Bias-Variance Tradeoff? An Overview of the Theory of Overparameterized Machine Learning”, Dar et al 2021`{=html} {#dar-et-al-2021-section} **[`A Farewell to the Bias-Variance Tradeoff? An Overview of the Theory of Overparameterized Machine Learning`{=html}](https://arxiv.org/abs/2109.02355 "'A Farewell to the Bias-Variance Tradeoff? An Overview of the Theory of Overparameterized Machine Learning', Dar et al 2021"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“General-Purpose Question-Answering With Macaw”, Tafjord & Clark 2021`{=html} {#tafjord-clark-2021-section} **[`General-Purpose Question-Answering with Macaw`{=html}](https://arxiv.org/abs/2109.02593#allen "'General-Purpose Question-Answering with Macaw', Tafjord & Clark 2021"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“An Empirical Exploration in Quality Filtering of Text Data”, Gao 2021`{=html} {#gao-2021-section} **[`An Empirical Exploration in Quality Filtering of Text Data`{=html}](https://arxiv.org/abs/2109.00698#eleutherai "'An Empirical Exploration in Quality Filtering of Text Data', Gao 2021"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“A Battle of Network Structures: An Empirical Study of CNN, Transformer, and MLP”, Zhao et al 2021`{=html} {#zhao-et-al-2021-4-section} **[`A Battle of Network Structures: An Empirical Study of CNN, Transformer, and MLP`{=html}](https://arxiv.org/abs/2108.13002#microsoft "'A Battle of Network Structures: An Empirical Study of CNN, Transformer, and MLP', Zhao et al 2021"){.link-annotated .id-not .include-annotation link-icon="MS" link-icon-type="text,sans,italic"}** ## `“Want To Reduce Labeling Cost? GPT-3 Can Help”, Wang et al 2021`{=html} {#wang-et-al-2021-gpt3labeling-section} **[`Want To Reduce Labeling Cost? GPT-3 Can Help`{=html}](https://arxiv.org/abs/2108.13487#microsoft "'Want To Reduce Labeling Cost? GPT-3 Can Help', Wang et al 2021"){.link-annotated .id-not .include-annotation link-icon="MS" link-icon-type="text,sans,italic"}** ## `“Data and Parameter Scaling Laws for Neural Machine Translation”, Gordon et al 2021`{=html} {#gordon-et-al-2021-section} **[`Data and Parameter Scaling Laws for Neural Machine Translation`{=html}](https://openreview.net/forum?id=IKA7MLxsLSu "'Data and Parameter Scaling Laws for Neural Machine Translation', Gordon et al 2021"){.link-annotated .id-not .include-annotation link-icon="OR" link-icon-type="text,sans" link-icon-color="#8c1b13"}** ## `“Do Vision Transformers See Like Convolutional Neural Networks?”, Raghu et al 2021`{=html} {#raghu-et-al-2021-section} **[`Do Vision Transformers See Like Convolutional Neural Networks?`{=html}](https://arxiv.org/abs/2108.08810#google "'Do Vision Transformers See Like Convolutional Neural Networks?', Raghu et al 2021"){.link-annotated .id-not .include-annotation link-icon="alphabet" link-icon-type="svg" link-icon-color="#4285f4"}** ## `“Modeling Protein Using Large-Scale Pretrain Language Model”, Xiao et al 2021`{=html} {#xiao-et-al-2021-3-section} **[`Modeling Protein Using Large-scale Pretrain Language Model`{=html}](https://arxiv.org/abs/2108.07435#baai "'Modeling Protein Using Large-scale Pretrain Language Model', Xiao et al 2021"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Scaling Laws for Deep Learning”, Rosenfeld 2021`{=html} {#rosenfeld-2021-section} **[`Scaling Laws for Deep Learning`{=html}](https://arxiv.org/abs/2108.07686 "'Scaling Laws for Deep Learning', Rosenfeld 2021"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Billion-Scale Pretraining With Vision Transformers for Multi-Task Visual Representations”, Beal et al 2021`{=html} {#beal-et-al-2021-section} **[`Billion-Scale Pretraining with Vision Transformers for Multi-Task Visual Representations`{=html}](https://arxiv.org/abs/2108.05887#pinterest "'Billion-Scale Pretraining with Vision Transformers for Multi-Task Visual Representations', Beal et al 2021"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Facebook AI WMT21 News Translation Task Submission”, Tran et al 2021`{=html} {#tran-et-al-2021-2-section} **[`Facebook AI WMT21 News Translation Task Submission`{=html}](https://arxiv.org/abs/2108.03265#facebook "'Facebook AI WMT21 News Translation Task Submission', Tran et al 2021"){.link-annotated .id-not .include-annotation link-icon="facebook" link-icon-type="svg" link-icon-color="#1877f2"}** ## `“EVA: An Open-Domain Chinese Dialogue System With Large-Scale Generative Pre-Training”, Zhou et al 2021`{=html} {#zhou-et-al-2021-2-section} **[`EVA: An Open-Domain Chinese Dialogue System with Large-Scale Generative Pre-Training`{=html}](https://arxiv.org/abs/2108.01547#baai "'EVA: An Open-Domain Chinese Dialogue System with Large-Scale Generative Pre-Training', Zhou et al 2021"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“The History of Speech Recognition to the Year 2030”, Hannun 2021`{=html} {#hannun-2021-2-section} **[`The History of Speech Recognition to the Year 2030`{=html}](https://arxiv.org/abs/2108.00084 "'The History of Speech Recognition to the Year 2030', Hannun 2021"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“The History of Speech Recognition to the Year 2030 [Blog]”, Hannun 2021`{=html} {#hannun-2021-1-section} **[`The History of Speech Recognition to the Year 2030 [blog]`{=html}](https://awni.github.io/future-speech/ "'The History of Speech Recognition to the Year 2030 [blog]', Hannun 2021"){.link-annotated-partial .id-not .include-annotation}** ## `“A Field Guide to Federated Optimization”, Wang et al 2021`{=html} {#wang-et-al-2021-05-section} **[`A Field Guide to Federated Optimization`{=html}](https://arxiv.org/abs/2107.06917#google "'A Field Guide to Federated Optimization', Wang et al 2021"){.link-annotated .id-not .include-annotation link-icon="alphabet" link-icon-type="svg" link-icon-color="#4285f4"}** ## `“HTLM: Hyper-Text Pre-Training and Prompting of Language Models”, Aghajanyan et al 2021`{=html} {#aghajanyan-et-al-2021-1-section} **[`HTLM: Hyper-Text Pre-Training and Prompting of Language Models`{=html}](https://arxiv.org/abs/2107.06955#facebook "'HTLM: Hyper-Text Pre-Training and Prompting of Language Models', Aghajanyan et al 2021"){.link-annotated .id-not .include-annotation link-icon="facebook" link-icon-type="svg" link-icon-color="#1877f2"}** ## `“Brain-Like Functional Specialization Emerges Spontaneously in Deep Neural Networks”, Dobs et al 2021`{=html} {#dobs-et-al-2021-section} **[`Brain-like functional specialization emerges spontaneously in deep neural networks`{=html}](https://www.biorxiv.org/content/10.1101/2021.07.05.451192.full "'Brain-like functional specialization emerges spontaneously in deep neural networks', Dobs et al 2021"){.link-annotated .id-not .include-annotation link-icon="chi-dna" link-icon-type="svg" link-icon-color="#bd2736"}** ## `“ERNIE 3.0: Large-Scale Knowledge Enhanced Pre-Training for Language Understanding and Generation”, Sun et al 2021`{=html} {#sun-et-al-2021-4-section} **[`ERNIE 3.0: Large-scale Knowledge Enhanced Pre-training for Language Understanding and Generation`{=html}](https://arxiv.org/abs/2107.02137#baidu "'ERNIE 3.0: Large-scale Knowledge Enhanced Pre-training for Language Understanding and Generation', Sun et al 2021"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Scarecrow: A Framework for Scrutinizing Machine Text”, Dou et al 2021`{=html} {#dou-et-al-2021-section} **[`Scarecrow: A Framework for Scrutinizing Machine Text`{=html}](https://arxiv.org/abs/2107.01294#allen "'Scarecrow: A Framework for Scrutinizing Machine Text', Dou et al 2021"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“The Dimpled Manifold Model of Adversarial Examples in Machine Learning”, Shamir et al 2021`{=html} {#shamir-et-al-2021-section} **[`The Dimpled Manifold Model of Adversarial Examples in Machine Learning`{=html}](https://arxiv.org/abs/2106.10151 "'The Dimpled Manifold Model of Adversarial Examples in Machine Learning', Shamir et al 2021"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Revisiting the Calibration of Modern Neural Networks”, Minderer et al 2021`{=html} {#minderer-et-al-2021-section} **[`Revisiting the Calibration of Modern Neural Networks`{=html}](https://arxiv.org/abs/2106.07998#google "'Revisiting the Calibration of Modern Neural Networks', Minderer et al 2021"){.link-annotated .id-not .include-annotation link-icon="alphabet" link-icon-type="svg" link-icon-color="#4285f4"}** ## `“Partial Success in Closing the Gap between Human and Machine Vision”, Geirhos et al 2021`{=html} {#geirhos-et-al-2021-section} **[`Partial success in closing the gap between human and machine vision`{=html}](https://arxiv.org/abs/2106.07411 "'Partial success in closing the gap between human and machine vision', Geirhos et al 2021"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Units”, Hsu et al 2021`{=html} {#hsu-et-al-2021-section} **[`HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Units`{=html}](https://arxiv.org/abs/2106.07447#facebook "'HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Units', Hsu et al 2021"){.link-annotated .id-not .include-annotation link-icon="facebook" link-icon-type="svg" link-icon-color="#1877f2"}** ## `“Scaling Laws for Acoustic Models”, Droppo & Elibol 2021`{=html} {#droppo-elibol-2021-section} **[`Scaling Laws for Acoustic Models`{=html}](https://arxiv.org/abs/2106.09488#amazon "'Scaling Laws for Acoustic Models', Droppo & Elibol 2021"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“CoAtNet: Marrying Convolution and Attention for All Data Sizes”, Dai et al 2021`{=html} {#dai-et-al-2021-2-section} **[`CoAtNet: Marrying Convolution and Attention for All Data Sizes`{=html}](https://arxiv.org/abs/2106.04803#google "'CoAtNet: Marrying Convolution and Attention for All Data Sizes', Dai et al 2021"){.link-annotated .id-not .include-annotation link-icon="alphabet" link-icon-type="svg" link-icon-color="#4285f4"}** ## `“Scaling Vision Transformers”, Zhai et al 2021`{=html} {#zhai-et-al-2021-3-section} **[`Scaling Vision Transformers`{=html}](https://arxiv.org/abs/2106.04560#google "'Scaling Vision Transformers', Zhai et al 2021"){.link-annotated .id-not .include-annotation link-icon="alphabet" link-icon-type="svg" link-icon-color="#4285f4"}** ## `“Exploring the Limits of Out-Of-Distribution Detection”, Fort et al 2021`{=html} {#fort-et-al-2021-section} **[`Exploring the Limits of Out-of-Distribution Detection`{=html}](https://arxiv.org/abs/2106.03004#google "'Exploring the Limits of Out-of-Distribution Detection', Fort et al 2021"){.link-annotated .id-not .include-annotation link-icon="alphabet" link-icon-type="svg" link-icon-color="#4285f4"}** ## `“Effect of Pre-Training Scale on Intra/Inter-Domain Full and Few-Shot Transfer Learning for Natural and Medical X-Ray Chest Images”, Cherti & Jitsev 2021`{=html} {#cherti-jitsev-2021-section} **[`Effect of Pre-Training Scale on Intra/Inter-Domain Full and Few-Shot Transfer Learning for Natural and Medical X-Ray Chest Images`{=html}](https://arxiv.org/abs/2106.00116 "'Effect of Pre-Training Scale on Intra/Inter-Domain Full and Few-Shot Transfer Learning for Natural and Medical X-Ray Chest Images', Cherti & Jitsev 2021"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“A Universal Law of Robustness via Isoperimetry”, Bubeck & Sellke 2021`{=html} {#bubeck-sellke-2021-section} **[`A Universal Law of Robustness via Isoperimetry`{=html}](https://arxiv.org/abs/2105.12806 "'A Universal Law of Robustness via Isoperimetry', Bubeck & Sellke 2021"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Naver Unveils First ‘Hyperscale’ AI Platform”, Jae-eun 2021`{=html} {#jae-eun-2021-section} **[`Naver unveils first ‘hyperscale’ AI platform`{=html}](https://m.koreaherald.com/view.php?ud=20210525000824#naver "'Naver unveils first ‘hyperscale’ AI platform', Jae-eun 2021"){.link-annotated .id-not .include-annotation}** ## `“Unsupervised Speech Recognition”, Baevski et al 2021`{=html} {#baevski-et-al-2021-section} **[`Unsupervised Speech Recognition`{=html}](https://arxiv.org/abs/2105.11084#facebook "'Unsupervised Speech Recognition', Baevski et al 2021"){.link-annotated .id-not .include-annotation link-icon="facebook" link-icon-type="svg" link-icon-color="#1877f2"}** ## `“One4all User Representation for Recommender Systems in E-Commerce”, Shin et al 2021`{=html} {#shin-et-al-2021-2-section} **[`One4all User Representation for Recommender Systems in E-commerce`{=html}](https://arxiv.org/abs/2106.00573#naver "'One4all User Representation for Recommender Systems in E-commerce', Shin et al 2021"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“RecPipe: Co-Designing Models and Hardware to Jointly Optimize Recommendation Quality and Performance”, Gupta et al 2021`{=html} {#gupta-et-al-2021-2-section} **[`RecPipe: Co-designing Models and Hardware to Jointly Optimize Recommendation Quality and Performance`{=html}](https://arxiv.org/abs/2105.08820#facebook "'RecPipe: Co-designing Models and Hardware to Jointly Optimize Recommendation Quality and Performance', Gupta et al 2021"){.link-annotated .id-not .include-annotation link-icon="facebook" link-icon-type="svg" link-icon-color="#1877f2"}** ## `“Google Details New AI Accelerator Chips”, Wiggers 2021`{=html} {#wiggers-2021-section} **[`Google details new AI accelerator chips`{=html}](https://venturebeat.com/ai/google-details-new-ai-accelerator-chips/ "'Google details new AI accelerator chips', Wiggers 2021"){.link-annotated .id-not .include-annotation link-icon="alphabet" link-icon-type="svg" link-icon-color="#4285f4"}** ## `“MLP-Mixer: An All-MLP Architecture for Vision”, Tolstikhin et al 2021`{=html} {#tolstikhin-et-al-2021-section} **[`MLP-Mixer: An all-MLP Architecture for Vision`{=html}](https://arxiv.org/abs/2105.01601#google "'MLP-Mixer: An all-MLP Architecture for Vision', Tolstikhin et al 2021"){.link-annotated .id-not .include-annotation link-icon="alphabet" link-icon-type="svg" link-icon-color="#4285f4"}** ## `“XLM-R XL: Larger-Scale Transformers for Multilingual Masked Language Modeling”, Goyal et al 2021`{=html} {#goyal-et-al-2021-xlmrxl-section} **[`XLM-R XL: Larger-Scale Transformers for Multilingual Masked Language Modeling`{=html}](https://arxiv.org/abs/2105.00572#facebook "'XLM-R XL: Larger-Scale Transformers for Multilingual Masked Language Modeling', Goyal et al 2021"){.link-annotated .id-not .include-annotation link-icon="facebook" link-icon-type="svg" link-icon-color="#1877f2"}** ## `“Scaling End-To-End Models for Large-Scale Multilingual ASR”, Li et al 2021`{=html} {#li-et-al-2021-7-section} **[`Scaling End-to-End Models for Large-Scale Multilingual ASR`{=html}](https://arxiv.org/abs/2104.14830#google "'Scaling End-to-End Models for Large-Scale Multilingual ASR', Li et al 2021"){.link-annotated .id-not .include-annotation link-icon="alphabet" link-icon-type="svg" link-icon-color="#4285f4"}** ## `“DINO: Emerging Properties in Self-Supervised Vision Transformers”, Caron et al 2021`{=html} {#caron-et-al-2021-section} **[`DINO: Emerging Properties in Self-Supervised Vision Transformers`{=html}](https://arxiv.org/abs/2104.14294#facebook "'DINO: Emerging Properties in Self-Supervised Vision Transformers', Caron et al 2021"){.link-annotated .id-not .include-annotation link-icon="facebook" link-icon-type="svg" link-icon-color="#1877f2"}** ## `“What Are Bayesian Neural Network Posteriors Really Like?”, Izmailov et al 2021`{=html} {#izmailov-et-al-2021-section} **[`What Are Bayesian Neural Network Posteriors Really Like?`{=html}](https://arxiv.org/abs/2104.14421#google "'What Are Bayesian Neural Network Posteriors Really Like?', Izmailov et al 2021"){.link-annotated .id-not .include-annotation link-icon="alphabet" link-icon-type="svg" link-icon-color="#4285f4"}** ## `“Machine Learning Scaling”, Gwern 2021`{=html} {#gwern-doc-ai-scaling-abstract-section} **[`Machine Learning Scaling`{=html}](/doc/ai/scaling/abstract "'Machine Learning Scaling', Gwern 2021"){.link-annotated-partial .id-not .include-annotation}** ## `“[Ali Released PLUG: 27 Billion Parameters, the Largest Pre-Trained Language Model in the Chinese Community]”, Yuying 2021`{=html} {#yuying-2021-section} **[`[Ali released PLUG: 27 billion parameters, the largest pre-trained language model in the Chinese community]`{=html}](https://www.infoq.cn/article/EFIHo75sQsVqLvFTruKE#alibaba "'[Ali released PLUG: 27 billion parameters, the largest pre-trained language model in the Chinese community]', Yuying 2021"){.link-annotated .id-not .include-annotation}** ## `“The Power of Scale for Parameter-Efficient Prompt Tuning”, Lester et al 2021`{=html} {#lester-et-al-2021-section} **[`The Power of Scale for Parameter-Efficient Prompt Tuning`{=html}](https://arxiv.org/abs/2104.08691#google "'The Power of Scale for Parameter-Efficient Prompt Tuning', Lester et al 2021"){.link-annotated .id-not .include-annotation link-icon="alphabet" link-icon-type="svg" link-icon-color="#4285f4"}** ## `“Revealing Persona Biases in Dialogue Systems”, Sheng et al 2021`{=html} {#sheng-et-al-2021-section} **[`Revealing Persona Biases in Dialogue Systems`{=html}](https://arxiv.org/abs/2104.08728 "'Revealing Persona Biases in Dialogue Systems', Sheng et al 2021"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“CrossFit: A Few-Shot Learning Challenge for Cross-Task Generalization in NLP”, Ye et al 2021`{=html} {#ye-et-al-2021-2-section} **[`CrossFit: A Few-shot Learning Challenge for Cross-task Generalization in NLP`{=html}](https://arxiv.org/abs/2104.08835 "'CrossFit: A Few-shot Learning Challenge for Cross-task Generalization in NLP', Ye et al 2021"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Probing Across Time: What Does RoBERTa Know and When?”, Liu et al 2021`{=html} {#liu-et-al-2021-roberta-section} **[`Probing Across Time: What Does RoBERTa Know and When?`{=html}](https://arxiv.org/abs/2104.07885 "'Probing Across Time: What Does RoBERTa Know and When?', Liu et al 2021"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Memorization versus Generalization in Pre-Trained Language Models”, Tänzer et al 2021`{=html} {#tänzer-et-al-2021-section} **[`Memorization versus Generalization in Pre-trained Language Models`{=html}](https://arxiv.org/abs/2105.00828 "'Memorization versus Generalization in Pre-trained Language Models', Tänzer et al 2021"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Large-Scale Self-Supervised and Semi-Supervised Learning for Speech Translation”, Wang et al 2021`{=html} {#wang-et-al-2021-wav2vec20-section} **[`Large-Scale Self-Supervised and Semi-Supervised Learning for Speech Translation`{=html}](https://arxiv.org/abs/2104.06678#facebook "'Large-Scale Self-Supervised and Semi-Supervised Learning for Speech Translation', Wang et al 2021"){.link-annotated .id-not .include-annotation link-icon="facebook" link-icon-type="svg" link-icon-color="#1877f2"}** ## `“Scaling Laws for Language Transfer Learning”, Kim 2021`{=html} {#kim-2021-section} **[`Scaling Laws for Language Transfer Learning`{=html}](https://christina.kim/2021/04/11/scaling-laws-for-language-transfer-learning/#openai "'Scaling Laws for Language Transfer Learning', Kim 2021"){.link-annotated .id-not .include-annotation link-icon="openai" link-icon-type="svg"}** ## `“Adapting Language Models for Zero-Shot Learning by Meta-Tuning on Dataset and Prompt Collections”, Zhong et al 2021`{=html} {#zhong-et-al-2021-2-section} **[`Adapting Language Models for Zero-shot Learning by Meta-tuning on Dataset and Prompt Collections`{=html}](https://arxiv.org/abs/2104.04670 "'Adapting Language Models for Zero-shot Learning by Meta-tuning on Dataset and Prompt Collections', Zhong et al 2021"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“SpeechStew: Simply Mix All Available Speech Recognition Data to Train One Large Neural Network”, Chan et al 2021`{=html} {#chan-et-al-2021-2-section} **[`SpeechStew: Simply Mix All Available Speech Recognition Data to Train One Large Neural Network`{=html}](https://arxiv.org/abs/2104.02133#google "'SpeechStew: Simply Mix All Available Speech Recognition Data to Train One Large Neural Network', Chan et al 2021"){.link-annotated .id-not .include-annotation link-icon="alphabet" link-icon-type="svg" link-icon-color="#4285f4"}** ## `“Understanding Robustness of Transformers for Image Classification”, Bhojanapalli et al 2021`{=html} {#bhojanapalli-et-al-2021-section} **[`Understanding Robustness of Transformers for Image Classification`{=html}](https://arxiv.org/abs/2103.14586#google "'Understanding Robustness of Transformers for Image Classification', Bhojanapalli et al 2021"){.link-annotated .id-not .include-annotation link-icon="alphabet" link-icon-type="svg" link-icon-color="#4285f4"}** ## `“UNICORN on RAINBOW: A Universal Commonsense Reasoning Model on a New Multitask Benchmark”, Lourie et al 2021`{=html} {#lourie-et-al-2021-section} **[`UNICORN on RAINBOW: A Universal Commonsense Reasoning Model on a New Multitask Benchmark`{=html}](https://arxiv.org/abs/2103.13009#allen "'UNICORN on RAINBOW: A Universal Commonsense Reasoning Model on a New Multitask Benchmark', Lourie et al 2021"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Controllable Generation from Pre-Trained Language Models via Inverse Prompting”, Zou et al 2021`{=html} {#zou-et-al-2021-section} **[`Controllable Generation from Pre-trained Language Models via Inverse Prompting`{=html}](https://arxiv.org/abs/2103.10685 "'Controllable Generation from Pre-trained Language Models via Inverse Prompting', Zou et al 2021"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“The Shape of Learning Curves: a Review”, Viering & Loog 2021`{=html} {#viering-loog-2021-section} **[`The Shape of Learning Curves: a Review`{=html}](https://arxiv.org/abs/2103.10948 "'The Shape of Learning Curves: a Review', Viering & Loog 2021"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Efficient Visual Pretraining With Contrastive Detection”, Hénaff et al 2021`{=html} {#hénaff-et-al-2021-section} **[`Efficient Visual Pretraining with Contrastive Detection`{=html}](https://arxiv.org/abs/2103.10957#deepmind "'Efficient Visual Pretraining with Contrastive Detection', Hénaff et al 2021"){.link-annotated .id-not .include-annotation link-icon="deepmind" link-icon-type="svg" link-icon-color="#4185f4"}** ## `“Revisiting ResNets: Improved Training and Scaling Strategies”, Bello et al 2021`{=html} {#bello-et-al-2021-section} **[`Revisiting ResNets: Improved Training and Scaling Strategies`{=html}](https://arxiv.org/abs/2103.07579#google "'Revisiting ResNets: Improved Training and Scaling Strategies', Bello et al 2021"){.link-annotated .id-not .include-annotation link-icon="alphabet" link-icon-type="svg" link-icon-color="#4285f4"}** ## `“Learning from Videos to Understand the World”, Zweig et al 2021`{=html} {#zweig-et-al-2021-section} **[`Learning from videos to understand the world`{=html}](https://ai.meta.com/blog/learning-from-videos-to-understand-the-world/ "'Learning from videos to understand the world', Zweig et al 2021"){.link-annotated .id-not .include-annotation link-icon="facebook" link-icon-type="svg" link-icon-color="#1877f2"}** ## `“WenLan: Bridging Vision and Language by Large-Scale Multi-Modal Pre-Training”, Huo et al 2021`{=html} {#huo-et-al-2021-section} **[`WenLan: Bridging Vision and Language by Large-Scale Multi-Modal Pre-Training`{=html}](https://arxiv.org/abs/2103.06561 "'WenLan: Bridging Vision and Language by Large-Scale Multi-Modal Pre-Training', Huo et al 2021"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Fast and Accurate Model Scaling”, Dollár et al 2021`{=html} {#dollár-et-al-2021-section} **[`Fast and Accurate Model Scaling`{=html}](https://arxiv.org/abs/2103.06877#facebook "'Fast and Accurate Model Scaling', Dollár et al 2021"){.link-annotated .id-not .include-annotation link-icon="facebook" link-icon-type="svg" link-icon-color="#1877f2"}** ## `“Pretrained Transformers As Universal Computation Engines”, Lu et al 2021`{=html} {#lu-et-al-2021-3-section} **[`Pretrained Transformers as Universal Computation Engines`{=html}](https://arxiv.org/abs/2103.05247 "'Pretrained Transformers as Universal Computation Engines', Lu et al 2021"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Greedy Hierarchical Variational Autoencoders (GHVAEs) for Large-Scale Video Prediction”, Wu et al 2021`{=html} {#wu-et-al-2021-ghvae-section} **[`Greedy Hierarchical Variational Autoencoders (GHVAEs) for Large-Scale Video Prediction`{=html}](https://arxiv.org/abs/2103.04174 "'Greedy Hierarchical Variational Autoencoders (GHVAEs) for Large-Scale Video Prediction', Wu et al 2021"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Measuring Mathematical Problem Solving With the MATH Dataset”, Hendrycks et al 2021`{=html} {#hendrycks-et-al-2021-4-section} **[`Measuring Mathematical Problem Solving With the MATH Dataset`{=html}](https://arxiv.org/abs/2103.03874 "'Measuring Mathematical Problem Solving With the MATH Dataset', Hendrycks et al 2021"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“A Law of Robustness for Two-Layers Neural Networks”, Bubeck et al 2021`{=html} {#bubeck-et-al-2021-section} **[`A law of robustness for two-layers neural networks`{=html}](https://openreview.net/forum?id=y4-e1K23GLC "'A law of robustness for two-layers neural networks', Bubeck et al 2021"){.link-annotated .id-not .include-annotation link-icon="OR" link-icon-type="text,sans" link-icon-color="#8c1b13"}** ## `“SEER: Self-Supervised Pretraining of Visual Features in the Wild”, Goyal et al 2021`{=html} {#goyal-et-al-2021-seer-section} **[`SEER: Self-supervised Pretraining of Visual Features in the Wild`{=html}](https://arxiv.org/abs/2103.01988#facebook "'SEER: Self-supervised Pretraining of Visual Features in the Wild', Goyal et al 2021"){.link-annotated .id-not .include-annotation link-icon="facebook" link-icon-type="svg" link-icon-color="#1877f2"}** ## `“M6: A Chinese Multimodal Pretrainer”, Lin et al 2021`{=html} {#lin-et-al-2021-m6-section} **[`M6: A Chinese Multimodal Pretrainer`{=html}](https://arxiv.org/abs/2103.00823#alibaba "'M6: A Chinese Multimodal Pretrainer', Lin et al 2021"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Zero-Shot Text-To-Image Generation”, Ramesh et al 2021`{=html} {#ramesh-et-al-2021-dalle-paper-section} **[`Zero-Shot Text-to-Image Generation`{=html}](https://arxiv.org/abs/2102.12092#openai "'Zero-Shot Text-to-Image Generation', Ramesh et al 2021"){.link-annotated .id-not .include-annotation link-icon="openai" link-icon-type="svg"}** ## `“Improved Denoising Diffusion Probabilistic Models”, Nichol & Dhariwal 2021`{=html} {#nichol-dhariwal-2021-section} **[`Improved Denoising Diffusion Probabilistic Models`{=html}](https://arxiv.org/abs/2102.09672#openai "'Improved Denoising Diffusion Probabilistic Models', Nichol & Dhariwal 2021"){.link-annotated .id-not .include-annotation link-icon="openai" link-icon-type="svg"}** ## `“Conceptual 12M: Pushing Web-Scale Image-Text Pre-Training To Recognize Long-Tail Visual Concepts”, Changpinyo et al 2021`{=html} {#changpinyo-et-al-2021-section} **[`Conceptual 12M: Pushing Web-Scale Image-Text Pre-Training To Recognize Long-Tail Visual Concepts`{=html}](https://arxiv.org/abs/2102.08981#google "'Conceptual 12M: Pushing Web-Scale Image-Text Pre-Training To Recognize Long-Tail Visual Concepts', Changpinyo et al 2021"){.link-annotated .id-not .include-annotation link-icon="alphabet" link-icon-type="svg" link-icon-color="#4285f4"}** ## `“A Large Batch Optimizer Reality Check: Traditional, Generic Optimizers Suffice Across Batch Sizes”, Nado et al 2021`{=html} {#nado-et-al-2021-section} **[`A Large Batch Optimizer Reality Check: Traditional, Generic Optimizers Suffice Across Batch Sizes`{=html}](https://arxiv.org/abs/2102.06356 "'A Large Batch Optimizer Reality Check: Traditional, Generic Optimizers Suffice Across Batch Sizes', Nado et al 2021"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Explaining Neural Scaling Laws”, Bahri et al 2021`{=html} {#bahri-et-al-2021-2-section} **[`Explaining Neural Scaling Laws`{=html}](https://arxiv.org/abs/2102.06701#deepmind "'Explaining Neural Scaling Laws', Bahri et al 2021"){.link-annotated .id-not .include-annotation link-icon="deepmind" link-icon-type="svg" link-icon-color="#4185f4"}** ## `“ALIGN: Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervision”, Jia et al 2021`{=html} {#jia-et-al-2021-section} **[`ALIGN: Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervision`{=html}](https://arxiv.org/abs/2102.05918#google "'ALIGN: Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervision', Jia et al 2021"){.link-annotated .id-not .include-annotation link-icon="alphabet" link-icon-type="svg" link-icon-color="#4285f4"}** ## `“NFNet: High-Performance Large-Scale Image Recognition Without Normalization”, Brock et al 2021`{=html} {#brock-et-al-2021-section} **[`NFNet: High-Performance Large-Scale Image Recognition Without Normalization`{=html}](https://arxiv.org/abs/2102.06171#deepmind "'NFNet: High-Performance Large-Scale Image Recognition Without Normalization', Brock et al 2021"){.link-annotated .id-not .include-annotation link-icon="deepmind" link-icon-type="svg" link-icon-color="#4185f4"}** ## `“Learning Curve Theory”, Hutter 2021`{=html} {#hutter-2021-section} **[`Learning Curve Theory`{=html}](https://arxiv.org/abs/2102.04074#deepmind "'Learning Curve Theory', Hutter 2021"){.link-annotated .id-not .include-annotation link-icon="deepmind" link-icon-type="svg" link-icon-color="#4185f4"}** ## `“1-Bit Adam: Communication Efficient Large-Scale Training With Adam’s Convergence Speed”, Tang et al 2021`{=html} {#tang-et-al-2021-1bitadam-section} **[`1-bit Adam: Communication Efficient Large-Scale Training with Adam’s Convergence Speed`{=html}](https://arxiv.org/abs/2102.02888#microsoft "'1-bit Adam: Communication Efficient Large-Scale Training with Adam’s Convergence Speed', Tang et al 2021"){.link-annotated .id-not .include-annotation link-icon="MS" link-icon-type="text,sans,italic"}** ## `“Mind the Gap: Assessing Temporal Generalization in Neural Language Models § Scaling”, Lazaridou et al 2021`{=html} {#lazaridou-et-al-2021-section} **[`Mind the Gap: Assessing Temporal Generalization in Neural Language Models § Scaling`{=html}](https://arxiv.org/abs/2102.01951#scaling&org=deepmind "'Mind the Gap: Assessing Temporal Generalization in Neural Language Models § Scaling', Lazaridou et al 2021"){.link-annotated .id-not .include-annotation link-icon="deepmind" link-icon-type="svg" link-icon-color="#4185f4"}** ## `“Mind the Gap: Assessing Temporal Generalization in Neural Language Models § Dynamic Evaluation”, Lazaridou et al 2021 (page 7 org deepmind)`{=html} {#lazaridou-et-al-2021-page-7-org-deepmind-section} **[`Mind the Gap: Assessing Temporal Generalization in Neural Language Models § Dynamic Evaluation`{=html}](https://arxiv.org/pdf/2102.01951#page=7&org=deepmind "'Mind the Gap: Assessing Temporal Generalization in Neural Language Models § Dynamic Evaluation', Lazaridou et al 2021 (page 7 org deepmind)"){.link-annotated .id-not .include-annotation link-icon="deepmind" link-icon-type="svg" link-icon-color="#4185f4"}** ## `“Scaling Laws for Transfer”, Hernandez et al 2021`{=html} {#hernandez-et-al-2021-2-section} **[`Scaling Laws for Transfer`{=html}](https://arxiv.org/abs/2102.01293#openai "'Scaling Laws for Transfer', Hernandez et al 2021"){.link-annotated .id-not .include-annotation link-icon="openai" link-icon-type="svg"}** ## `“Automatic Curation of Large-Scale Datasets for Audio-Visual Representation Learning”, Lee et al 2021`{=html} {#lee-et-al-2021-acav100m-section} **[`Automatic Curation of Large-Scale Datasets for Audio-Visual Representation Learning`{=html}](https://arxiv.org/abs/2101.10803 "'Automatic Curation of Large-Scale Datasets for Audio-Visual Representation Learning', Lee et al 2021"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Muppet: Massive Multi-Task Representations With Pre-Finetuning”, Aghajanyan et al 2021`{=html} {#aghajanyan-et-al-2021-2-section} **[`Muppet: Massive Multi-task Representations with Pre-Finetuning`{=html}](https://arxiv.org/abs/2101.11038#facebook "'Muppet: Massive Multi-task Representations with Pre-Finetuning', Aghajanyan et al 2021"){.link-annotated .id-not .include-annotation link-icon="facebook" link-icon-type="svg" link-icon-color="#1877f2"}** ## `“Language Processing in Brains and Deep Neural Networks: Computational Convergence and Its Limits”, Caucheteux & King 2021`{=html} {#caucheteux-king-2021-section} **[`Language processing in brains and deep neural networks: computational convergence and its limits`{=html}](https://www.biorxiv.org/content/10.1101/2020.07.03.186288.full "'Language processing in brains and deep neural networks: computational convergence and its limits', Caucheteux & King 2021"){.link-annotated .id-not .include-annotation link-icon="chi-dna" link-icon-type="svg" link-icon-color="#bd2736"}** ## `“Meta Pseudo Labels”, Pham et al 2021`{=html} {#pham-et-al-2021-2-section} **[`Meta Pseudo Labels`{=html}](https://arxiv.org/abs/2003.10580#google "'Meta Pseudo Labels', Pham et al 2021"){.link-annotated .id-not .include-annotation link-icon="alphabet" link-icon-type="svg" link-icon-color="#4285f4"}** ## `“CLIP: Learning Transferable Visual Models From Natural Language Supervision”, Radford et al 2021`{=html} {#radford-et-al-2021-section} **[`CLIP: Learning Transferable Visual Models From Natural Language Supervision`{=html}](https://cdn.openai.com/papers/Learning_Transferable_Visual_Models_From_Natural_Language_Supervision.pdf "'CLIP: Learning Transferable Visual Models From Natural Language Supervision', Radford et al 2021"){.link-annotated .id-not .include-annotation link-icon="openai" link-icon-type="svg"}** ## `“VoxPopuli: A Large-Scale Multilingual Speech Corpus for Representation Learning, Semi-Supervised Learning and Interpretation”, Wang et al 2021`{=html} {#wang-et-al-2021-voxpopuli-section} **[`VoxPopuli: A Large-Scale Multilingual Speech Corpus for Representation Learning, Semi-Supervised Learning and Interpretation`{=html}](https://arxiv.org/abs/2101.00390#facebook "'VoxPopuli: A Large-Scale Multilingual Speech Corpus for Representation Learning, Semi-Supervised Learning and Interpretation', Wang et al 2021"){.link-annotated .id-not .include-annotation link-icon="facebook" link-icon-type="svg" link-icon-color="#1877f2"}** ## `“CDLM: Cross-Document Language Modeling”, Caciularu et al 2021`{=html} {#caciularu-et-al-2021-section} **[`CDLM: Cross-Document Language Modeling`{=html}](https://arxiv.org/abs/2101.00406 "'CDLM: Cross-Document Language Modeling', Caciularu et al 2021"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“VinVL: Revisiting Visual Representations in Vision-Language Models”, Zhang et al 2021`{=html} {#zhang-et-al-2021-12-section} **[`VinVL: Revisiting Visual Representations in Vision-Language Models`{=html}](https://arxiv.org/abs/2101.00529#microsoft "'VinVL: Revisiting Visual Representations in Vision-Language Models', Zhang et al 2021"){.link-annotated .id-not .include-annotation link-icon="MS" link-icon-type="text,sans,italic"}** ## `“Parameter Count vs Training Dataset Size (1952–2021)”, Adlam 2021`{=html} {#adlam-2021-section} **[`Parameter count vs Training dataset size (1952–2021)`{=html}](/doc/ai/scaling/2021-adlam.pdf "'Parameter count vs Training dataset size (1952–2021)', Adlam 2021"){.link-annotated-partial .id-not .include-annotation link-icon="pdf" link-icon-type="svg" link-icon-color="#f40f02"}** ## `“Process for Adapting Language Models to Society (PALMS) With Values-Targeted Datasets”, Solaiman & Dennison 2021`{=html} {#solaiman-dennison-2021-section} **[`Process for Adapting Language Models to Society (PALMS) with Values-Targeted Datasets`{=html}](https://proceedings.neurips.cc/paper/2021/file/2e855f9489df0712b4bd8ea9e2848c5a-Paper.pdf#openai "'Process for Adapting Language Models to Society (PALMS) with Values-Targeted Datasets', Solaiman & Dennison 2021"){.link-annotated .id-not .include-annotation link-icon="openai" link-icon-type="svg"}** ## `“Intrinsic Dimensionality Explains the Effectiveness of Language Model Fine-Tuning”, Aghajanyan et al 2020`{=html} {#aghajanyan-et-al-2020-section} **[`Intrinsic Dimensionality Explains the Effectiveness of Language Model Fine-Tuning`{=html}](https://arxiv.org/abs/2012.13255 "'Intrinsic Dimensionality Explains the Effectiveness of Language Model Fine-Tuning', Aghajanyan et al 2020"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Extrapolating GPT-N Performance”, Finnveden 2020`{=html} {#finnveden-2020-section} **[`Extrapolating GPT-N performance`{=html}](https://www.alignmentforum.org/posts/k2SNji3jXaLGhBeYP/extrapolating-gpt-n-performance "'Extrapolating GPT-N performance', Finnveden 2020"){.link-annotated .id-not .include-annotation link-icon="AF" link-icon-type="text,sans" link-icon-color="#3f51b5"}** ## `“Biological Structure and Function Emerge from Scaling Unsupervised Learning to 250 Million Protein Sequences”, Rives et al 2020`{=html} {#rives-et-al-2020-section} **[`Biological structure and function emerge from scaling unsupervised learning to 250 million protein sequences`{=html}](https://www.biorxiv.org/content/10.1101/622803.full#facebook "'Biological structure and function emerge from scaling unsupervised learning to 250 million protein sequences', Rives et al 2020"){.link-annotated .id-not .include-annotation link-icon="facebook" link-icon-type="svg" link-icon-color="#1877f2"}** ## `“CPM: A Large-Scale Generative Chinese Pre-Trained Language Model”, Zhang et al 2020`{=html} {#zhang-et-al-2020-04-section} **[`CPM: A Large-scale Generative Chinese Pre-trained Language Model`{=html}](https://arxiv.org/abs/2012.00413 "'CPM: A Large-scale Generative Chinese Pre-trained Language Model', Zhang et al 2020"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Very Deep VAEs Generalize Autoregressive Models and Can Outperform Them on Images”, Child 2020`{=html} {#child-2020-section} **[`Very Deep VAEs Generalize Autoregressive Models and Can Outperform Them on Images`{=html}](https://arxiv.org/abs/2011.10650#openai "'Very Deep VAEs Generalize Autoregressive Models and Can Outperform Them on Images', Child 2020"){.link-annotated .id-not .include-annotation link-icon="openai" link-icon-type="svg"}** ## `“When Do You Need Billions of Words of Pretraining Data?”, Zhang et al 2020`{=html} {#zhang-et-al-2020-05-section} **[`When Do You Need Billions of Words of Pretraining Data?`{=html}](https://arxiv.org/abs/2011.04946 "'When Do You Need Billions of Words of Pretraining Data?', Zhang et al 2020"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Scaling Laws for Autoregressive Generative Modeling”, Henighan et al 2020`{=html} {#henighan-et-al-2020-section} **[`Scaling Laws for Autoregressive Generative Modeling`{=html}](https://arxiv.org/abs/2010.14701#openai "'Scaling Laws for Autoregressive Generative Modeling', Henighan et al 2020"){.link-annotated .id-not .include-annotation link-icon="openai" link-icon-type="svg"}** ## `“Language ID in the Wild: Unexpected Challenges on the Path to a Thousand-Language Web Text Corpus”, Caswell et al 2020`{=html} {#caswell-et-al-2020-section} **[`Language ID in the Wild: Unexpected Challenges on the Path to a Thousand-Language Web Text Corpus`{=html}](https://arxiv.org/abs/2010.14571#google "'Language ID in the Wild: Unexpected Challenges on the Path to a Thousand-Language Web Text Corpus', Caswell et al 2020"){.link-annotated .id-not .include-annotation link-icon="alphabet" link-icon-type="svg" link-icon-color="#4285f4"}** ## `“MT5: A Massively Multilingual Pre-Trained Text-To-Text Transformer”, Xue et al 2020`{=html} {#xue-et-al-2020-1-section} **[`mT5: A massively multilingual pre-trained text-to-text transformer`{=html}](https://arxiv.org/abs/2010.11934#google "'mT5: A massively multilingual pre-trained text-to-text transformer', Xue et al 2020"){.link-annotated .id-not .include-annotation link-icon="alphabet" link-icon-type="svg" link-icon-color="#4285f4"}** ## `“Beyond English-Centric Multilingual Machine Translation”, Fan et al 2020`{=html} {#fan-et-al-2020-1-section} **[`Beyond English-Centric Multilingual Machine Translation`{=html}](https://arxiv.org/abs/2010.11125#facebook "'Beyond English-Centric Multilingual Machine Translation', Fan et al 2020"){.link-annotated .id-not .include-annotation link-icon="facebook" link-icon-type="svg" link-icon-color="#1877f2"}** ## `“Pushing the Limits of Semi-Supervised Learning for Automatic Speech Recognition”, Zhang et al 2020`{=html} {#zhang-et-al-2020-06-section} **[`Pushing the Limits of Semi-Supervised Learning for Automatic Speech Recognition`{=html}](https://arxiv.org/abs/2010.10504#google "'Pushing the Limits of Semi-Supervised Learning for Automatic Speech Recognition', Zhang et al 2020"){.link-annotated .id-not .include-annotation link-icon="alphabet" link-icon-type="svg" link-icon-color="#4285f4"}** ## `“Towards End-To-End In-Image Neural Machine Translation”, Mansimov et al 2020`{=html} {#mansimov-et-al-2020-section} **[`Towards End-to-End In-Image Neural Machine Translation`{=html}](https://arxiv.org/abs/2010.10648#google "'Towards End-to-End In-Image Neural Machine Translation', Mansimov et al 2020"){.link-annotated .id-not .include-annotation link-icon="alphabet" link-icon-type="svg" link-icon-color="#4285f4"}** ## `“The First AI Model That Translates 100 Languages without Relying on English Data”, Fan 2020`{=html} {#fan-2020-section} **[`The first AI model that translates 100 languages without relying on English data`{=html}](https://ai.meta.com/blog/introducing-many-to-many-multilingual-machine-translation/ "'The first AI model that translates 100 languages without relying on English data', Fan 2020"){.link-annotated .id-not .include-annotation link-icon="facebook" link-icon-type="svg" link-icon-color="#1877f2"}** ## `“The Deep Bootstrap Framework: Good Online Learners Are Good Offline Generalizers”, Nakkiran et al 2020`{=html} {#nakkiran-et-al-2020-section} **[`The Deep Bootstrap Framework: Good Online Learners are Good Offline Generalizers`{=html}](https://arxiv.org/abs/2010.08127#google "'The Deep Bootstrap Framework: Good Online Learners are Good Offline Generalizers', Nakkiran et al 2020"){.link-annotated .id-not .include-annotation link-icon="alphabet" link-icon-type="svg" link-icon-color="#4285f4"}** ## `“Learning Which Features Matter: RoBERTa Acquires a Preference for Linguistic Generalizations (Eventually)”, Warstadt et al 2020`{=html} {#warstadt-et-al-2020-section} **[`Learning Which Features Matter: RoBERTa Acquires a Preference for Linguistic Generalizations (Eventually)`{=html}](https://arxiv.org/abs/2010.05358 "'Learning Which Features Matter: RoBERTa Acquires a Preference for Linguistic Generalizations (Eventually)', Warstadt et al 2020"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“The Neural Architecture of Language: Integrative Reverse-Engineering Converges on a Model for Predictive Processing”, Schrimpf et al 2020`{=html} {#schrimpf-et-al-2020-section} **[`The neural architecture of language: Integrative reverse-engineering converges on a model for predictive processing`{=html}](https://www.biorxiv.org/content/10.1101/2020.06.26.174482.full "'The neural architecture of language: Integrative reverse-engineering converges on a model for predictive processing', Schrimpf et al 2020"){.link-annotated .id-not .include-annotation link-icon="chi-dna" link-icon-type="svg" link-icon-color="#bd2736"}** ## `“Uncovering the Limits of Adversarial Training against Norm-Bounded Adversarial Examples”, Gowal et al 2020`{=html} {#gowal-et-al-2020-section} **[`Uncovering the Limits of Adversarial Training against Norm-Bounded Adversarial Examples`{=html}](https://arxiv.org/abs/2010.03593#deepmind "'Uncovering the Limits of Adversarial Training against Norm-Bounded Adversarial Examples', Gowal et al 2020"){.link-annotated .id-not .include-annotation link-icon="deepmind" link-icon-type="svg" link-icon-color="#4185f4"}** ## `“Fast Stencil-Code Computation on a Wafer-Scale Processor”, Rocki et al 2020`{=html} {#rocki-et-al-2020-section} **[`Fast Stencil-Code Computation on a Wafer-Scale Processor`{=html}](https://arxiv.org/abs/2010.03660#cerebras "'Fast Stencil-Code Computation on a Wafer-Scale Processor', Rocki et al 2020"){.link-annotated .id-not .include-annotation link-icon="C" link-icon-type="text,sans" link-icon-color="#f05a29"}** ## `“Vision Transformer: An Image Is Worth 16×16 Words: Transformers for Image Recognition at Scale”, Dosovitskiy et al 2020`{=html} {#dosovitskiy-et-al-2020-section} **[`Vision Transformer: An Image is Worth 16×16 Words: Transformers for Image Recognition at Scale`{=html}](https://arxiv.org/abs/2010.11929#google "'Vision Transformer: An Image is Worth 16×16 Words: Transformers for Image Recognition at Scale', Dosovitskiy et al 2020"){.link-annotated .id-not .include-annotation link-icon="alphabet" link-icon-type="svg" link-icon-color="#4285f4"}** ## `“Small Data, Big Decisions: Model Selection in the Small-Data Regime”, Bornschein et al 2020`{=html} {#bornschein-et-al-2020-section} **[`Small Data, Big Decisions: Model Selection in the Small-Data Regime`{=html}](https://arxiv.org/abs/2009.12583#deepmind "'Small Data, Big Decisions: Model Selection in the Small-Data Regime', Bornschein et al 2020"){.link-annotated .id-not .include-annotation link-icon="deepmind" link-icon-type="svg" link-icon-color="#4185f4"}** ## `“New Report on How Much Computational Power It Takes to Match the Human Brain”, Carlsmith 2020`{=html} {#carlsmith-2020-section} **[`New Report on How Much Computational Power It Takes to Match the Human Brain`{=html}](https://www.openphilanthropy.org/research/new-report-on-how-much-computational-power-it-takes-to-match-the-human-brain/ "'New Report on How Much Computational Power It Takes to Match the Human Brain', Carlsmith 2020"){.link-annotated .id-not .include-annotation link-icon="open-philanthropy" link-icon-type="svg"}** ## `“Generative Language Modeling for Automated Theorem Proving”, Polu & Sutskever 2020`{=html} {#polu-sutskever-2020-section} **[`Generative Language Modeling for Automated Theorem Proving`{=html}](https://arxiv.org/abs/2009.03393#openai "'Generative Language Modeling for Automated Theorem Proving', Polu & Sutskever 2020"){.link-annotated .id-not .include-annotation link-icon="openai" link-icon-type="svg"}** ## `“GrokNet: Unified Computer Vision Model Trunk and Embeddings For Commerce”, Bell et al 2020`{=html} {#bell-et-al-2020-section} **[`GrokNet: Unified Computer Vision Model Trunk and Embeddings For Commerce`{=html}](/doc/ai/scaling/2020-bell.pdf#facebook "'GrokNet: Unified Computer Vision Model Trunk and Embeddings For Commerce', Bell et al 2020"){.link-annotated .id-not .include-annotation link-icon="facebook" link-icon-type="svg" link-icon-color="#1877f2"}** ## `“Accuracy and Performance Comparison of Video Action Recognition Approaches”, Hutchinson et al 2020`{=html} {#hutchinson-et-al-2020-section} **[`Accuracy and Performance Comparison of Video Action Recognition Approaches`{=html}](https://arxiv.org/abs/2008.09037 "'Accuracy and Performance Comparison of Video Action Recognition Approaches', Hutchinson et al 2020"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Generative Models Are Unsupervised Predictors of Page Quality: A Colossal-Scale Study”, Bahri et al 2020`{=html} {#bahri-et-al-2020-section} **[`Generative Models are Unsupervised Predictors of Page Quality: A Colossal-Scale Study`{=html}](https://arxiv.org/abs/2008.13533#google "'Generative Models are Unsupervised Predictors of Page Quality: A Colossal-Scale Study', Bahri et al 2020"){.link-annotated .id-not .include-annotation link-icon="alphabet" link-icon-type="svg" link-icon-color="#4285f4"}** ## `“Matt Botvinick on the Spontaneous Emergence of Learning Algorithms”, Scholl 2020`{=html} {#scholl-2020-section} **[`Matt Botvinick on the spontaneous emergence of learning algorithms`{=html}](https://www.lesswrong.com/posts/Wnqua6eQkewL3bqsF/matt-botvinick-on-the-spontaneous-emergence-of-learning "'Matt Botvinick on the spontaneous emergence of learning algorithms', Scholl 2020"){.link-annotated .id-not .include-annotation link-icon="LW" link-icon-type="text" link-icon-color="#7faf83"}** ## `“Self-Supervised Learning through the Eyes of a Child”, Orhan et al 2020`{=html} {#orhan-et-al-2020-section} **[`Self-supervised learning through the eyes of a child`{=html}](https://arxiv.org/abs/2007.16189 "'Self-supervised learning through the eyes of a child', Orhan et al 2020"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“On Robustness and Transferability of Convolutional Neural Networks”, Djolonga et al 2020`{=html} {#djolonga-et-al-2020-section} **[`On Robustness and Transferability of Convolutional Neural Networks`{=html}](https://arxiv.org/abs/2007.08558#google "'On Robustness and Transferability of Convolutional Neural Networks', Djolonga et al 2020"){.link-annotated .id-not .include-annotation link-icon="alphabet" link-icon-type="svg" link-icon-color="#4285f4"}** ## `“Hopfield Networks Is All You Need”, Ramsauer et al 2020`{=html} {#ramsauer-et-al-2020-section} **[`Hopfield Networks is All You Need`{=html}](https://arxiv.org/abs/2008.02217 "'Hopfield Networks is All You Need', Ramsauer et al 2020"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“ProtTrans: Towards Cracking the Language of Life’s Code Through Self-Supervised Deep Learning and High Performance Computing”, Elnaggar et al 2020`{=html} {#elnaggar-et-al-2020-section} **[`ProtTrans: Towards Cracking the Language of Life’s Code Through Self-Supervised Deep Learning and High Performance Computing`{=html}](https://arxiv.org/abs/2007.06225 "'ProtTrans: Towards Cracking the Language of Life’s Code Through Self-Supervised Deep Learning and High Performance Computing', Elnaggar et al 2020"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“NVAE: A Deep Hierarchical Variational Autoencoder”, Vahdat & Kautz 2020`{=html} {#vahdat-kautz-2020-section} **[`NVAE: A Deep Hierarchical Variational Autoencoder`{=html}](https://arxiv.org/abs/2007.03898#nvidia "'NVAE: A Deep Hierarchical Variational Autoencoder', Vahdat & Kautz 2020"){.link-annotated .id-not .include-annotation link-icon="n" link-icon-type="text,sans,italic" link-icon-color="#77ba00"}** ## `“Measuring Robustness to Natural Distribution Shifts in Image Classification”, Taori et al 2020`{=html} {#taori-et-al-2020-section} **[`Measuring Robustness to Natural Distribution Shifts in Image Classification`{=html}](https://arxiv.org/abs/2007.00644 "'Measuring Robustness to Natural Distribution Shifts in Image Classification', Taori et al 2020"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“WinoGrande: An Adversarial Winograd Schema Challenge at Scale”, Sakaguchi et al 2020`{=html} {#sakaguchi-et-al-2020-section} **[`WinoGrande: An Adversarial Winograd Schema Challenge at Scale`{=html}](https://arxiv.org/abs/1907.10641#allen "'WinoGrande: An Adversarial Winograd Schema Challenge at Scale', Sakaguchi et al 2020"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Is SGD a Bayesian Sampler? Well, Almost”, Mingard et al 2020`{=html} {#mingard-et-al-2020-section} **[`Is SGD a Bayesian sampler? Well, almost`{=html}](https://arxiv.org/abs/2006.15191 "'Is SGD a Bayesian sampler? Well, almost', Mingard et al 2020"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Unsupervised Cross-Lingual Representation Learning for Speech Recognition”, Conneau et al 2020`{=html} {#conneau-et-al-2020-section} **[`Unsupervised Cross-lingual Representation Learning for Speech Recognition`{=html}](https://arxiv.org/abs/2006.13979#facebook "'Unsupervised Cross-lingual Representation Learning for Speech Recognition', Conneau et al 2020"){.link-annotated .id-not .include-annotation link-icon="facebook" link-icon-type="svg" link-icon-color="#1877f2"}** ## `“Logarithmic Pruning Is All You Need”, Orseau et al 2020`{=html} {#orseau-et-al-2020-section} **[`Logarithmic Pruning is All You Need`{=html}](https://arxiv.org/abs/2006.12156#deepmind "'Logarithmic Pruning is All You Need', Orseau et al 2020"){.link-annotated .id-not .include-annotation link-icon="deepmind" link-icon-type="svg" link-icon-color="#4185f4"}** ## `“Wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations”, Baevski et al 2020`{=html} {#baevski-et-al-2020-section} **[`wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations`{=html}](https://arxiv.org/abs/2006.11477#facebook "'wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations', Baevski et al 2020"){.link-annotated .id-not .include-annotation link-icon="facebook" link-icon-type="svg" link-icon-color="#1877f2"}** ## `“Denoising Diffusion Probabilistic Models”, Ho et al 2020`{=html} {#ho-et-al-2020-3-section} **[`Denoising Diffusion Probabilistic Models`{=html}](https://arxiv.org/abs/2006.11239 "'Denoising Diffusion Probabilistic Models', Ho et al 2020"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“On the Predictability of Pruning Across Scales”, Rosenfeld et al 2020`{=html} {#rosenfeld-et-al-2020-section} **[`On the Predictability of Pruning Across Scales`{=html}](https://arxiv.org/abs/2006.10621 "'On the Predictability of Pruning Across Scales', Rosenfeld et al 2020"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“IGPT: Generative Pretraining from Pixels”, Chen et al 2020`{=html} {#chen-igpt-paper-section} **[`iGPT: Generative Pretraining from Pixels`{=html}](/doc/ai/nn/transformer/gpt/dall-e/1/2020-chen-2.pdf#openai "'iGPT: Generative Pretraining from Pixels', Chen et al 2020"){.link-annotated .id-not .include-annotation link-icon="openai" link-icon-type="svg"}** ## `“SwAV: Unsupervised Learning of Visual Features by Contrasting Cluster Assignments”, Caron et al 2020`{=html} {#caron-et-al-2020-section} **[`SwAV: Unsupervised Learning of Visual Features by Contrasting Cluster Assignments`{=html}](https://arxiv.org/abs/2006.09882#facebook "'SwAV: Unsupervised Learning of Visual Features by Contrasting Cluster Assignments', Caron et al 2020"){.link-annotated .id-not .include-annotation link-icon="facebook" link-icon-type="svg" link-icon-color="#1877f2"}** ## `“SimCLRv2: Big Self-Supervised Models Are Strong Semi-Supervised Learners”, Chen et al 2020`{=html} {#chen-et-al-2020-simclrv2-section} **[`SimCLRv2: Big Self-Supervised Models are Strong Semi-Supervised Learners`{=html}](https://arxiv.org/abs/2006.10029#google "'SimCLRv2: Big Self-Supervised Models are Strong Semi-Supervised Learners', Chen et al 2020"){.link-annotated .id-not .include-annotation link-icon="alphabet" link-icon-type="svg" link-icon-color="#4285f4"}** ## `“Image GPT (IGPT): We Find That, Just As a Large Transformer Model Trained on Language Can Generate Coherent Text, the Same Exact Model Trained on Pixel Sequences Can Generate Coherent Image Completions and Samples”, Chen et al 2020`{=html} {#chen-et-al-2020-blog-section} **[`Image GPT (iGPT): We find that, just as a large transformer model trained on language can generate coherent text, the same exact model trained on pixel sequences can generate coherent image completions and samples`{=html}](https://openai.com/index/image-gpt/ "'Image GPT (iGPT): We find that, just as a large transformer model trained on language can generate coherent text, the same exact model trained on pixel sequences can generate coherent image completions and samples', Chen et al 2020"){.link-annotated .id-not .include-annotation link-icon="openai" link-icon-type="svg"}** ## `“Are We Done With ImageNet?”, Beyer et al 2020`{=html} {#beyer-et-al-2020-section} **[`Are we done with ImageNet?`{=html}](https://arxiv.org/abs/2006.07159#google "'Are we done with ImageNet?', Beyer et al 2020"){.link-annotated .id-not .include-annotation link-icon="alphabet" link-icon-type="svg" link-icon-color="#4285f4"}** ## `“OpenAI API”, Brockman et al 2020`{=html} {#brockman-et-al-2020-section} **[`OpenAI API`{=html}](https://openai.com/blog/openai-api/ "'OpenAI API', Brockman et al 2020"){.link-annotated .id-not .include-annotation link-icon="openai" link-icon-type="svg"}** ## `“Object Segmentation Without Labels With Large-Scale Generative Models”, Voynov et al 2020`{=html} {#voynov-et-al-2020-section} **[`Object Segmentation Without Labels with Large-Scale Generative Models`{=html}](https://arxiv.org/abs/2006.04988 "'Object Segmentation Without Labels with Large-Scale Generative Models', Voynov et al 2020"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“How Big Should My Language Model Be?”, Scao 2020`{=html} {#scao-2020-section} **[`How Big Should My Language Model Be?`{=html}](https://huggingface.co/spaces/teven-projects/calculator "'How Big Should My Language Model Be?', Scao 2020"){.link-annotated .id-not .include-annotation link-icon="🤗" link-icon-type="text" link-icon-color="#ffcc4d"}** ## `“GPT-3 Paper § Figure F.1: Four Uncurated Completions from a Context Suggesting the Model Compose a Poem in the Style of Wallace Stevens With the Title ‘Shadows on the Way’”, GPT-3 2020 (page 48)`{=html} {#gpt-3-2020-page-48-section} **[`GPT-3 paper § Figure F.1: Four uncurated completions from a context suggesting the model compose a poem in the style of Wallace Stevens with the title ‘Shadows on the Way’`{=html}](https://arxiv.org/pdf/2005.14165.pdf&org=openai#page=48 "'GPT-3 paper § Figure F.1: Four uncurated completions from a context suggesting the model compose a poem in the style of Wallace Stevens with the title ‘Shadows on the Way’', GPT-3 2020 (page 48)"){.link-annotated .id-not .include-annotation link-icon="openai" link-icon-type="svg"}** ## `“Danny Hernandez on Forecasting and the Drivers of AI Progress”, Koehler et al 2020`{=html} {#koehler-et-al-2020-section} **[`Danny Hernandez on forecasting and the drivers of AI progress`{=html}](https://80000hours.org/podcast/episodes/danny-hernandez-forecasting-ai-progress/ "'Danny Hernandez on forecasting and the drivers of AI progress', Koehler et al 2020"){.link-annotated .id-not .include-annotation link-icon="80k" link-icon-type="text,tri,sans" link-icon-color="#2ebdd1"}** ## `“Powered by AI: Advancing Product Understanding and Building New Shopping Experiences”, Berg et al 2020`{=html} {#berg-et-al-2020-section} **[`Powered by AI: Advancing product understanding and building new shopping experiences`{=html}](https://ai.meta.com/blog/powered-by-ai-advancing-product-understanding-and-building-new-shopping-experiences/ "'Powered by AI: Advancing product understanding and building new shopping experiences', Berg et al 2020"){.link-annotated .id-not .include-annotation link-icon="facebook" link-icon-type="svg" link-icon-color="#1877f2"}** ## `“ZeRO-2 & DeepSpeed: Shattering Barriers of Deep Learning Speed & Scale”, Team 2020`{=html} {#team-2020-1-section} **[`ZeRO-2 & DeepSpeed: Shattering barriers of deep learning speed & scale`{=html}](https://www.microsoft.com/en-us/research/blog/zero-2-deepspeed-shattering-barriers-of-deep-learning-speed-scale/ "'ZeRO-2 & DeepSpeed: Shattering barriers of deep learning speed & scale', Team 2020"){.link-annotated .id-not .include-annotation link-icon="MS" link-icon-type="text,sans,italic"}** ## `“Measuring the Algorithmic Efficiency of Neural Networks”, Hernandez & Brown 2020`{=html} {#hernandez-brown-2020-paper-section} **[`Measuring the Algorithmic Efficiency of Neural Networks`{=html}](https://arxiv.org/abs/2005.04305#openai "'Measuring the Algorithmic Efficiency of Neural Networks', Hernandez & Brown 2020"){.link-annotated .id-not .include-annotation link-icon="openai" link-icon-type="svg"}** ## `“Pushing the Limit of Molecular Dynamics With ab Initio Accuracy to 100 Million Atoms With Machine Learning”, Jia et al 2020`{=html} {#jia-et-al-2020-section} **[`Pushing the limit of molecular dynamics with ab initio accuracy to 100 million atoms with machine learning`{=html}](https://arxiv.org/abs/2005.00223 "'Pushing the limit of molecular dynamics with ab initio accuracy to 100 million atoms with machine learning', Jia et al 2020"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Jukebox: We’re Introducing Jukebox, a Neural Net That Generates Music, including Rudimentary Singing, As Raw Audio in a Variety of Genres and Artist Styles. We’re Releasing the Model Weights and Code, along With a Tool to Explore the Generated Samples.”, Dhariwal et al 2020`{=html} {#jukebox-blog-section} **[`Jukebox: We’re introducing Jukebox, a neural net that generates music, including rudimentary singing, as raw audio in a variety of genres and artist styles. We’re releasing the model weights and code, along with a tool to explore the generated samples.`{=html}](https://openai.com/research/jukebox "'Jukebox: We’re introducing Jukebox, a neural net that generates music, including rudimentary singing, as raw audio in a variety of genres and artist styles. We’re releasing the model weights and code, along with a tool to explore the generated samples.', Dhariwal et al 2020"){.link-annotated .id-not .include-annotation link-icon="openai" link-icon-type="svg"}** ## `“Blender: A State-Of-The-Art Open Source Chatbot”, Roller et al 2020`{=html} {#blender-blog-section} **[`Blender: A state-of-the-art open source chatbot`{=html}](https://ai.meta.com/blog/state-of-the-art-open-source-chatbot/ "'Blender: A state-of-the-art open source chatbot', Roller et al 2020"){.link-annotated .id-not .include-annotation link-icon="facebook" link-icon-type="svg" link-icon-color="#1877f2"}** ## `“A Review of Winograd Schema Challenge Datasets and Approaches”, Kocijan et al 2020`{=html} {#kocijan-et-al-2020-section} **[`A Review of Winograd Schema Challenge Datasets and Approaches`{=html}](https://arxiv.org/abs/2004.13831 "'A Review of Winograd Schema Challenge Datasets and Approaches', Kocijan et al 2020"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Scaling Laws from the Data Manifold Dimension”, Sharma & Kaplan 2020`{=html} {#sharma-kaplan-2020-section} **[`Scaling Laws from the Data Manifold Dimension`{=html}](https://arxiv.org/abs/2004.10802 "'Scaling Laws from the Data Manifold Dimension', Sharma & Kaplan 2020"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“DynamicEmbedding: Extending TensorFlow for Colossal-Scale Applications”, Zeng et al 2020`{=html} {#zeng-et-al-2020-section} **[`DynamicEmbedding: Extending TensorFlow for Colossal-Scale Applications`{=html}](https://arxiv.org/abs/2004.08366#google "'DynamicEmbedding: Extending TensorFlow for Colossal-Scale Applications', Zeng et al 2020"){.link-annotated .id-not .include-annotation link-icon="alphabet" link-icon-type="svg" link-icon-color="#4285f4"}** ## `“PALM: Pre-Training an Autoencoding & Autoregressive Language Model for Context-Conditioned Generation”, Bi et al 2020`{=html} {#bi-et-al-2020-section} **[`PALM: Pre-training an Autoencoding & Autoregressive Language Model for Context-conditioned Generation`{=html}](https://arxiv.org/abs/2004.07159#alibaba "'PALM: Pre-training an Autoencoding & Autoregressive Language Model for Context-conditioned Generation', Bi et al 2020"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Deep Learning Training in Facebook Data Centers: Design of Scale-Up and Scale-Out Systems”, Naumov et al 2020`{=html} {#naumov-et-al-2020-section} **[`Deep Learning Training in Facebook Data Centers: Design of Scale-up and Scale-out Systems`{=html}](https://arxiv.org/abs/2003.09518#facebook "'Deep Learning Training in Facebook Data Centers: Design of Scale-up and Scale-out Systems', Naumov et al 2020"){.link-annotated .id-not .include-annotation link-icon="facebook" link-icon-type="svg" link-icon-color="#1877f2"}** ## `“TTTTTackling WinoGrande Schemas”, Lin et al 2020`{=html} {#lin-et-al-2020-4-section} **[`TTTTTackling WinoGrande Schemas`{=html}](https://arxiv.org/abs/2003.08380#google "'TTTTTackling WinoGrande Schemas', Lin et al 2020"){.link-annotated .id-not .include-annotation link-icon="alphabet" link-icon-type="svg" link-icon-color="#4285f4"}** ## `“A Metric Learning Reality Check”, Musgrave et al 2020`{=html} {#musgrave-et-al-2020-section} **[`A Metric Learning Reality Check`{=html}](https://arxiv.org/abs/2003.08505 "'A Metric Learning Reality Check', Musgrave et al 2020"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Zoom In: An Introduction to Circuits—By Studying the Connections between Neurons, We Can Find Meaningful Algorithms in the Weights of Neural Networks”, Olah et al 2020`{=html} {#olah-et-al-2020-section} **[`Zoom In: An Introduction to Circuits—By studying the connections between neurons, we can find meaningful algorithms in the weights of neural networks`{=html}](https://distill.pub/2020/circuits/zoom-in/#openai "'Zoom In: An Introduction to Circuits—By studying the connections between neurons, we can find meaningful algorithms in the weights of neural networks', Olah et al 2020"){.link-annotated .id-not .include-annotation link-icon="openai" link-icon-type="svg"}** ## `“Rethinking Parameter Counting in Deep Models: Effective Dimensionality Revisited”, Maddox et al 2020`{=html} {#maddox-et-al-2020-section} **[`Rethinking Parameter Counting in Deep Models: Effective Dimensionality Revisited`{=html}](https://arxiv.org/abs/2003.02139 "'Rethinking Parameter Counting in Deep Models: Effective Dimensionality Revisited', Maddox et al 2020"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Rethinking Bias-Variance Trade-Off for Generalization of Neural Networks”, Yang et al 2020`{=html} {#yang-et-al-2020-6-section} **[`Rethinking Bias-Variance Trade-off for Generalization of Neural Networks`{=html}](https://arxiv.org/abs/2002.11328 "'Rethinking Bias-Variance Trade-off for Generalization of Neural Networks', Yang et al 2020"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Train Large, Then Compress: Rethinking Model Size for Efficient Training and Inference of Transformers”, Li et al 2020`{=html} {#li-et-al-2020-5-section} **[`Train Large, Then Compress: Rethinking Model Size for Efficient Training and Inference of Transformers`{=html}](https://arxiv.org/abs/2002.11794 "'Train Large, Then Compress: Rethinking Model Size for Efficient Training and Inference of Transformers', Li et al 2020"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“The Messy, Secretive Reality behind OpenAI’s Bid to save the World: The AI Moonshot Was Founded in the Spirit of Transparency. This Is the inside Story of How Competitive Pressure Eroded That Idealism”, Hao 2020`{=html} {#hao-2020-section} **[`The messy, secretive reality behind OpenAI’s bid to save the world: The AI moonshot was founded in the spirit of transparency. This is the inside story of how competitive pressure eroded that idealism`{=html}](https://www.technologyreview.com/2020/02/17/844721/ai-openai-moonshot-elon-musk-sam-altman-greg-brockman-messy-secretive-reality/ "'The messy, secretive reality behind OpenAI’s bid to save the world: The AI moonshot was founded in the spirit of transparency. This is the inside story of how competitive pressure eroded that idealism', Hao 2020"){.link-annotated .id-not .include-annotation link-icon="T" link-icon-type="text,sans"}** ## `“Pitfalls of In-Domain Uncertainty Estimation and Ensembling in Deep Learning”, Ashukha et al 2020`{=html} {#ashukha-et-al-2020-section} **[`Pitfalls of In-Domain Uncertainty Estimation and Ensembling in Deep Learning`{=html}](https://arxiv.org/abs/2002.06470 "'Pitfalls of In-Domain Uncertainty Estimation and Ensembling in Deep Learning', Ashukha et al 2020"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“The Next Decade in AI: Four Steps Towards Robust Artificial Intelligence”, Marcus 2020`{=html} {#marcus-2020-section} **[`The Next Decade in AI: Four Steps Towards Robust Artificial Intelligence`{=html}](https://arxiv.org/abs/2002.06177 "'The Next Decade in AI: Four Steps Towards Robust Artificial Intelligence', Marcus 2020"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“A Simple Framework for Contrastive Learning of Visual Representations”, Chen et al 2020`{=html} {#chen-et-al-2020-simclrv1-section} **[`A Simple Framework for Contrastive Learning of Visual Representations`{=html}](https://arxiv.org/abs/2002.05709#google "'A Simple Framework for Contrastive Learning of Visual Representations', Chen et al 2020"){.link-annotated .id-not .include-annotation link-icon="alphabet" link-icon-type="svg" link-icon-color="#4285f4"}** ## `“How Much Knowledge Can You Pack Into the Parameters of a Language Model?”, Roberts et al 2020`{=html} {#roberts-et-al-2020-2-section} **[`How Much Knowledge Can You Pack Into the Parameters of a Language Model?`{=html}](https://arxiv.org/abs/2002.08910#google "'How Much Knowledge Can You Pack Into the Parameters of a Language Model?', Roberts et al 2020"){.link-annotated .id-not .include-annotation link-icon="alphabet" link-icon-type="svg" link-icon-color="#4285f4"}** ## `“Turing-NLG: A 17-Billion-Parameter Language Model by Microsoft”, Rosset 2020`{=html} {#rosset-2020-section} **[`Turing-NLG: A 17-billion-parameter language model by Microsoft`{=html}](https://www.microsoft.com/en-us/research/blog/turing-nlg-a-17-billion-parameter-language-model-by-microsoft/ "'Turing-NLG: A 17-billion-parameter language model by Microsoft', Rosset 2020"){.link-annotated .id-not .include-annotation link-icon="MS" link-icon-type="text,sans,italic"}** ## `“Quasi-Equivalence of Width and Depth of Neural Networks”, Fan et al 2020`{=html} {#fan-et-al-2020-4-section} **[`Quasi-Equivalence of Width and Depth of Neural Networks`{=html}](https://arxiv.org/abs/2002.02515 "'Quasi-Equivalence of Width and Depth of Neural Networks', Fan et al 2020"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Impact of ImageNet Model Selection on Domain Adaptation”, Zhang & Davison 2020`{=html} {#zhang-davison-2020-section} **[`Impact of ImageNet Model Selection on Domain Adaptation`{=html}](https://arxiv.org/abs/2002.02559 "'Impact of ImageNet Model Selection on Domain Adaptation', Zhang & Davison 2020"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Direct Fit to Nature: An Evolutionary Perspective on Biological and Artificial Neural Networks”, Hasson et al 2020`{=html} {#hasson-et-al-2020-section} **[`Direct Fit to Nature: An Evolutionary Perspective on Biological and Artificial Neural Networks`{=html}](/doc/ai/scaling/2020-hasson.pdf "'Direct Fit to Nature: An Evolutionary Perspective on Biological and Artificial Neural Networks', Hasson et al 2020"){.link-annotated .id-not .include-annotation link-icon="pdf" link-icon-type="svg" link-icon-color="#f40f02"}** ## `“Towards a Conversational Agent That Can Chat About…Anything”, Adiwardana & Luong 2020`{=html} {#adiwardana-luong-2020-section} **[`Towards a Conversational Agent that Can Chat About…Anything`{=html}](https://research.google/blog/towards-a-conversational-agent-that-can-chat-aboutanything/ "'Towards a Conversational Agent that Can Chat About…Anything', Adiwardana & Luong 2020"){.link-annotated .id-not .include-annotation link-icon="alphabet" link-icon-type="svg" link-icon-color="#4285f4"}** ## `“Towards a Human-Like Open-Domain Chatbot”, Adiwardana et al 2020`{=html} {#adiwardana-et-al-2020-section} **[`Towards a Human-like Open-Domain Chatbot`{=html}](https://arxiv.org/abs/2001.09977#google "'Towards a Human-like Open-Domain Chatbot', Adiwardana et al 2020"){.link-annotated .id-not .include-annotation link-icon="alphabet" link-icon-type="svg" link-icon-color="#4285f4"}** ## `“Scaling Laws for Neural Language Models”, Kaplan et al 2020`{=html} {#kaplan-et-al-2020-section} **[`Scaling Laws for Neural Language Models`{=html}](https://arxiv.org/abs/2001.08361#openai "'Scaling Laws for Neural Language Models', Kaplan et al 2020"){.link-annotated .id-not .include-annotation link-icon="openai" link-icon-type="svg"}** ## `“Scaling Laws for Neural Language Models: Figure 15: Far beyond the Model Sizes We Study Empirically, We Find a Contradiction between Our Equations § Pg17”, Kaplan 2020 (page 17 org openai)`{=html} {#kaplan-2020-page-17-org-openai-section} **[`Scaling Laws for Neural Language Models: Figure 15: Far beyond the model sizes we study empirically, we find a contradiction between our equations § pg17`{=html}](https://arxiv.org/pdf/2001.08361.pdf#page=17&org=openai "'Scaling Laws for Neural Language Models: Figure 15: Far beyond the model sizes we study empirically, we find a contradiction between our equations § pg17', Kaplan 2020 (page 17 org openai)"){.id-not .include-annotation link-icon="openai" link-icon-type="svg"}[  :]{.mobile-not}** ::: aux-links-transclude-file ::: {.collapse .mobile-not} **View PDF**: [`/doc/www/arxiv.org/20d126b9c3baf640f8d1d5dff3e253faac2e8242.pdf#page=17&org=openai`](/doc/www/arxiv.org/20d126b9c3baf640f8d1d5dff3e253faac2e8242.pdf#page=17&org=openai){.id-not .link-annotated-not .include-content .include-lazy link-icon="openai" link-icon-type="svg"} ::: ::: ## `“The Importance of Deconstruction”, Weinberger 2020`{=html} {#weinberger-2020-section} **[`The Importance of Deconstruction`{=html}](https://www.youtube.com/watch?v=kY2NHSKBi10 "'The Importance of Deconstruction', Weinberger 2020"){.link-annotated .id-not .include-annotation link-icon="youtube" link-icon-type="svg" link-icon-color="#ff0033"}** ## `“Big Transfer (BiT): General Visual Representation Learning”, Kolesnikov et al 2019`{=html} {#kolesnikov-et-al-2019-section} **[`Big Transfer (BiT): General Visual Representation Learning`{=html}](https://arxiv.org/abs/1912.11370#google "'Big Transfer (BiT): General Visual Representation Learning', Kolesnikov et al 2019"){.link-annotated .id-not .include-annotation link-icon="alphabet" link-icon-type="svg" link-icon-color="#4285f4"}** ## `“More Data Can Hurt for Linear Regression: Sample-Wise Double Descent”, Nakkiran 2019`{=html} {#nakkiran-2019-section} **[`More Data Can Hurt for Linear Regression: Sample-wise Double Descent`{=html}](https://arxiv.org/abs/1912.07242 "'More Data Can Hurt for Linear Regression: Sample-wise Double Descent', Nakkiran 2019"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“12-In-1: Multi-Task Vision and Language Representation Learning”, Lu et al 2019`{=html} {#lu-et-al-2019-2-section} **[`12-in-1: Multi-Task Vision and Language Representation Learning`{=html}](https://arxiv.org/abs/1912.02315#facebook "'12-in-1: Multi-Task Vision and Language Representation Learning', Lu et al 2019"){.link-annotated .id-not .include-annotation link-icon="facebook" link-icon-type="svg" link-icon-color="#1877f2"}** ## `“Deep Double Descent: We Show That the Double Descent Phenomenon Occurs in CNNs, ResNets, and Transformers: Performance First Improves, Then Gets Worse, and Then Improves Again With Increasing Model Size, Data Size, or Training Time”, Nakkiran et al 2019`{=html} {#nakkiran-et-al-2019-1-section} **[`Deep Double Descent: We show that the double descent phenomenon occurs in CNNs, ResNets, and transformers: performance first improves, then gets worse, and then improves again with increasing model size, data size, or training time`{=html}](https://openai.com/research/deep-double-descent "'Deep Double Descent: We show that the double descent phenomenon occurs in CNNs, ResNets, and transformers: performance first improves, then gets worse, and then improves again with increasing model size, data size, or training time', Nakkiran et al 2019"){.link-annotated .id-not .include-annotation link-icon="openai" link-icon-type="svg"}** ## `“Deep Double Descent: Where Bigger Models and More Data Hurt”, Nakkiran et al 2019`{=html} {#nakkiran-et-al-2019-2-section} **[`Deep Double Descent: Where Bigger Models and More Data Hurt`{=html}](https://arxiv.org/abs/1912.02292#openai "'Deep Double Descent: Where Bigger Models and More Data Hurt', Nakkiran et al 2019"){.link-annotated .id-not .include-annotation link-icon="openai" link-icon-type="svg"}** ## `“What’s Hidden in a Randomly Weighted Neural Network?”, Ramanujan et al 2019`{=html} {#ramanujan-et-al-2019-section} **[`What’s Hidden in a Randomly Weighted Neural Network?`{=html}](https://arxiv.org/abs/1911.13299 "'What’s Hidden in a Randomly Weighted Neural Network?', Ramanujan et al 2019"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Understanding the Generalization of ‘Lottery Tickets’ in Neural Networks”, Morcos & Tian 2019`{=html} {#morcos-tian-2019-section} **[`Understanding the generalization of ‘lottery tickets’ in neural networks`{=html}](https://ai.meta.com/blog/understanding-the-generalization-of-lottery-tickets-in-neural-networks/ "'Understanding the generalization of ‘lottery tickets’ in neural networks', Morcos & Tian 2019"){.link-annotated .id-not .include-annotation link-icon="facebook" link-icon-type="svg" link-icon-color="#1877f2"}** ## `“The Deep Learning Revolution and Its Implications for Computer Architecture and Chip Design”, Dean 2019`{=html} {#dean-2019-section} **[`The Deep Learning Revolution and Its Implications for Computer Architecture and Chip Design`{=html}](https://arxiv.org/abs/1911.05289#google "'The Deep Learning Revolution and Its Implications for Computer Architecture and Chip Design', Dean 2019"){.link-annotated .id-not .include-annotation link-icon="alphabet" link-icon-type="svg" link-icon-color="#4285f4"}** ## `“Momentum Contrast for Unsupervised Visual Representation Learning”, He et al 2019`{=html} {#he-et-al-2019-1-section} **[`Momentum Contrast for Unsupervised Visual Representation Learning`{=html}](https://arxiv.org/abs/1911.05722#facebook "'Momentum Contrast for Unsupervised Visual Representation Learning', He et al 2019"){.link-annotated .id-not .include-annotation link-icon="facebook" link-icon-type="svg" link-icon-color="#1877f2"}** ## `“SimpleShot: Revisiting Nearest-Neighbor Classification for Few-Shot Learning”, Wang et al 2019`{=html} {#wang-et-al-2019-section} **[`SimpleShot: Revisiting Nearest-Neighbor Classification for Few-Shot Learning`{=html}](https://arxiv.org/abs/1911.04623 "'SimpleShot: Revisiting Nearest-Neighbor Classification for Few-Shot Learning', Wang et al 2019"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Self-Training With Noisy Student Improves ImageNet Classification”, Xie et al 2019`{=html} {#xie-et-al-2019-2-section} **[`Self-training with Noisy Student improves ImageNet classification`{=html}](https://arxiv.org/abs/1911.04252#google "'Self-training with Noisy Student improves ImageNet classification', Xie et al 2019"){.link-annotated .id-not .include-annotation link-icon="alphabet" link-icon-type="svg" link-icon-color="#4285f4"}** ## `“CCMatrix: Mining Billions of High-Quality Parallel Sentences on the WEB”, Schwenk et al 2019`{=html} {#schwenk-et-al-2019-section} **[`CCMatrix: Mining Billions of High-Quality Parallel Sentences on the WEB`{=html}](https://arxiv.org/abs/1911.04944#facebook "'CCMatrix: Mining Billions of High-Quality Parallel Sentences on the WEB', Schwenk et al 2019"){.link-annotated .id-not .include-annotation link-icon="facebook" link-icon-type="svg" link-icon-color="#1877f2"}** ## `“CCAligned: A Massive Collection of Cross-Lingual Web-Document Pairs”, El-Kishky et al 2019`{=html} {#el-kishky-et-al-2019-section} **[`CCAligned: A Massive Collection of Cross-Lingual Web-Document Pairs`{=html}](https://arxiv.org/abs/1911.06154#facebook "'CCAligned: A Massive Collection of Cross-Lingual Web-Document Pairs', El-Kishky et al 2019"){.link-annotated .id-not .include-annotation link-icon="facebook" link-icon-type="svg" link-icon-color="#1877f2"}** ## `“XLM-R: State-Of-The-Art Cross-Lingual Understanding through Self-Supervision”, FAIR 2019`{=html} {#fair-2019-section} **[`XLM-R: State-of-the-art cross-lingual understanding through self-supervision`{=html}](https://ai.meta.com/blog/-xlm-r-state-of-the-art-cross-lingual-understanding-through-self-supervision/ "'XLM-R: State-of-the-art cross-lingual understanding through self-supervision', FAIR 2019"){.link-annotated .id-not .include-annotation link-icon="facebook" link-icon-type="svg" link-icon-color="#1877f2"}** ## `“High Fidelity Video Prediction With Large Stochastic Recurrent Neural Networks”, Villegas et al 2019`{=html} {#villegas-et-al-2019-section} **[`High Fidelity Video Prediction with Large Stochastic Recurrent Neural Networks`{=html}](https://arxiv.org/abs/1911.01655#google "'High Fidelity Video Prediction with Large Stochastic Recurrent Neural Networks', Villegas et al 2019"){.link-annotated .id-not .include-annotation link-icon="alphabet" link-icon-type="svg" link-icon-color="#4285f4"}** ## `“Unsupervised Cross-Lingual Representation Learning at Scale”, Conneau et al 2019`{=html} {#conneau-et-al-2019-section} **[`Unsupervised Cross-lingual Representation Learning at Scale`{=html}](https://arxiv.org/abs/1911.02116#facebook "'Unsupervised Cross-lingual Representation Learning at Scale', Conneau et al 2019"){.link-annotated .id-not .include-annotation link-icon="facebook" link-icon-type="svg" link-icon-color="#1877f2"}** ## `“T5: Exploring the Limits of Transfer Learning With a Unified Text-To-Text Transformer”, Raffel et al 2019`{=html} {#raffel-et-al-2019-section} **[`T5: Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer`{=html}](https://arxiv.org/abs/1910.10683#google "'T5: Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer', Raffel et al 2019"){.link-annotated .id-not .include-annotation link-icon="alphabet" link-icon-type="svg" link-icon-color="#4285f4"}** ## `“ZeRO: Memory Optimizations Toward Training Trillion Parameter Models”, Rajbhandari et al 2019`{=html} {#rajbhandari-et-al-2019-section} **[`ZeRO: Memory Optimizations Toward Training Trillion Parameter Models`{=html}](https://arxiv.org/abs/1910.02054#microsoft "'ZeRO: Memory Optimizations Toward Training Trillion Parameter Models', Rajbhandari et al 2019"){.link-annotated .id-not .include-annotation link-icon="MS" link-icon-type="text,sans,italic"}** ## `“Environmental Drivers of Systematicity and Generalization in a Situated Agent”, Hill et al 2019`{=html} {#hill-et-al-2019-2-section} **[`Environmental drivers of systematicity and generalization in a situated agent`{=html}](https://arxiv.org/abs/1910.00571#deepmind "'Environmental drivers of systematicity and generalization in a situated agent', Hill et al 2019"){.link-annotated .id-not .include-annotation link-icon="deepmind" link-icon-type="svg" link-icon-color="#4185f4"}** ## `“A Constructive Prediction of the Generalization Error Across Scales”, Rosenfeld et al 2019`{=html} {#rosenfeld-et-al-2019-section} **[`A Constructive Prediction of the Generalization Error Across Scales`{=html}](https://arxiv.org/abs/1909.12673 "'A Constructive Prediction of the Generalization Error Across Scales', Rosenfeld et al 2019"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Large-Scale Pretraining for Neural Machine Translation With Tens of Billions of Sentence Pairs”, Meng et al 2019`{=html} {#meng-et-al-2019-section} **[`Large-scale Pretraining for Neural Machine Translation with Tens of Billions of Sentence Pairs`{=html}](https://arxiv.org/abs/1909.11861 "'Large-scale Pretraining for Neural Machine Translation with Tens of Billions of Sentence Pairs', Meng et al 2019"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“UNITER: UNiversal Image-TExt Representation Learning”, Chen et al 2019`{=html} {#chen-et-al-2019-3-section} **[`UNITER: UNiversal Image-TExt Representation Learning`{=html}](https://arxiv.org/abs/1909.11740 "'UNITER: UNiversal Image-TExt Representation Learning', Chen et al 2019"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Exascale Deep Learning for Scientific Inverse Problems”, Laanait et al 2019`{=html} {#laanait-et-al-2019-section} **[`Exascale Deep Learning for Scientific Inverse Problems`{=html}](https://arxiv.org/abs/1909.11150 "'Exascale Deep Learning for Scientific Inverse Problems', Laanait et al 2019"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Simple, Scalable Adaptation for Neural Machine Translation”, Bapna et al 2019`{=html} {#bapna-et-al-2019-section} **[`Simple, Scalable Adaptation for Neural Machine Translation`{=html}](https://arxiv.org/abs/1909.08478#google "'Simple, Scalable Adaptation for Neural Machine Translation', Bapna et al 2019"){.link-annotated .id-not .include-annotation link-icon="alphabet" link-icon-type="svg" link-icon-color="#4285f4"}** ## `“CTRL: A Conditional Transformer Language Model For Controllable Generation”, Keskar et al 2019`{=html} {#keskar-et-al-2019-section} **[`CTRL: A Conditional Transformer Language Model For Controllable Generation`{=html}](https://arxiv.org/abs/1909.05858#salesforce "'CTRL: A Conditional Transformer Language Model For Controllable Generation', Keskar et al 2019"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Show Your Work: Improved Reporting of Experimental Results”, Dodge et al 2019`{=html} {#dodge-et-al-2019-section} **[`Show Your Work: Improved Reporting of Experimental Results`{=html}](https://arxiv.org/abs/1909.03004 "'Show Your Work: Improved Reporting of Experimental Results', Dodge et al 2019"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“MegatronLM: Training Billion+ Parameter Language Models Using GPU Model Parallelism”, ADLR 2019`{=html} {#adlr-2019-section} **[`MegatronLM: Training Billion+ Parameter Language Models Using GPU Model Parallelism`{=html}](https://nv-adlr.github.io/MegatronLM "'MegatronLM: Training Billion+ Parameter Language Models Using GPU Model Parallelism', ADLR 2019"){.link-annotated .id-not .include-annotation link-icon="n" link-icon-type="text,sans,italic" link-icon-color="#77ba00"}** ## `“RoBERTa: A Robustly Optimized BERT Pretraining Approach”, Liu et al 2019`{=html} {#liu-et-al-2019-roberta-section} **[`RoBERTa: A Robustly Optimized BERT Pretraining Approach`{=html}](https://arxiv.org/abs/1907.11692#facebook "'RoBERTa: A Robustly Optimized BERT Pretraining Approach', Liu et al 2019"){.link-annotated .id-not .include-annotation link-icon="facebook" link-icon-type="svg" link-icon-color="#1877f2"}** ## `“Robustness Properties of Facebook’s ResNeXt WSL Models”, Orhan 2019`{=html} {#orhan-2019-section} **[`Robustness properties of Facebook’s ResNeXt WSL models`{=html}](https://arxiv.org/abs/1907.07640 "'Robustness properties of Facebook’s ResNeXt WSL models', Orhan 2019"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Massively Multilingual Neural Machine Translation in the Wild: Findings and Challenges”, Arivazhagan et al 2019`{=html} {#arivazhagan-et-al-2019-section} **[`Massively Multilingual Neural Machine Translation in the Wild: Findings and Challenges`{=html}](https://arxiv.org/abs/1907.05019#google "'Massively Multilingual Neural Machine Translation in the Wild: Findings and Challenges', Arivazhagan et al 2019"){.link-annotated .id-not .include-annotation link-icon="alphabet" link-icon-type="svg" link-icon-color="#4285f4"}** ## `“Large Scale Adversarial Representation Learning”, Donahue & Simonyan 2019`{=html} {#donahue-simonyan-2019-section} **[`Large Scale Adversarial Representation Learning`{=html}](https://arxiv.org/abs/1907.02544 "'Large Scale Adversarial Representation Learning', Donahue & Simonyan 2019"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“One Epoch Is All You Need”, Komatsuzaki 2019`{=html} {#komatsuzaki-2019-section} **[`One Epoch Is All You Need`{=html}](https://arxiv.org/abs/1906.06669 "'One Epoch Is All You Need', Komatsuzaki 2019"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Does Learning Require Memorization? A Short Tale about a Long Tail”, Feldman 2019`{=html} {#feldman-2019-section} **[`Does Learning Require Memorization? A Short Tale about a Long Tail`{=html}](https://arxiv.org/abs/1906.05271#google "'Does Learning Require Memorization? A Short Tale about a Long Tail', Feldman 2019"){.link-annotated .id-not .include-annotation link-icon="alphabet" link-icon-type="svg" link-icon-color="#4285f4"}** ## `“Intriguing Properties of Adversarial Training at Scale”, Xie & Yuille 2019`{=html} {#xie-yuille-2019-section} **[`Intriguing properties of adversarial training at scale`{=html}](https://arxiv.org/abs/1906.03787 "'Intriguing properties of adversarial training at scale', Xie & Yuille 2019"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Scaling Autoregressive Video Models”, Weissenborn et al 2019`{=html} {#weissenborn-et-al-2019-section} **[`Scaling Autoregressive Video Models`{=html}](https://arxiv.org/abs/1906.02634#google "'Scaling Autoregressive Video Models', Weissenborn et al 2019"){.link-annotated .id-not .include-annotation link-icon="alphabet" link-icon-type="svg" link-icon-color="#4285f4"}** ## `“A Mathematical Theory of Semantic Development in Deep Neural Networks”, Saxe et al 2019`{=html} {#saxe-et-al-2019-section} **[`A mathematical theory of semantic development in deep neural networks`{=html}](https://www.pnas.org/doi/10.1073/pnas.1820226116 "'A mathematical theory of semantic development in deep neural networks', Saxe et al 2019"){.link-annotated .id-not .include-annotation link-icon="PNAS" link-icon-type="text,quad" link-icon-color="#1f75b9"}** ## `“Adversarially Robust Generalization Just Requires More Unlabeled Data”, Zhai et al 2019`{=html} {#zhai-et-al-2019-section} **[`Adversarially Robust Generalization Just Requires More Unlabeled Data`{=html}](https://arxiv.org/abs/1906.00555 "'Adversarially Robust Generalization Just Requires More Unlabeled Data', Zhai et al 2019"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“ICML 2019 Notes”, Abel 2019`{=html} {#abel-2019-section} **[`ICML 2019 Notes`{=html}](https://david-abel.github.io/notes/icml_2019.pdf "'ICML 2019 Notes', Abel 2019"){.link-annotated .id-not .include-annotation link-icon="pdf" link-icon-type="svg" link-icon-color="#f40f02"}** ## `“Are Labels Required for Improving Adversarial Robustness?”, Uesato et al 2019`{=html} {#uesato-et-al-2019-section} **[`Are Labels Required for Improving Adversarial Robustness?`{=html}](https://arxiv.org/abs/1905.13725 "'Are Labels Required for Improving Adversarial Robustness?', Uesato et al 2019"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks”, Tan & Le 2019`{=html} {#tan-le-2019-section} **[`EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks`{=html}](https://arxiv.org/abs/1905.11946#google "'EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks', Tan & Le 2019"){.link-annotated .id-not .include-annotation link-icon="alphabet" link-icon-type="svg" link-icon-color="#4285f4"}** ## `“SpArSe: Sparse Architecture Search for CNNs on Resource-Constrained Microcontrollers”, Fedorov et al 2019`{=html} {#fedorov-et-al-2019-section} **[`SpArSe: Sparse Architecture Search for CNNs on Resource-Constrained Microcontrollers`{=html}](https://arxiv.org/abs/1905.12107 "'SpArSe: Sparse Architecture Search for CNNs on Resource-Constrained Microcontrollers', Fedorov et al 2019"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Asymptotic Learning Curves of Kernel Methods: Empirical Data versus Teacher-Student Paradigm”, Spigler et al 2019`{=html} {#spigler-et-al-2019-section} **[`Asymptotic learning curves of kernel methods: empirical data versus Teacher-Student paradigm`{=html}](https://arxiv.org/abs/1905.10843 "'Asymptotic learning curves of kernel methods: empirical data versus Teacher-Student paradigm', Spigler et al 2019"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“UniLM: Unified Language Model Pre-Training for Natural Language Understanding and Generation”, Dong et al 2019`{=html} {#dong-et-al-2019-section} **[`UniLM: Unified Language Model Pre-training for Natural Language Understanding and Generation`{=html}](https://arxiv.org/abs/1905.03197 "'UniLM: Unified Language Model Pre-training for Natural Language Understanding and Generation', Dong et al 2019"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Adversarial Examples Are Not Bugs, They Are Features”, Ilyas et al 2019`{=html} {#ilyas-et-al-2019-section} **[`Adversarial Examples Are Not Bugs, They Are Features`{=html}](https://arxiv.org/abs/1905.02175 "'Adversarial Examples Are Not Bugs, They Are Features', Ilyas et al 2019"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Billion-Scale Semi-Supervised Learning for Image Classification”, Yalniz et al 2019`{=html} {#yalniz-et-al-2019-section} **[`Billion-scale semi-supervised learning for image classification`{=html}](https://arxiv.org/abs/1905.00546#facebook "'Billion-scale semi-supervised learning for image classification', Yalniz et al 2019"){.link-annotated .id-not .include-annotation link-icon="facebook" link-icon-type="svg" link-icon-color="#1877f2"}** ## `“VideoBERT: A Joint Model for Video and Language Representation Learning”, Sun et al 2019`{=html} {#sun-et-al-2019-2-section} **[`VideoBERT: A Joint Model for Video and Language Representation Learning`{=html}](https://arxiv.org/abs/1904.01766#google "'VideoBERT: A Joint Model for Video and Language Representation Learning', Sun et al 2019"){.link-annotated .id-not .include-annotation link-icon="alphabet" link-icon-type="svg" link-icon-color="#4285f4"}** ## `“Benchmarking Neural Network Robustness to Common Corruptions and Perturbations”, Hendrycks & Dietterich 2019`{=html} {#hendrycks-dietterich-2019-section} **[`Benchmarking Neural Network Robustness to Common Corruptions and Perturbations`{=html}](https://arxiv.org/abs/1903.12261 "'Benchmarking Neural Network Robustness to Common Corruptions and Perturbations', Hendrycks & Dietterich 2019"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Surprises in High-Dimensional Ridgeless Least Squares Interpolation”, Hastie et al 2019`{=html} {#hastie-et-al-2019-section} **[`Surprises in High-Dimensional Ridgeless Least Squares Interpolation`{=html}](https://arxiv.org/abs/1903.08560 "'Surprises in High-Dimensional Ridgeless Least Squares Interpolation', Hastie et al 2019"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“The Bitter Lesson”, Sutton 2019`{=html} {#sutton-2019-2-section} **[`The Bitter Lesson`{=html}](http://www.incompleteideas.net/IncIdeas/BitterLesson.html "'The Bitter Lesson', Sutton 2019"){.link-annotated .id-not .include-annotation}** ## `“GPT-2 As Step Toward General Intelligence”, Alexander 2019`{=html} {#alexander-2019-2-section} **[`GPT-2 As Step Toward General Intelligence`{=html}](https://slatestarcodex.com/2019/02/19/gpt-2-as-step-toward-general-intelligence/ "'GPT-2 As Step Toward General Intelligence', Alexander 2019"){.link-annotated .id-not .include-annotation link-icon="SSC" link-icon-type="text,tri" link-icon-color="#5175c2"}** ## `“Deep Learning Hardware: Past, Present, & Future”, LeCun 2019`{=html} {#lecun-2019-section} **[`Deep Learning Hardware: Past, Present, & Future`{=html}](/doc/ai/scaling/2019-02-18-lecun-isscc-talk-deeplearninghardwarepastpresentandfuture.pdf "'Deep Learning Hardware: Past, Present, & Future', LeCun 2019"){.link-annotated-partial .id-not .include-annotation link-icon="pdf" link-icon-type="svg" link-icon-color="#f40f02"}** ## `“Language Models Are Unsupervised Multitask Learners”, Radford et al 2019`{=html} {#gpt-2-paper-section} **[`Language Models are Unsupervised Multitask Learners`{=html}](/doc/ai/nn/transformer/gpt/2/2019-radford.pdf#openai "'Language Models are Unsupervised Multitask Learners', Radford et al 2019"){.link-annotated .id-not .include-annotation link-icon="openai" link-icon-type="svg"}** ## `“Better Language Models and Their Implications”, Radford et al 2019`{=html} {#gpt-2-blog-section} **[`Better Language Models and Their Implications`{=html}](https://openai.com/index/better-language-models/ "'Better Language Models and Their Implications', Radford et al 2019"){.link-annotated .id-not .include-annotation link-icon="openai" link-icon-type="svg"}** ## `“Do ImageNet Classifiers Generalize to ImageNet?”, Recht et al 2019`{=html} {#recht-et-al-2019-section} **[`Do ImageNet Classifiers Generalize to ImageNet?`{=html}](https://arxiv.org/abs/1902.10811 "'Do ImageNet Classifiers Generalize to ImageNet?', Recht et al 2019"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Cross-Lingual Language Model Pretraining”, Lample & Conneau 2019`{=html} {#lample-conneau-2019-section} **[`Cross-lingual Language Model Pretraining`{=html}](https://arxiv.org/abs/1901.07291#facebook "'Cross-lingual Language Model Pretraining', Lample & Conneau 2019"){.link-annotated .id-not .include-annotation link-icon="facebook" link-icon-type="svg" link-icon-color="#1877f2"}** ## `“Artificial Intelligence: A Guide for Thinking Humans § Prologue: Terrified”, Mitchell 2019`{=html} {#mitchell-2019-section} **[`Artificial Intelligence: A Guide for Thinking Humans § Prologue: Terrified`{=html}](https://melaniemitchell.me/aibook/ "'Artificial Intelligence: A Guide for Thinking Humans § Prologue: Terrified', Mitchell 2019"){.link-annotated .id-not .include-annotation}** ## `“High Fidelity Video Prediction With Large Stochastic Recurrent Neural Networks: Videos”, Villegas et al 2019`{=html} {#villegas-et-al-2019-2-section} **[`High Fidelity Video Prediction with Large Stochastic Recurrent Neural Networks: Videos`{=html}](https://sites.google.com/view/videopredictioncapacity "'High Fidelity Video Prediction with Large Stochastic Recurrent Neural Networks: Videos', Villegas et al 2019"){.link-annotated .id-not .include-annotation link-icon="alphabet" link-icon-type="svg" link-icon-color="#4285f4"}** ## `“Reconciling Modern Machine Learning Practice and the Bias-Variance Trade-Off”, Belkin et al 2018`{=html} {#belkin-et-al-2018-section} **[`Reconciling modern machine learning practice and the bias-variance trade-off`{=html}](https://arxiv.org/abs/1812.11118 "'Reconciling modern machine learning practice and the bias-variance trade-off', Belkin et al 2018"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Nocaps: Novel Object Captioning at Scale”, Agrawal et al 2018`{=html} {#agrawal-et-al-2018-section} **[`nocaps: novel object captioning at scale`{=html}](https://arxiv.org/abs/1812.08658 "'nocaps: novel object captioning at scale', Agrawal et al 2018"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“On Lazy Training in Differentiable Programming”, Chizat et al 2018`{=html} {#chizat-et-al-2018-section} **[`On Lazy Training in Differentiable Programming`{=html}](https://arxiv.org/abs/1812.07956 "'On Lazy Training in Differentiable Programming', Chizat et al 2018"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“How AI Training Scales”, McCandlish et al 2018`{=html} {#mccandlish-et-al-2018-section} **[`How AI Training Scales`{=html}](https://openai.com/research/how-ai-training-scales "'How AI Training Scales', McCandlish et al 2018"){.link-annotated .id-not .include-annotation link-icon="openai" link-icon-type="svg"}** ## `“Is Science Slowing Down?”, Alexander 2018`{=html} {#alexander-2018-2-section} **[`Is Science Slowing Down?`{=html}](https://slatestarcodex.com/2018/11/26/is-science-slowing-down-2/ "'Is Science Slowing Down?', Alexander 2018"){.link-annotated .id-not .include-annotation link-icon="SSC" link-icon-type="text,tri" link-icon-color="#5175c2"}** ## `“Large Scale GAN Training for High Fidelity Natural Image Synthesis”, Brock et al 2018`{=html} {#brock-et-al-2018-section} **[`Large Scale GAN Training for High Fidelity Natural Image Synthesis`{=html}](https://arxiv.org/abs/1809.11096#deepmind "'Large Scale GAN Training for High Fidelity Natural Image Synthesis', Brock et al 2018"){.link-annotated .id-not .include-annotation link-icon="deepmind" link-icon-type="svg" link-icon-color="#4185f4"}** ## `“BigGAN: Large Scale GAN Training For High Fidelity Natural Image Synthesis § 5.2 Additional Evaluation On JFT-300M”, Brock et al 2018 (page 8 org deepmind)`{=html} {#brock-et-al-2018-page-8-org-deepmind-section} **[`BigGAN: Large Scale GAN Training For High Fidelity Natural Image Synthesis § 5.2 Additional Evaluation On JFT-300M`{=html}](https://arxiv.org/pdf/1809.11096#page=8&org=deepmind "'BigGAN: Large Scale GAN Training For High Fidelity Natural Image Synthesis § 5.2 Additional Evaluation On JFT-300M', Brock et al 2018 (page 8 org deepmind)"){.link-annotated .id-not .include-annotation link-icon="deepmind" link-icon-type="svg" link-icon-color="#4185f4"}** ## `“Measurement Invariance Explains the Universal Law of Generalization for Psychological Perception”, Frank 2018`{=html} {#frank-2018-section} **[`Measurement invariance explains the universal law of generalization for psychological perception`{=html}](https://www.pnas.org/doi/10.1073/pnas.1809787115 "'Measurement invariance explains the universal law of generalization for psychological perception', Frank 2018"){.link-annotated .id-not .include-annotation link-icon="PNAS" link-icon-type="text,quad" link-icon-color="#1f75b9"}** ## `“CurriculumNet: Weakly Supervised Learning from Large-Scale Web Images”, Guo et al 2018`{=html} {#guo-et-al-2018-2-section} **[`CurriculumNet: Weakly Supervised Learning from Large-Scale Web Images`{=html}](https://arxiv.org/abs/1808.01097 "'CurriculumNet: Weakly Supervised Learning from Large-Scale Web Images', Guo et al 2018"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Large-Scale Visual Speech Recognition”, Shillingford et al 2018`{=html} {#shillingford-et-al-2018-section} **[`Large-Scale Visual Speech Recognition`{=html}](https://arxiv.org/abs/1807.05162 "'Large-Scale Visual Speech Recognition', Shillingford et al 2018"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Troubling Trends in Machine Learning Scholarship”, Lipton & Steinhardt 2018`{=html} {#lipton-steinhardt-2018-section} **[`Troubling Trends in Machine Learning Scholarship`{=html}](https://arxiv.org/abs/1807.03341 "'Troubling Trends in Machine Learning Scholarship', Lipton & Steinhardt 2018"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Benchmarking Neural Network Robustness to Common Corruptions and Surface Variations”, Hendrycks & Dietterich 2018`{=html} {#hendrycks-dietterich-2018-section} **[`Benchmarking Neural Network Robustness to Common Corruptions and Surface Variations`{=html}](https://arxiv.org/abs/1807.01697 "'Benchmarking Neural Network Robustness to Common Corruptions and Surface Variations', Hendrycks & Dietterich 2018"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Neural Scene Representation and Rendering”, Eslami et al 2018`{=html} {#eslami-et-al-2018-section} **[`Neural scene representation and rendering`{=html}](/doc/reinforcement-learning/model/2018-eslami.pdf#deepmind "'Neural scene representation and rendering', Eslami et al 2018"){.link-annotated .id-not .include-annotation link-icon="deepmind" link-icon-type="svg" link-icon-color="#4185f4"}** ## `“GPT-1: Improving Language Understanding With Unsupervised Learning”, OpenAI 2018`{=html} {#openai-2018-1-section} **[`GPT-1: Improving Language Understanding with Unsupervised Learning`{=html}](https://openai.com/research/language-unsupervised "'GPT-1: Improving Language Understanding with Unsupervised Learning', OpenAI 2018"){.link-annotated .id-not .include-annotation link-icon="openai" link-icon-type="svg"}** ## `“GPT-1: Improving Language Understanding by Generative Pre-Training”, Radford et al 2018`{=html} {#radford-et-al-2018-section} **[`GPT-1: Improving Language Understanding by Generative Pre-Training`{=html}](https://s3-us-west-2.amazonaws.com/openai-assets/research-covers/language-unsupervised/language_understanding_paper.pdf "'GPT-1: Improving Language Understanding by Generative Pre-Training', Radford et al 2018"){.link-annotated .id-not .include-annotation link-icon="pdf" link-icon-type="svg" link-icon-color="#f40f02"}** ## `“GPT-1: Improving Language Understanding by Generative Pre-Training § Model Specifications”, Radford et al 2018 (page 5)`{=html} {#radford-et-al-2018-page-5-section} **[`GPT-1: Improving Language Understanding by Generative Pre-Training § Model specifications`{=html}](https://s3-us-west-2.amazonaws.com/openai-assets/research-covers/language-unsupervised/language_understanding_paper.pdf#page=5 "'GPT-1: Improving Language Understanding by Generative Pre-Training § Model specifications', Radford et al 2018 (page 5)"){.link-annotated .id-not .include-annotation link-icon="pdf" link-icon-type="svg" link-icon-color="#f40f02"}** ## `“Do CIFAR-10 Classifiers Generalize to CIFAR-10?”, Recht et al 2018`{=html} {#recht-et-al-2018-section} **[`Do CIFAR-10 Classifiers Generalize to CIFAR-10?`{=html}](https://arxiv.org/abs/1806.00451 "'Do CIFAR-10 Classifiers Generalize to CIFAR-10?', Recht et al 2018"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Deep Learning Generalizes Because the Parameter-Function Map Is Biased towards Simple Functions”, Valle-Pérez et al 2018`{=html} {#valle-pérez-et-al-2018-section} **[`Deep learning generalizes because the parameter-function map is biased towards simple functions`{=html}](https://arxiv.org/abs/1805.08522 "'Deep learning generalizes because the parameter-function map is biased towards simple functions', Valle-Pérez et al 2018"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Google DeepMind Founder and Leader in Artificial Intelligence Returns to Hamilton”, Tantau 2018`{=html} {#tantau-2018-section} **[`Google DeepMind founder and leader in artificial intelligence returns to Hamilton`{=html}](https://web.archive.org/web/20210426084422/https://www.stuff.co.nz/technology/103500435/google-deepmind-founder-and-leader-in-artificial-intelligence-returns-to-hamilton "'Google DeepMind founder and leader in artificial intelligence returns to Hamilton', Tantau 2018"){.link-annotated .id-not .include-annotation link-icon="deepmind" link-icon-type="svg" link-icon-color="#4185f4"}** ## `“Exploring the Limits of Weakly Supervised Pretraining”, Mahajan et al 2018`{=html} {#mahajan-et-al-2018-2-section} **[`Exploring the Limits of Weakly Supervised Pretraining`{=html}](https://arxiv.org/abs/1805.00932#facebook "'Exploring the Limits of Weakly Supervised Pretraining', Mahajan et al 2018"){.link-annotated .id-not .include-annotation link-icon="facebook" link-icon-type="svg" link-icon-color="#1877f2"}** ## `“One Big Net For Everything”, Schmidhuber 2018`{=html} {#schmidhuber-2018-section} **[`One Big Net For Everything`{=html}](https://arxiv.org/abs/1802.08864#schmidhuber "'One Big Net For Everything', Schmidhuber 2018"){.link-annotated .id-not .include-annotation link-icon="SMDH" link-icon-type="text,quad,sans"}** ## `“Sensitivity and Generalization in Neural Networks: an Empirical Study”, Novak et al 2018`{=html} {#novak-et-al-2018-section} **[`Sensitivity and Generalization in Neural Networks: an Empirical Study`{=html}](https://arxiv.org/abs/1802.08760#google "'Sensitivity and Generalization in Neural Networks: an Empirical Study', Novak et al 2018"){.link-annotated .id-not .include-annotation link-icon="alphabet" link-icon-type="svg" link-icon-color="#4285f4"}** ## `“The Description Length of Deep Learning Models”, Blier & Ollivier 2018`{=html} {#blier-ollivier-2018-section} **[`The Description Length of Deep Learning Models`{=html}](https://arxiv.org/abs/1802.07044 "'The Description Length of Deep Learning Models', Blier & Ollivier 2018"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Learning and Memorization”, Chatterjee 2018`{=html} {#chatterjee-2018-section} **[`Learning and Memorization`{=html}](https://openreview.net/forum?id=S1XFzYyPMw "'Learning and Memorization', Chatterjee 2018"){.link-annotated .id-not .include-annotation link-icon="OR" link-icon-type="text,sans" link-icon-color="#8c1b13"}** ## `“ULMFiT: Universal Language Model Fine-Tuning for Text Classification”, Howard & Ruder 2018`{=html} {#howard-ruder-2018-section} **[`ULMFiT: Universal Language Model Fine-tuning for Text Classification`{=html}](https://arxiv.org/abs/1801.06146 "'ULMFiT: Universal Language Model Fine-tuning for Text Classification', Howard & Ruder 2018"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“GPipe: Easy Scaling With Micro-Batch Pipeline Parallelism § Pg4”, Huang 2018 (page 4 org google)`{=html} {#huang-2018-page-4-org-google-section} **[`GPipe: Easy Scaling with Micro-Batch Pipeline Parallelism § pg4`{=html}](https://arxiv.org/pdf/1811.06965.pdf#page=4&org=google "'GPipe: Easy Scaling with Micro-Batch Pipeline Parallelism § pg4', Huang 2018 (page 4 org google)"){.id-not .include-annotation link-icon="alphabet" link-icon-type="svg" link-icon-color="#4285f4"}[  :]{.mobile-not}** ::: aux-links-transclude-file ::: {.collapse .mobile-not} **View PDF**: [`/doc/www/arxiv.org/a8efcc8272af6f434119f87a00c2edaf84241597.pdf#page=4&org=google`](/doc/www/arxiv.org/a8efcc8272af6f434119f87a00c2edaf84241597.pdf#page=4&org=google){.id-not .link-annotated-not .include-content .include-lazy link-icon="alphabet" link-icon-type="svg" link-icon-color="#4285f4"} ::: ::: ## `“The Compute and Data Moats Are Dead”, Merity 2018`{=html} {#merity-2018-section} **[`The compute and data moats are dead`{=html}](https://smerity.com/articles/2018/limited_compute.html "'The compute and data moats are dead', Merity 2018"){.link-annotated .id-not .include-annotation}** ## `“Deep Image Reconstruction from Human Brain Activity”, Shen et al 2017`{=html} {#shen-et-al-2017-1-section} **[`Deep image reconstruction from human brain activity`{=html}](https://www.biorxiv.org/content/10.1101/240317.full "'Deep image reconstruction from human brain activity', Shen et al 2017"){.link-annotated .id-not .include-annotation link-icon="chi-dna" link-icon-type="svg" link-icon-color="#bd2736"}** ## `“Deep Learning Scaling Is Predictable, Empirically”, Hestness et al 2017`{=html} {#hestness-et-al-2017-section} **[`Deep Learning Scaling is Predictable, Empirically`{=html}](https://arxiv.org/abs/1712.00409#baidu "'Deep Learning Scaling is Predictable, Empirically', Hestness et al 2017"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Are GANs Created Equal? A Large-Scale Study”, Lucic et al 2017`{=html} {#lucic-et-al-2017-section} **[`Are GANs Created Equal? A Large-Scale Study`{=html}](https://arxiv.org/abs/1711.10337#google "'Are GANs Created Equal? A Large-Scale Study', Lucic et al 2017"){.link-annotated .id-not .include-annotation link-icon="alphabet" link-icon-type="svg" link-icon-color="#4285f4"}** ## `“Knowledge Concentration: Learning 100K Object Classifiers in a Single CNN”, Gao et al 2017`{=html} {#gao-et-al-2017-section} **[`Knowledge Concentration: Learning 100K Object Classifiers in a Single CNN`{=html}](https://arxiv.org/abs/1711.07607#google "'Knowledge Concentration: Learning 100K Object Classifiers in a Single CNN', Gao et al 2017"){.link-annotated .id-not .include-annotation link-icon="alphabet" link-icon-type="svg" link-icon-color="#4285f4"}** ## `“Rethinking Generalization Requires Revisiting Old Ideas: Statistical Mechanics Approaches and Complex Learning Behavior”, Martin & Mahoney 2017`{=html} {#martin-mahoney-2017-section} **[`Rethinking generalization requires revisiting old ideas: statistical mechanics approaches and complex learning behavior`{=html}](https://arxiv.org/abs/1710.09553 "'Rethinking generalization requires revisiting old ideas: statistical mechanics approaches and complex learning behavior', Martin & Mahoney 2017"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“There’s No Fire Alarm for Artificial General Intelligence”, Yudkowsky 2017`{=html} {#yudkowsky-2017-section} **[`There’s No Fire Alarm for Artificial General Intelligence`{=html}](https://intelligence.org/2017/10/13/fire-alarm/ "'There’s No Fire Alarm for Artificial General Intelligence', Yudkowsky 2017"){.link-annotated .id-not .include-annotation link-icon="miri" link-icon-type="svg" link-icon-color="#234e80"}** ## `“The Devil Is in the Tails: Fine-Grained Classification in the Wild”, Horn & Perona 2017`{=html} {#horn-perona-2017-section} **[`The Devil is in the Tails: Fine-grained Classification in the Wild`{=html}](https://arxiv.org/abs/1709.01450 "'The Devil is in the Tails: Fine-grained Classification in the Wild', Horn & Perona 2017"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“WebVision Database: Visual Learning and Understanding from Web Data”, Li et al 2017`{=html} {#li-et-al-2017-2-section} **[`WebVision Database: Visual Learning and Understanding from Web Data`{=html}](https://arxiv.org/abs/1708.02862 "'WebVision Database: Visual Learning and Understanding from Web Data', Li et al 2017"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Revisiting Unreasonable Effectiveness of Data in Deep Learning Era”, Sun et al 2017`{=html} {#sun-et-al-2017-2-section} **[`Revisiting Unreasonable Effectiveness of Data in Deep Learning Era`{=html}](https://arxiv.org/abs/1707.02968#google "'Revisiting Unreasonable Effectiveness of Data in Deep Learning Era', Sun et al 2017"){.link-annotated .id-not .include-annotation link-icon="alphabet" link-icon-type="svg" link-icon-color="#4285f4"}** ## `“Towards Deep Learning Models Resistant to Adversarial Attacks”, Madry et al 2017`{=html} {#madry-et-al-2017-section} **[`Towards Deep Learning Models Resistant to Adversarial Attacks`{=html}](https://arxiv.org/abs/1706.06083 "'Towards Deep Learning Models Resistant to Adversarial Attacks', Madry et al 2017"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Gradient Diversity: a Key Ingredient for Scalable Distributed Learning”, Yin et al 2017`{=html} {#yin-et-al-2017-1-section} **[`Gradient Diversity: a Key Ingredient for Scalable Distributed Learning`{=html}](https://arxiv.org/abs/1706.05699 "'Gradient Diversity: a Key Ingredient for Scalable Distributed Learning', Yin et al 2017"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Learning to Learn from Noisy Web Videos”, Yeung et al 2017`{=html} {#yeung-et-al-2017-section} **[`Learning to Learn from Noisy Web Videos`{=html}](https://arxiv.org/abs/1706.02884 "'Learning to Learn from Noisy Web Videos', Yeung et al 2017"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour”, Goyal et al 2017`{=html} {#goyal-et-al-2017-section} **[`Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour`{=html}](https://arxiv.org/abs/1706.02677#facebook "'Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour', Goyal et al 2017"){.link-annotated .id-not .include-annotation link-icon="facebook" link-icon-type="svg" link-icon-color="#1877f2"}** ## `“A Simple Neural Network Module for Relational Reasoning”, Santoro et al 2017`{=html} {#santoro-et-al-2017-section} **[`A simple neural network module for relational reasoning`{=html}](https://arxiv.org/abs/1706.01427#deepmind "'A simple neural network module for relational reasoning', Santoro et al 2017"){.link-annotated .id-not .include-annotation link-icon="deepmind" link-icon-type="svg" link-icon-color="#4185f4"}** ## `“Deep Learning Is Robust to Massive Label Noise”, Rolnick et al 2017`{=html} {#rolnick-et-al-2017-section} **[`Deep Learning is Robust to Massive Label Noise`{=html}](https://arxiv.org/abs/1705.10694 "'Deep Learning is Robust to Massive Label Noise', Rolnick et al 2017"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Quo Vadis, Action Recognition? A New Model I3D and the Kinetics Dataset”, Carreira & Zisserman 2017`{=html} {#carreira-zisserman-2017-section} **[`Quo Vadis, Action Recognition? A New Model I3D and the Kinetics Dataset`{=html}](https://arxiv.org/abs/1705.07750#deepmind "'Quo Vadis, Action Recognition? A New Model I3D and the Kinetics Dataset', Carreira & Zisserman 2017"){.link-annotated .id-not .include-annotation link-icon="deepmind" link-icon-type="svg" link-icon-color="#4185f4"}** ## `“WebVision Challenge: Visual Learning and Understanding With Web Data”, Li et al 2017`{=html} {#li-et-al-2017-1-section} **[`WebVision Challenge: Visual Learning and Understanding With Web Data`{=html}](https://arxiv.org/abs/1705.05640 "'WebVision Challenge: Visual Learning and Understanding With Web Data', Li et al 2017"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Geometry of Optimization and Implicit Regularization in Deep Learning”, Neyshabur et al 2017`{=html} {#neyshabur-et-al-2017-section} **[`Geometry of Optimization and Implicit Regularization in Deep Learning`{=html}](https://arxiv.org/abs/1705.03071 "'Geometry of Optimization and Implicit Regularization in Deep Learning', Neyshabur et al 2017"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“On the Impossibility of Supersized Machines”, Garfinkel et al 2017`{=html} {#garfinkel-et-al-2017-section} **[`On the Impossibility of Supersized Machines`{=html}](https://arxiv.org/abs/1703.10987 "'On the Impossibility of Supersized Machines', Garfinkel et al 2017"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Parallel Multiscale Autoregressive Density Estimation”, Reed et al 2017`{=html} {#reed-et-al-2017-section} **[`Parallel Multiscale Autoregressive Density Estimation`{=html}](https://arxiv.org/abs/1703.03664#deepmindopenai "'Parallel Multiscale Autoregressive Density Estimation', Reed et al 2017"){.link-annotated .id-not .include-annotation link-icon="deepmind" link-icon-type="svg" link-icon-color="#4185f4"}** ## `“Universal Representations: The Missing Link between Faces, Text, Planktons, and Cat Breeds”, Bilen & Vedaldi 2017`{=html} {#bilen-vedaldi-2017-section} **[`Universal representations: The missing link between faces, text, planktons, and cat breeds`{=html}](https://arxiv.org/abs/1701.07275 "'Universal representations: The missing link between faces, text, planktons, and cat breeds', Bilen & Vedaldi 2017"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Estimation of Gap Between Current Language Models and Human Performance”, Shen et al 2017`{=html} {#shen-et-al-2017-2-section} **[`Estimation of Gap Between Current Language Models and Human Performance`{=html}](/doc/ai/scaling/2017-shen.pdf "'Estimation of Gap Between Current Language Models and Human Performance', Shen et al 2017"){.link-annotated .id-not .include-annotation link-icon="pdf" link-icon-type="svg" link-icon-color="#f40f02"}** ## `“Simple and Scalable Predictive Uncertainty Estimation Using Deep Ensembles”, Lakshminarayanan et al 2016`{=html} {#lakshminarayanan-et-al-2016-section} **[`Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles`{=html}](https://arxiv.org/abs/1612.01474 "'Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles', Lakshminarayanan et al 2016"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Google’s Multilingual Neural Machine Translation System: Enabling Zero-Shot Translation”, Johnson et al 2016`{=html} {#johnson-et-al-2016-2-section} **[`Google’s Multilingual Neural Machine Translation System: Enabling Zero-Shot Translation`{=html}](https://arxiv.org/abs/1611.04558#google "'Google’s Multilingual Neural Machine Translation System: Enabling Zero-Shot Translation', Johnson et al 2016"){.link-annotated .id-not .include-annotation link-icon="alphabet" link-icon-type="svg" link-icon-color="#4285f4"}** ## `“Understanding Deep Learning Requires Rethinking Generalization”, Zhang et al 2016`{=html} {#zhang-et-al-2016-2-section} **[`Understanding deep learning requires rethinking generalization`{=html}](https://arxiv.org/abs/1611.03530#google "'Understanding deep learning requires rethinking generalization', Zhang et al 2016"){.link-annotated .id-not .include-annotation link-icon="alphabet" link-icon-type="svg" link-icon-color="#4285f4"}** ## `“Why Does Deep and Cheap Learning Work so Well?”, Lin et al 2016`{=html} {#lin-et-al-2016-2-section} **[`Why does deep and cheap learning work so well?`{=html}](https://arxiv.org/abs/1608.08225 "'Why does deep and cheap learning work so well?', Lin et al 2016"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“The LAMBADA Dataset: Word Prediction Requiring a Broad Discourse Context”, Paperno et al 2016`{=html} {#paperno-et-al-2016-section} **[`The LAMBADA dataset: Word prediction requiring a broad discourse context`{=html}](https://arxiv.org/abs/1606.06031 "'The LAMBADA dataset: Word prediction requiring a broad discourse context', Paperno et al 2016"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Residual Networks Behave Like Ensembles of Relatively Shallow Networks”, Veit et al 2016`{=html} {#veit-et-al-2016-section} **[`Residual Networks Behave Like Ensembles of Relatively Shallow Networks`{=html}](https://arxiv.org/abs/1605.06431 "'Residual Networks Behave Like Ensembles of Relatively Shallow Networks', Veit et al 2016"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Do Deep Convolutional Nets Really Need to Be Deep and Convolutional?”, Urban et al 2016`{=html} {#urban-et-al-2016-section} **[`Do Deep Convolutional Nets Really Need to be Deep and Convolutional?`{=html}](https://arxiv.org/abs/1603.05691 "'Do Deep Convolutional Nets Really Need to be Deep and Convolutional?', Urban et al 2016"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“PlaNet—Photo Geolocation With Convolutional Neural Networks”, Weyand et al 2016`{=html} {#weyand-et-al-2016-section} **[`PlaNet—Photo Geolocation with Convolutional Neural Networks`{=html}](https://arxiv.org/abs/1602.05314#deepmind "'PlaNet—Photo Geolocation with Convolutional Neural Networks', Weyand et al 2016"){.link-annotated .id-not .include-annotation link-icon="deepmind" link-icon-type="svg" link-icon-color="#4185f4"}** ## `“Exploring the Limits of Language Modeling”, Jozefowicz et al 2016`{=html} {#jozefowicz-et-al-2016-section} **[`Exploring the Limits of Language Modeling`{=html}](https://arxiv.org/abs/1602.02410#google "'Exploring the Limits of Language Modeling', Jozefowicz et al 2016"){.link-annotated .id-not .include-annotation link-icon="alphabet" link-icon-type="svg" link-icon-color="#4285f4"}** ## `“The Singularity: A Philosophical Analysis”, Chalmers 2016`{=html} {#chalmers-2016-section} **[`The Singularity: A Philosophical Analysis`{=html}](/doc/existential-risk/2016-chalmers.pdf "'The Singularity: A Philosophical Analysis', Chalmers 2016"){.link-annotated .id-not .include-annotation link-icon="pdf" link-icon-type="svg" link-icon-color="#f40f02"}** ## `“Microsoft Researchers Win ImageNet Computer Vision Challenge”, Linn 2015`{=html} {#linn-2015-section} **[`Microsoft researchers win ImageNet computer vision challenge`{=html}](https://blogs.microsoft.com/ai/microsoft-researchers-win-imagenet-computer-vision-challenge/ "'Microsoft researchers win ImageNet computer vision challenge', Linn 2015"){.link-annotated .id-not .include-annotation link-icon="MS" link-icon-type="text,sans,italic"}** ## `“Deep Speech 2: End-To-End Speech Recognition in English and Mandarin”, Amodei et al 2015`{=html} {#amodei-et-al-2015-section} **[`Deep Speech 2: End-to-End Speech Recognition in English and Mandarin`{=html}](https://arxiv.org/abs/1512.02595#baidu "'Deep Speech 2: End-to-End Speech Recognition in English and Mandarin', Amodei et al 2015"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“The Unreasonable Effectiveness of Noisy Data for Fine-Grained Recognition”, Krause et al 2015`{=html} {#krause-et-al-2015-section} **[`The Unreasonable Effectiveness of Noisy Data for Fine-Grained Recognition`{=html}](https://arxiv.org/abs/1511.06789#google "'The Unreasonable Effectiveness of Noisy Data for Fine-Grained Recognition', Krause et al 2015"){.link-annotated .id-not .include-annotation link-icon="alphabet" link-icon-type="svg" link-icon-color="#4285f4"}** ## `“Net2Net: Accelerating Learning via Knowledge Transfer”, Chen et al 2015`{=html} {#chen-et-al-2015-1-section} **[`Net2Net: Accelerating Learning via Knowledge Transfer`{=html}](https://arxiv.org/abs/1511.05641 "'Net2Net: Accelerating Learning via Knowledge Transfer', Chen et al 2015"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Generative Concatenative Nets Jointly Learn to Write and Classify Reviews”, Lipton et al 2015`{=html} {#lipton-et-al-2015-section} **[`Generative Concatenative Nets Jointly Learn to Write and Classify Reviews`{=html}](https://arxiv.org/abs/1511.03683 "'Generative Concatenative Nets Jointly Learn to Write and Classify Reviews', Lipton et al 2015"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Learning Visual Features from Large Weakly Supervised Data”, Joulin et al 2015`{=html} {#joulin-et-al-2015-section} **[`Learning Visual Features from Large Weakly Supervised Data`{=html}](https://arxiv.org/abs/1511.02251#facebook "'Learning Visual Features from Large Weakly Supervised Data', Joulin et al 2015"){.link-annotated .id-not .include-annotation link-icon="facebook" link-icon-type="svg" link-icon-color="#1877f2"}** ## `“LSUN: Construction of a Large-Scale Image Dataset Using Deep Learning With Humans in the Loop”, Yu et al 2015`{=html} {#yu-et-al-2015-section} **[`LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loop`{=html}](https://arxiv.org/abs/1506.03365 "'LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loop', Yu et al 2015"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Clothing-1M: Learning from Massive Noisy Labeled Data for Image Classification”, Xiao et al 2015`{=html} {#xiao-et-al-2015-section} **[`Clothing-1M: Learning from Massive Noisy Labeled Data for Image Classification`{=html}](https://openaccess.thecvf.com/content_cvpr_2015/papers/Xiao_Learning_From_Massive_2015_CVPR_paper.pdf#baidu "'Clothing-1M: Learning from Massive Noisy Labeled Data for Image Classification', Xiao et al 2015"){.link-annotated .id-not .include-annotation link-icon="pdf" link-icon-type="svg" link-icon-color="#f40f02"}** ## `“The Unreasonable Effectiveness of Recurrent Neural Networks”, Karpathy 2015`{=html} {#karpathy-2015-section} **[`The Unreasonable Effectiveness of Recurrent Neural Networks`{=html}](https://karpathy.github.io/2015/05/21/rnn-effectiveness/ "'The Unreasonable Effectiveness of Recurrent Neural Networks', Karpathy 2015"){.link-annotated .id-not .include-annotation}** ## `“LSTM: A Search Space Odyssey”, Greff et al 2015`{=html} {#greff-et-al-2015-section} **[`LSTM: A Search Space Odyssey`{=html}](https://arxiv.org/abs/1503.04069 "'LSTM: A Search Space Odyssey', Greff et al 2015"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“YFCC100M: The New Data in Multimedia Research”, Thomee et al 2015`{=html} {#thomee-et-al-2015-section} **[`YFCC100M: The New Data in Multimedia Research`{=html}](https://arxiv.org/abs/1503.01817#flickr "'YFCC100M: The New Data in Multimedia Research', Thomee et al 2015"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Machine Intelligence, Part 1”, Altman 2015`{=html} {#altman-2015-2-section} **[`Machine intelligence, part 1`{=html}](https://blog.samaltman.com/machine-intelligence-part-1 "'Machine intelligence, part 1', Altman 2015"){.link-annotated .id-not .include-annotation link-icon="sama" link-icon-type="text,quad,mono"}** ## `“50 Years of Deep Learning and Beyond: an Interview With Jürgen Schmidhuber”, Schmidhuber 2015`{=html} {#schmidhuber-2015-section} **[`50 Years of Deep Learning and Beyond: an Interview with Jürgen Schmidhuber`{=html}](https://innsbigdata.wordpress.com/2015/02/09/interview-with-juergen-schmidhuber/ "'50 Years of Deep Learning and Beyond: an Interview with Jürgen Schmidhuber', Schmidhuber 2015"){.link-annotated-partial .id-not .include-annotation link-icon="SMDH" link-icon-type="text,quad,sans"}** ## `“Evolution of the Human Brain: From Matter to Mind”, Hofman 2015`{=html} {#hofman-2015-section} **[`Evolution of the Human Brain: From Matter to Mind`{=html}](/doc/iq/high/2015-hofman.pdf "'Evolution of the Human Brain: From Matter to Mind', Hofman 2015"){.link-annotated .id-not .include-annotation link-icon="pdf" link-icon-type="svg" link-icon-color="#f40f02"}** ## `“In Search of the Real Inductive Bias: On the Role of Implicit Regularization in Deep Learning”, Neyshabur et al 2014`{=html} {#neyshabur-et-al-2014-section} **[`In Search of the Real Inductive Bias: On the Role of Implicit Regularization in Deep Learning`{=html}](https://arxiv.org/abs/1412.6614 "'In Search of the Real Inductive Bias: On the Role of Implicit Regularization in Deep Learning', Neyshabur et al 2014"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Deep Speech 1: Scaling up End-To-End Speech Recognition”, Hannun et al 2014`{=html} {#hannun-et-al-2014-section} **[`Deep Speech 1: Scaling up end-to-end speech recognition`{=html}](https://arxiv.org/abs/1412.5567#baidu "'Deep Speech 1: Scaling up end-to-end speech recognition', Hannun et al 2014"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Jumping NLP Curves: A Review of Natural Language Processing Research [Review Article]”, Cambria & White 2014`{=html} {#cambria-white-2014-section} **[`Jumping NLP Curves: A Review of Natural Language Processing Research [Review Article]`{=html}](/doc/ai/scaling/2014-cambria.pdf "'Jumping NLP Curves: A Review of Natural Language Processing Research [Review Article]', Cambria & White 2014"){.link-annotated .id-not .include-annotation link-icon="pdf" link-icon-type="svg" link-icon-color="#f40f02"}** ## `“Neural Networks, Manifolds, and Topology”, Olah 2014`{=html} {#olah-2014-section} **[`Neural Networks, Manifolds, and Topology`{=html}](https://colah.github.io/posts/2014-03-NN-Manifolds-Topology/ "'Neural Networks, Manifolds, and Topology', Olah 2014"){.link-annotated .id-not .include-annotation}** ## `“Computing’s Energy Problem (And What We Can Do about It)”, Horowitz 2014b`{=html} {#horowitz-2014b-section} **[`Computing’s Energy Problem (and what we can do about it)`{=html}](/doc/cs/hardware/2014-horowitz-2.pdf "'Computing’s Energy Problem (and what we can do about it)', Horowitz 2014b"){.link-annotated .id-not .include-annotation link-icon="pdf" link-icon-type="svg" link-icon-color="#f40f02"}** ## `“On the Number of Linear Regions of Deep Neural Networks”, Montúfar et al 2014`{=html} {#montúfar-et-al-2014-section} **[`On the Number of Linear Regions of Deep Neural Networks`{=html}](https://arxiv.org/abs/1402.1869 "'On the Number of Linear Regions of Deep Neural Networks', Montúfar et al 2014"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“N-Gram Counts and Language Models from the Common Crawl”, Buck et al 2014`{=html} {#buck-et-al-2014-section} **[`N-gram Counts and Language Models from the Common Crawl`{=html}](http://www.lrec-conf.org/proceedings/lrec2014/pdf/1097_Paper.pdf "'N-gram Counts and Language Models from the Common Crawl', Buck et al 2014"){.link-annotated .id-not .include-annotation link-icon="pdf" link-icon-type="svg" link-icon-color="#f40f02"}** ## `“Evolution of the Human Brain: When Bigger Is Better”, Hofman 2014`{=html} {#hofman-2014-section} **[`Evolution of the human brain: when bigger is better`{=html}](https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3973910/ "'Evolution of the human brain: when bigger is better', Hofman 2014"){.link-annotated .id-not .include-annotation link-icon="nlm-ncbi" link-icon-type="svg" link-icon-color="#20558a"}** ## `“One Billion Word Benchmark for Measuring Progress in Statistical Language Modeling”, Chelba et al 2013`{=html} {#chelba-et-al-2013-section} **[`One Billion Word Benchmark for Measuring Progress in Statistical Language Modeling`{=html}](https://arxiv.org/abs/1312.3005 "'One Billion Word Benchmark for Measuring Progress in Statistical Language Modeling', Chelba et al 2013"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Algorithmic Progress in Six Domains”, Grace 2013`{=html} {#grace-2013-section} **[`Algorithmic Progress in Six Domains`{=html}](/doc/ai/scaling/2013-grace.pdf#miri "'Algorithmic Progress in Six Domains', Grace 2013"){.link-annotated .id-not .include-annotation link-icon="pdf" link-icon-type="svg" link-icon-color="#f40f02"}** ## `“Large–Scale Machine Learning Revisited [Slides]”, Bottou 2013`{=html} {#bottou-2013-section} **[`Large–Scale Machine Learning Revisited [slides]`{=html}](/doc/ai/scaling/2013-bottou.pdf "'Large–Scale Machine Learning Revisited [slides]', Bottou 2013"){.link-annotated-partial .id-not .include-annotation link-icon="pdf" link-icon-type="svg" link-icon-color="#f40f02"}** ## `“20 Years of Bitext”, Brown et al 2013`{=html} {#brown-et-al-2013-section} **[`20 Years of Bitext`{=html}](/doc/psychology/linguistics/bilingual/2013-10-brown-20yearsofbitext.html "'20 Years of Bitext', Brown et al 2013"){.link-annotated-partial .id-not .include-annotation link-icon="internet-archive" link-icon-type="svg"}** ## `“Intelligence Explosion Microeconomics”, Yudkowsky 2013`{=html} {#yudkowsky-2013-section} **[`Intelligence Explosion Microeconomics`{=html}](/doc/ai/scaling/2013-yudkowsky.pdf#miri "'Intelligence Explosion Microeconomics', Yudkowsky 2013"){.link-annotated .id-not .include-annotation link-icon="pdf" link-icon-type="svg" link-icon-color="#f40f02"}** ## `“Scalable Modified Kneser-Ney Language Model Estimation”, Heafield et al 2013`{=html} {#heafield-et-al-2013-section} **[`Scalable Modified Kneser-Ney Language Model Estimation`{=html}](https://aclanthology.org/P13-2121.pdf "'Scalable Modified Kneser-Ney Language Model Estimation', Heafield et al 2013"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Large Scale Language Modeling in Automatic Speech Recognition”, Chelba et al 2012`{=html} {#chelba-et-al-2012-section} **[`Large Scale Language Modeling in Automatic Speech Recognition`{=html}](https://arxiv.org/abs/1210.8440#google "'Large Scale Language Modeling in Automatic Speech Recognition', Chelba et al 2012"){.link-annotated .id-not .include-annotation link-icon="alphabet" link-icon-type="svg" link-icon-color="#4285f4"}** ## `“The Remarkable, yet Not Extraordinary, Human Brain As a Scaled-Up Primate Brain and Its Associated Cost”, Herculano-Houzel 2012`{=html} {#herculano-houzel-2012-section} **[`The remarkable, yet not extraordinary, human brain as a scaled-up primate brain and its associated cost`{=html}](/doc/psychology/neuroscience/2012-herculanohouzel.pdf "'The remarkable, yet not extraordinary, human brain as a scaled-up primate brain and its associated cost', Herculano-Houzel 2012"){.link-annotated .id-not .include-annotation link-icon="pdf" link-icon-type="svg" link-icon-color="#f40f02"}** ## `“Advantages of Artificial Intelligences, Uploads, and Digital Minds”, Sotala 2012`{=html} {#sotala-2012-section} **[`Advantages of Artificial Intelligences, Uploads, and Digital Minds`{=html}](https://philpapers.org/archive/SOTAOA.pdf#miri "'Advantages of Artificial Intelligences, Uploads, and Digital Minds', Sotala 2012"){.link-annotated .id-not .include-annotation link-icon="PP" link-icon-type="text" link-icon-color="#109d49"}** ## `“Natural Language Processing (Almost) from Scratch”, Collobert et al 2011`{=html} {#collobert-et-al-2011-section} **[`Natural Language Processing (Almost) from Scratch`{=html}](/doc/psychology/linguistics/2011-collobert.pdf "'Natural Language Processing (Almost) from Scratch', Collobert et al 2011"){.link-annotated .id-not .include-annotation link-icon="pdf" link-icon-type="svg" link-icon-color="#f40f02"}** ## `“Recurrent Neural Network Based Language Model”, Mikolov et al 2010`{=html} {#mikolov-et-al-2010-section} **[`Recurrent Neural Network Based Language Model`{=html}](/doc/ai/nn/rnn/2010-mikolov.pdf "'Recurrent Neural Network Based Language Model', Mikolov et al 2010"){.link-annotated .id-not .include-annotation link-icon="pdf" link-icon-type="svg" link-icon-color="#f40f02"}** ## `“Understanding Sources of Inefficiency in General-Purpose Chips”, Hameed et al 2010`{=html} {#hameed-et-al-2010-section} **[`Understanding sources of inefficiency in general-purpose chips`{=html}](/doc/cs/hardware/2010-hameed.pdf "'Understanding sources of inefficiency in general-purpose chips', Hameed et al 2010"){.link-annotated .id-not .include-annotation link-icon="pdf" link-icon-type="svg" link-icon-color="#f40f02"}** ## `“The Teenies”, Legg 2009`{=html} {#legg-2009-the-teenies-section} **[`The Teenies`{=html}](https://www.vetta.org/2009/12/the-teenies/ "'The Teenies', Legg 2009"){.link-annotated .id-not .include-annotation link-icon="Legg" link-icon-type="text,quad,sans" link-icon-color="#4185f4"}** ## `“Tick, Tock, Tick, Tock… BING”, Legg 2009`{=html} {#legg-2009-tick-tock-section} **[`Tick, tock, tick, tock… BING`{=html}](https://www.vetta.org/2009/12/tick-tock-tick-tock-bing/ "'Tick, tock, tick, tock… BING', Legg 2009"){.link-annotated .id-not .include-annotation link-icon="Legg" link-icon-type="text,quad,sans" link-icon-color="#4185f4"}** ## `“Halloween Nightmare Scenario, Early 2020’s”, Wood 2009`{=html} {#wood-2009-section} **[`Halloween nightmare scenario, early 2020’s`{=html}](https://dw2blog.com/2009/11/02/halloween-nightmare-scenario-early-2020s/ "'Halloween nightmare scenario, early 2020’s', Wood 2009"){.link-annotated .id-not .include-annotation}** ## `“Matrix Factorization Techniques for Recommender Systems”, Koren et al 2009`{=html} {#koren-et-al-2009-section} **[`Matrix factorization techniques for recommender systems`{=html}](/doc/ai/tabular/2009-koren.pdf "'Matrix factorization techniques for recommender systems', Koren et al 2009"){.link-annotated .id-not .include-annotation link-icon="pdf" link-icon-type="svg" link-icon-color="#f40f02"}** ## `“The Unreasonable Effectiveness of Data”, Halevy et al 2009`{=html} {#halevy-et-al-2009-section} **[`The Unreasonable Effectiveness of Data`{=html}](/doc/ai/scaling/2009-halevy.pdf "'The Unreasonable Effectiveness of Data', Halevy et al 2009"){.link-annotated .id-not .include-annotation link-icon="pdf" link-icon-type="svg" link-icon-color="#f40f02"}** ## `“Economics Of The Singularity: Stuffed into Skyscrapers by the Billion, Brainy Bugbots Will Be the Knowledge Workers of the Future”, Hanson 2008`{=html} {#hanson-2008-2-section} **[`Economics Of The Singularity: Stuffed into skyscrapers by the billion, brainy bugbots will be the knowledge workers of the future`{=html}](https://spectrum.ieee.org/economics-of-the-singularity "'Economics Of The Singularity: Stuffed into skyscrapers by the billion, brainy bugbots will be the knowledge workers of the future', Hanson 2008"){.link-annotated .id-not .include-annotation link-icon="IEEE" link-icon-type="text,mono,quad" link-icon-color="#006699"}** ## `“Large Language Models in Machine Translation”, Brants et al 2007`{=html} {#brants-et-al-2007-section} **[`Large Language Models in Machine Translation`{=html}](/doc/ai/scaling/2007-brants.pdf#google "'Large Language Models in Machine Translation', Brants et al 2007"){.link-annotated .id-not .include-annotation link-icon="alphabet" link-icon-type="svg" link-icon-color="#4285f4"}** ## `“The Tradeoffs of Large-Scale Learning”, Bottou & Bousquet 2007`{=html} {#bottou-bousquet-2007-section} **[`The Tradeoffs of Large-Scale Learning`{=html}](/doc/ai/scaling/2007-bottou.pdf "'The Tradeoffs of Large-Scale Learning', Bottou & Bousquet 2007"){.link-annotated .id-not .include-annotation link-icon="pdf" link-icon-type="svg" link-icon-color="#f40f02"}** ## `“Cellular Scaling Rules for Primate Brains”, Herculano-Houzel et al 2007`{=html} {#herculano-houzel-et-al-2007-section} **[`Cellular scaling rules for primate brains`{=html}](https://www.ncbi.nlm.nih.gov/pmc/articles/PMC1805542/ "'Cellular scaling rules for primate brains', Herculano-Houzel et al 2007"){.link-annotated .id-not .include-annotation link-icon="nlm-ncbi" link-icon-type="svg" link-icon-color="#20558a"}** ## `“Robot Predictions Evolution”, Moravec 2004`{=html} {#moravec-2004-section} **[`Robot Predictions Evolution`{=html}](https://web.archive.org/web/20230718144747/https://frc.ri.cmu.edu/~hpm/project.archive/robot.papers/2004/Predictions.html "'Robot Predictions Evolution', Moravec 2004"){.link-annotated .id-not .include-annotation link-icon="internet-archive" link-icon-type="svg"}** ## `“Tree Induction versus Logistic Regression: A Learning-Curve Analysis”, Perlich et al 2003`{=html} {#perlich-et-al-2003-section} **[`Tree Induction versus Logistic Regression: A Learning-Curve Analysis`{=html}](/doc/ai/scaling/2003-perlich.pdf "'Tree Induction versus Logistic Regression: A Learning-Curve Analysis', Perlich et al 2003"){.link-annotated .id-not .include-annotation link-icon="pdf" link-icon-type="svg" link-icon-color="#f40f02"}** ## `“Analytic and Algorithmic Solution of Random Satisfiability Problems”, Mezard et al 2002`{=html} {#mezard-et-al-2002-section} **[`Analytic and Algorithmic Solution of Random Satisfiability Problems`{=html}](/doc/cs/computable/2002-mezard.pdf "'Analytic and Algorithmic Solution of Random Satisfiability Problems', Mezard et al 2002"){.link-annotated .id-not .include-annotation link-icon="pdf" link-icon-type="svg" link-icon-color="#f40f02"}** ## `“A Bit of Progress in Language Modeling”, Goodman 2001`{=html} {#goodman-2001-section} **[`A Bit of Progress in Language Modeling`{=html}](https://arxiv.org/abs/cs/0108005#microsoft "'A Bit of Progress in Language Modeling', Goodman 2001"){.link-annotated .id-not .include-annotation link-icon="MS" link-icon-type="text,sans,italic"}** ## `“Scaling to Very Very Large Corpora for Natural Language Disambiguation”, Banko & Brill 2001`{=html} {#banko-brill-2001-section} **[`Scaling to Very Very Large Corpora for Natural Language Disambiguation`{=html}](/doc/ai/scaling/2001-banko.pdf#microsoft "'Scaling to Very Very Large Corpora for Natural Language Disambiguation', Banko & Brill 2001"){.link-annotated .id-not .include-annotation link-icon="MS" link-icon-type="text,sans,italic"}** ## `“On Discriminative versus Generative Classifiers: A Comparison of Logistic Regression and Naive Bayes”, Ng & Jordan 2001`{=html} {#ng-jordan-2001-section} **[`On Discriminative versus Generative Classifiers: A comparison of logistic regression and naive Bayes`{=html}](/doc/ai/scaling/2001-ng.pdf "'On Discriminative versus Generative Classifiers: A comparison of logistic regression and naive Bayes', Ng & Jordan 2001"){.link-annotated .id-not .include-annotation link-icon="pdf" link-icon-type="svg" link-icon-color="#f40f02"}** ## `“A Survey of Methods for Scaling Up Inductive Algorithms”, Provost & Kolluri 1999`{=html} {#provost-kolluri-1999-section} **[`A Survey of Methods for Scaling Up Inductive Algorithms`{=html}](/doc/ai/tabular/1999-provost.pdf "'A Survey of Methods for Scaling Up Inductive Algorithms', Provost & Kolluri 1999"){.link-annotated .id-not .include-annotation link-icon="pdf" link-icon-type="svg" link-icon-color="#f40f02"}** ## `“On The Effect of Data Set Size on Bias And Variance in Classification Learning”, Brain & Webb 1999`{=html} {#brain-webb-1999-section} **[`On The Effect of Data Set Size on Bias And Variance in Classification Learning`{=html}](/doc/ai/scaling/1999-brain.pdf "'On The Effect of Data Set Size on Bias And Variance in Classification Learning', Brain & Webb 1999"){.link-annotated .id-not .include-annotation link-icon="pdf" link-icon-type="svg" link-icon-color="#f40f02"}** ## `“The Anatomy of a Large-Scale Hypertextual Web Search Engine”, Brin & Page 1998`{=html} {#brin-page-1998-section} **[`The Anatomy of a Large-Scale Hypertextual Web Search Engine`{=html}](http://infolab.stanford.edu/~backrub/google.html "'The Anatomy of a Large-Scale Hypertextual Web Search Engine', Brin & Page 1998"){.link-annotated .id-not .include-annotation link-icon="alphabet" link-icon-type="svg" link-icon-color="#4285f4"}** ## `“The Effects of Training Set Size on Decision Tree Complexity”, Oates & Jensen 1997`{=html} {#oates-jensen-1997-section} **[`The Effects of Training Set Size on Decision Tree Complexity`{=html}](/doc/ai/tabular/1997-oates.pdf "'The Effects of Training Set Size on Decision Tree Complexity', Oates & Jensen 1997"){.link-annotated .id-not .include-annotation link-icon="pdf" link-icon-type="svg" link-icon-color="#f40f02"}** ## `“Rigorous Learning Curve Bounds from Statistical Mechanics”, Haussler et al 1996`{=html} {#haussler-et-al-1996-section} **[`Rigorous Learning Curve Bounds from Statistical Mechanics`{=html}](/doc/ai/scaling/1996-haussler.pdf "'Rigorous Learning Curve Bounds from Statistical Mechanics', Haussler et al 1996"){.link-annotated .id-not .include-annotation link-icon="pdf" link-icon-type="svg" link-icon-color="#f40f02"}** ## `“Scaling up the Accuracy of Naive-Bayes Classifiers: a Decision-Tree Hybrid”, Kohavi 1996`{=html} {#kohavi-1996-section} **[`Scaling up the accuracy of Naive-Bayes classifiers: a decision-tree hybrid`{=html}](/doc/ai/tabular/1996-kohavi.pdf "'Scaling up the accuracy of Naive-Bayes classifiers: a decision-tree hybrid', Kohavi 1996"){.link-annotated .id-not .include-annotation link-icon="pdf" link-icon-type="svg" link-icon-color="#f40f02"}** ## `“Reflections After Refereeing Papers for NIPS”, Breiman 1995`{=html} {#breiman-1995-section} **[`Reflections After Refereeing Papers for NIPS`{=html}](/doc/ai/scaling/1995-breiman.pdf "'Reflections After Refereeing Papers for NIPS', Breiman 1995"){.link-annotated .id-not .include-annotation link-icon="pdf" link-icon-type="svg" link-icon-color="#f40f02"}** ## `“Biological Limits to Information Processing in the Human Brain”, Cochrane et al 1995`{=html} {#cochrane-et-al-1995-section} **[`Biological limits to information processing in the human brain`{=html}](/doc/iq/1995-cochrane-biologicallimitstoinformationprocessinginthebrain.html "'Biological limits to information processing in the human brain', Cochrane et al 1995"){.link-annotated .id-not .include-annotation link-icon="internet-archive" link-icon-type="svg"}** ## `“Handwritten Character Classification Using Nearest Neighbor in Large Databases”, Smith et al 1994`{=html} {#smith-et-al-1994-section} **[`Handwritten Character Classification Using Nearest Neighbor in Large Databases`{=html}](/doc/ai/tabular/1994-smith.pdf "'Handwritten Character Classification Using Nearest Neighbor in Large Databases', Smith et al 1994"){.link-annotated .id-not .include-annotation link-icon="pdf" link-icon-type="svg" link-icon-color="#f40f02"}** ## `“Building a Large Annotated Corpus of English: The Penn Treebank”, Marcus et al 1993`{=html} {#marcus-et-al-1993-section} **[`Building a Large Annotated Corpus of English: The Penn Treebank`{=html}](/doc/cs/algorithm/1993-marcus.pdf "'Building a Large Annotated Corpus of English: The Penn Treebank', Marcus et al 1993"){.link-annotated .id-not .include-annotation link-icon="pdf" link-icon-type="svg" link-icon-color="#f40f02"}** ## `“Statistical Theory of Learning Curves under Entropic Loss Criterion”, Amari & Murata 1993`{=html} {#amari-murata-1993-section} **[`Statistical Theory of Learning Curves under Entropic Loss Criterion`{=html}](/doc/ai/scaling/1993-amari.pdf "'Statistical Theory of Learning Curves under Entropic Loss Criterion', Amari & Murata 1993"){.link-annotated .id-not .include-annotation link-icon="pdf" link-icon-type="svg" link-icon-color="#f40f02"}** ## `“Learning Curves: Asymptotic Values and Rate of Convergence”, Cortes et al 1993`{=html} {#cortes-et-al-1993-section} **[`Learning Curves: Asymptotic Values and Rate of Convergence`{=html}](/doc/ai/scaling/1993-cortes.pdf "'Learning Curves: Asymptotic Values and Rate of Convergence', Cortes et al 1993"){.link-annotated .id-not .include-annotation link-icon="pdf" link-icon-type="svg" link-icon-color="#f40f02"}** ## `“Exhaustive Learning”, Schwartz et al 1990`{=html} {#schwartz-et-al-1990-section} **[`Exhaustive Learning`{=html}](/doc/ai/nn/1990-schwartz.pdf "'Exhaustive Learning', Schwartz et al 1990"){.link-annotated .id-not .include-annotation link-icon="pdf" link-icon-type="svg" link-icon-color="#f40f02"}** ## `“Don’t Worry—It Can’t Happen”, Harrington 1940`{=html} {#harrington-1940-section} **[`Don’t Worry—It Can’t Happen`{=html}](/doc/existential-risk/1940-sciam-harrington-nuclearweapons-dontworryitcanthappen.pdf "'Don’t Worry—It Can’t Happen', Harrington 1940"){.link-annotated .id-not .include-annotation link-icon="pdf" link-icon-type="svg" link-icon-color="#f40f02"}** ## `“Neural Net Stack-More-Layers Meme Template”, Anonymous 2026`{=html} {#_CwMBTtwa-section} **[`Neural Net Stack-More-Layers Meme Template`{=html}](/doc/fiction/humor/comics/meme-nnlayers-nnlol-template.png "'Neural Net Stack-More-Layers Meme Template', Anonymous 2026"){.link-annotated-partial .id-not .include-annotation link-icon="image" link-icon-type="svg"}** ## `“Eric Michaud on Neural Quantum Interpretability”`{=html} {#_twlVioRW-section} **[`Eric Michaud on Neural Quantum Interpretability`{=html}](http://theinsideview.ai/eric#the-quantization-model-of-neural-scaling "Eric Michaud on Neural Quantum Interpretability"){.id-not .include-annotation}** ::: aux-links-transclude-file ::: collapse **View HTML**: [`/doc/www/theinsideview.ai/3d4ef31011b49fa3442733759bb92f0b3bb8b6c5.html#the-quantization-model-of-neural-scaling`](/doc/www/theinsideview.ai/3d4ef31011b49fa3442733759bb92f0b3bb8b6c5.html#the-quantization-model-of-neural-scaling){.id-not .link-annotated-not .include-content .include-lazy link-icon="internet-archive" link-icon-type="svg"} ::: ::: ## `“Billion-Scale Semi-Supervised Learning for State-Of-The-Art Image and Video Classification”`{=html} {#_wyrkrFQT-section} **[`Billion-scale semi-supervised learning for state-of-the-art image and video classification`{=html}](https://ai.meta.com/blog/billion-scale-semi-supervised-learning/ "Billion-scale semi-supervised learning for state-of-the-art image and video classification"){.link-annotated-partial .id-not .include-annotation link-icon="facebook" link-icon-type="svg" link-icon-color="#1877f2"}** ## `“No Physics? No Problem. AI Weather Forecasting Is Already Making Huge Strides.”`{=html} {#_pJuwccH8-section} **[`No physics? No problem. AI weather forecasting is already making huge strides.`{=html}](https://arstechnica.com/ai/2024/06/as-a-potentially-historic-hurricane-season-looms-can-ai-forecast-models-help/ "No physics? No problem. AI weather forecasting is already making huge strides."){.link-annotated-partial .id-not .include-annotation link-icon="ars" link-icon-type="text,tri,sans" link-icon-color="#ff4e00"}** ## `“Report Describes Apple’s ‘Organizational Dysfunction’ and ‘Lack of Ambition’ in AI”`{=html} {#_pgDbyvb_-section} **[`Report describes Apple’s ‘organizational dysfunction’ and ‘lack of ambition’ in AI`{=html}](https://arstechnica.com/gadgets/2023/04/report-describes-apples-organizational-dysfunction-and-lack-of-ambition-in-ai/ "Report describes Apple’s ‘organizational dysfunction’ and ‘lack of ambition’ in AI"){.id-not .include-annotation link-icon="ars" link-icon-type="text,tri,sans" link-icon-color="#ff4e00"}** ## `“StyleGAN-2 512px Trained on Danbooru2019”`{=html} {#_2aYG3s6b-section} **[`StyleGAN-2 512px trained on Danbooru2019`{=html}](https://aydao.ai/experiment/2020/12/05/dbstylegan512.html "StyleGAN-2 512px trained on Danbooru2019"){.link-annotated-partial .id-not .include-annotation}** ## `“Blake Bordelon”, Bordelon 2026`{=html} {#_Pl8kCo1X-section} **[`Blake Bordelon`{=html}](https://blakebordelon.github.io/ "'Blake Bordelon', Bordelon 2026"){.link-annotated-partial .id-not .include-annotation}** ## `“How to Train the Best Embedding Model in the World: One PhD Later, I’m Giving My Secrets Away for Free”, Morris 2026`{=html} {#_VZDsjpId-section} **[`How to train the best embedding model in the world: one PhD later, I’m giving my secrets away for free`{=html}](https://blog.jxmo.io/p/how-to-train-the-best-embedding-model "'How to train the best embedding model in the world: one PhD later, I’m giving my secrets away for free', Morris 2026"){.link-annotated-partial .id-not .include-annotation}** ## `“Scaling Karpathy’s Autoresearch: What Happens When the Agent Gets a GPU Cluster”`{=html} {#_LS0QaJKg-section} **[`Scaling Karpathy’s Autoresearch: What Happens When the Agent Gets a GPU Cluster`{=html}](https://blog.skypilot.co/scaling-autoresearch/ "Scaling Karpathy’s Autoresearch: What Happens When the Agent Gets a GPU Cluster"){.link-annotated-partial .id-not .include-annotation}** ## `“Inside the CodeBot: A Gentle Introduction to How LLMs Understand Nullability”`{=html} {#_4kKqbl1m-section} **[`Inside the CodeBot: A Gentle Introduction to How LLMs Understand Nullability`{=html}](https://dmodel.ai/nullability-gentle/ "Inside the CodeBot: A Gentle Introduction to How LLMs Understand Nullability"){.id-not .include-annotation}** ## `“Komodo 8: the Smartphone vs Desktop Challenge”`{=html} {#_LAQMMPWq-section} **[`Komodo 8: the smartphone vs desktop challenge`{=html}](https://en.chessbase.com/post/komodo-8-the-smartphone-vs-desktop-challenge "Komodo 8: the smartphone vs desktop challenge"){.link-annotated-partial .id-not .include-annotation}** ## `“Trading Off Compute in Training and Inference § Pruning”`{=html} {#_pGokXlMr-section} **[`Trading Off Compute in Training and Inference § Pruning`{=html}](https://epoch.ai/blog/trading-off-compute-in-training-and-inference#pruning "Trading Off Compute in Training and Inference § Pruning"){.id-not .include-annotation}** ## `“Eric Tang”`{=html} {#_snjCiacS-section} **[`Eric Tang`{=html}](https://erictang000.github.io/ "Eric Tang"){.link-annotated-partial .id-not .include-annotation}** ## `“How Can We Make Robotics More like Generative Modeling?”`{=html} {#_vKXe25yQ-section} **[`How Can We Make Robotics More like Generative Modeling?`{=html}](https://evjang.com/2022/07/23/robotics-generative.html "How Can We Make Robotics More like Generative Modeling?"){.link-annotated-partial .id-not .include-annotation}** ## `“True Story of Pangu: 诺亚盘古大模型研发背后的真正的心酸与黑暗的故事。[Model Plagiarism]”, Whistleblower 2026`{=html} {#_17c6M0zf-section} **[`True Story of Pangu: 诺亚盘古大模型研发背后的真正的心酸与黑暗的故事。[model plagiarism]`{=html}](https://github.com/HW-whistleblower/True-Story-of-Pangu "'True Story of Pangu: 诺亚盘古大模型研发背后的真正的心酸与黑暗的故事。[model plagiarism]', Whistleblower 2026"){.link-annotated-partial .id-not .include-annotation link-icon="github" link-icon-type="svg"}** ## `“Compression Represents Intelligence Linearly [Code]”`{=html} {#_sgFCix_w-section} **[`Compression Represents Intelligence Linearly [code]`{=html}](https://github.com/hkust-nlp/llm-compression-intelligence "Compression Represents Intelligence Linearly [code]"){.link-annotated-partial .id-not .include-annotation link-icon="github" link-icon-type="svg"}** ## `“Inverse-Scaling/prize: A Prize for Finding Tasks That Cause Large Language Models to Show Inverse Scaling”`{=html} {#_2wZETXBY-section} **[`inverse-scaling/prize: A prize for finding tasks that cause large language models to show inverse scaling`{=html}](https://github.com/inverse-scaling/prize "inverse-scaling/prize: A prize for finding tasks that cause large language models to show inverse scaling"){.id-not .include-annotation link-icon="github" link-icon-type="svg"}** ## `“Scaling up StyleGAN-2”`{=html} {#_wdw3i3n_-section} **[`Scaling up StyleGAN-2`{=html}](https://github.com/l4rz/scaling-up-stylegan2 "Scaling up StyleGAN-2"){.link-annotated-partial .id-not .include-annotation link-icon="github" link-icon-type="svg"}** ## `“Finally, a Replacement for BERT: Introducing ModernBERT”`{=html} {#_2Y9aB8G0-section} **[`Finally, a Replacement for BERT: Introducing ModernBERT`{=html}](https://huggingface.co/blog/modernbert "Finally, a Replacement for BERT: Introducing ModernBERT"){.link-annotated-partial .id-not .include-annotation link-icon="🤗" link-icon-type="text" link-icon-color="#ffcc4d"}** ## `“Llm-Compression Data”`{=html} {#_3-85l0T7-section} **[`llm-compression data`{=html}](https://huggingface.co/datasets/hkust-nlp/llm-compression "llm-compression data"){.id-not .include-annotation link-icon="🤗" link-icon-type="text" link-icon-color="#ffcc4d"}** ## `“Semi Supervised Learning”`{=html} {#_Gb_0ucu_-section} **[`Semi Supervised Learning`{=html}](https://lilianweng.github.io/lil-log/2021/12/05/semi-supervised-learning.html "Semi Supervised Learning"){.link-annotated-partial .id-not .include-annotation}** ## `“Homepage of Paul F. Christiano”, Christiano 2026`{=html} {#_Rx5I2W4d-section} **[`Homepage of Paul F. Christiano`{=html}](https://paulfchristiano.com/ "'Homepage of Paul F. Christiano', Christiano 2026"){.link-annotated .id-not .include-annotation link-icon="PFC" link-icon-type="text,tri"}** ## `“Statistical Modeling: The Two Cultures”, Breiman 2026`{=html} {#_P5IBSzmZ-section} **[`Statistical Modeling: The Two Cultures`{=html}](https://projecteuclid.org/download/pdf_1/euclid.ss/1009213726 "'Statistical Modeling: The Two Cultures', Breiman 2026"){.link-annotated-partial .id-not .include-annotation link-icon="pdf" link-icon-type="svg" link-icon-color="#f40f02"}** ## `“NanoGPT Slowrun—Q”`{=html} {#_tE3qQZXn-section} **[`NanoGPT Slowrun—Q`{=html}](https://qlabs.sh/slowrun "NanoGPT Slowrun—Q"){.link-annotated-partial .id-not .include-annotation}** ## `“T-Rex As: ‘The Computer Scientist’”, North 2026`{=html} {#_MhJ3YK9F-section} **[`T-Rex As: ‘The Computer Scientist’`{=html}](https://qwantz.com/index.php?comic=354 "'T-Rex As: ‘The Computer Scientist’', North 2026"){.link-annotated-partial .id-not .include-annotation link-icon="DC" link-icon-type="text,sans" link-icon-color="#40d53a"}** ## `“Have Chinese AI Models Caught Up to the US Frontier? [Probably Not]”`{=html} {#_uSaq2duE-section} **[`Have Chinese AI Models Caught Up to the US Frontier? [probably not]`{=html}](https://scaling01.substack.com/p/have-chinese-ai-models-caught-up "Have Chinese AI Models Caught Up to the US Frontier? [probably not]"){.id-not .include-annotation link-icon="substack" link-icon-type="svg" link-icon-color="#ff6719"}** ## `“Jared Kaplan”`{=html} {#_i2_IVecp-section} **[`Jared Kaplan`{=html}](https://sites.krieger.jhu.edu/jared-kaplan/ "Jared Kaplan"){.link-annotated-partial .id-not .include-annotation}** ## `“Safe Superintelligence Inc.”`{=html} {#_sDl1DEMh-section} **[`Safe Superintelligence Inc.`{=html}](https://ssi.inc/ "Safe Superintelligence Inc."){.link-annotated-partial .id-not .include-annotation}** ## `“OpenAI Disbands Its Robotics Research Team”`{=html} {#_i5L3bXqe-section} **[`OpenAI disbands its robotics research team`{=html}](https://venturebeat.com/business/openai-disbands-its-robotics-research-team/ "OpenAI disbands its robotics research team"){.id-not .include-annotation}** ::: aux-links-transclude-file ::: collapse **View External Link**: [`https://venturebeat.com/business/openai-disbands-its-robotics-research-team/`](https://venturebeat.com/business/openai-disbands-its-robotics-research-team/){.id-not .link-annotated-not .include-content .include-lazy} ::: ::: ## `“Google Workloads for Consumer Devices: Mitigating Data Movement Bottlenecks”`{=html} {#_RdtNd2lI-section} **[`Google workloads for consumer devices: mitigating data movement bottlenecks`{=html}](https://web.archive.org/web/20190131021221/https://blog.acolyer.org/2018/04/18/google-workloads-for-consumer-devices-mitigating-data-movement-bottlenecks/ "Google workloads for consumer devices: mitigating data movement bottlenecks"){.id-not .include-annotation link-icon="alphabet" link-icon-type="svg" link-icon-color="#4285f4"}** ::: aux-links-transclude-file ::: collapse **View External Link**: [`https://web.archive.org/web/20190131021221/https://blog.acolyer.org/2018/04/18/google-workloads-for-consumer-devices-mitigating-data-movement-bottlenecks/`](https://web.archive.org/web/20190131021221/https://blog.acolyer.org/2018/04/18/google-workloads-for-consumer-devices-mitigating-data-movement-bottlenecks/){.id-not .link-annotated-not .include-content .include-lazy link-icon="alphabet" link-icon-type="svg" link-icon-color="#4285f4"} ::: ::: ## `“We Tracked a Shipment of Rare Books. It Ended at an Amazon AI Training Facility”`{=html} {#_NaMdQFKx-section} **[`We Tracked a Shipment of Rare Books. It Ended at an Amazon AI Training Facility`{=html}](https://web.archive.org/web/20260817141552/https://www.404media.co/we-tracked-a-shipment-of-rare-books-it-ended-at-an-amazon-ai-training-facility/ "We Tracked a Shipment of Rare Books. It Ended at an Amazon AI Training Facility"){.link-modified-recently .link-annotated-partial .id-not .include-annotation link-icon="internet-archive" link-icon-type="svg"}** ## `“The Uneasy Relationship between Deep Learning and (Classical) Statistics”`{=html} {#_5zq4TWyc-section} **[`The uneasy relationship between deep learning and (classical) statistics`{=html}](https://windowsontheory.org/2022/06/20/the-uneasy-relationship-between-deep-learning-and-classical-statistics/ "The uneasy relationship between deep learning and (classical) statistics"){.link-annotated-partial .id-not .include-annotation}** ## `“Parameter Counts in Machine Learning”`{=html} {#_QJuwN0Ft-section} **[`Parameter counts in Machine Learning`{=html}](https://www.alignmentforum.org/posts/GzoWcYibWYwJva8aL/parameter-counts-in-machine-learning "Parameter counts in Machine Learning"){.id-not .include-annotation link-icon="AF" link-icon-type="text,sans" link-icon-color="#3f51b5"}** ::: aux-links-transclude-file ::: collapse **View External Link**: [`https://www.alignmentforum.org/posts/GzoWcYibWYwJva8aL/parameter-counts-in-machine-learning`](https://www.alignmentforum.org/posts/GzoWcYibWYwJva8aL/parameter-counts-in-machine-learning){.id-not .link-annotated-not .include-content .include-lazy link-icon="AF" link-icon-type="text,sans" link-icon-color="#3f51b5"} ::: ::: ## `“Nathan Goes to China—Part 2: AI Safety With Chinese Characteristics”`{=html} {#_ArXiL0ZQ-section} **[`Nathan Goes to China—Part 2: AI Safety with Chinese Characteristics`{=html}](https://www.cognitiverevolution.ai/nathan-goes-to-china-part-2-ai-safety-with-chinese-characteristics/ "Nathan Goes to China—Part 2: AI Safety with Chinese Characteristics"){.link-modified-recently .id-not .include-annotation}** ## `“Can LLMs Learn from a Single Example?”`{=html} {#_OqoHp8uc-section} **[`Can LLMs learn from a single example?`{=html}](https://www.fast.ai/posts/2023-09-04-learning-jumps/ "Can LLMs learn from a single example?"){.link-annotated-partial .id-not .include-annotation link-icon="Fai" link-icon-type="text,tri" link-icon-color="#3399f3"}** ## `“Deciphering China’s AI Dream”`{=html} {#_e6RiW0Mt-section} **[`Deciphering China’s AI Dream`{=html}](https://www.fhi.ox.ac.uk/deciphering-chinas-ai-dream/ "Deciphering China’s AI Dream"){.link-annotated-partial .id-not .include-annotation}** ## `“Jason Wei”`{=html} {#_yb2381Jp-section} **[`Jason Wei`{=html}](https://www.jasonwei.net/ "Jason Wei"){.id-not .include-annotation}** ## `“Appendix: More Is Different In Other Domains”`{=html} {#_g8Y-GzQC-section} **[`Appendix: More Is Different In Other Domains`{=html}](https://www.lesswrong.com/posts/3daPPjWbjYNP6nbre/appendix-more-is-different-in-other-domains "Appendix: More Is Different In Other Domains"){.id-not .include-annotation link-icon="LW" link-icon-type="text" link-icon-color="#7faf83"}** ::: aux-links-transclude-file ::: collapse **View External Link**: [`https://www.lesswrong.com/posts/3daPPjWbjYNP6nbre/appendix-more-is-different-in-other-domains`](https://www.lesswrong.com/posts/3daPPjWbjYNP6nbre/appendix-more-is-different-in-other-domains){.id-not .link-annotated-not .include-content .include-lazy link-icon="LW" link-icon-type="text" link-icon-color="#7faf83"} ::: ::: ## `“Beware General Claims about ‘Generalizable Reasoning Capabilities’ (Of Modern AI Systems)”`{=html} {#_t8w9kEcz-section} **[`Beware General Claims about ‘Generalizable Reasoning Capabilities’ (of Modern AI Systems)`{=html}](https://www.lesswrong.com/posts/5uw26uDdFbFQgKzih/beware-general-claims-about-generalizable-reasoning "Beware General Claims about ‘Generalizable Reasoning Capabilities’ (of Modern AI Systems)"){.link-annotated-partial .id-not .include-annotation link-icon="LW" link-icon-type="text" link-icon-color="#7faf83"}** ## `“Understanding ‘Deep Double Descent’”`{=html} {#_riunlRMo-section} **[`Understanding ‘Deep Double Descent’`{=html}](https://www.lesswrong.com/posts/FRv7ryoqtvSuqBxuT/understanding-deep-double-descent "Understanding ‘Deep Double Descent’"){.id-not .include-annotation link-icon="LW" link-icon-type="text" link-icon-color="#7faf83"}** ::: aux-links-transclude-file ::: collapse **View External Link**: [`https://www.lesswrong.com/posts/FRv7ryoqtvSuqBxuT/understanding-deep-double-descent`](https://www.lesswrong.com/posts/FRv7ryoqtvSuqBxuT/understanding-deep-double-descent){.id-not .link-annotated-not .include-content .include-lazy link-icon="LW" link-icon-type="text" link-icon-color="#7faf83"} ::: ::: ## `“How Much Compute Was Used to Train DeepMind’s Generally Capable Agents?”`{=html} {#_Nj_dJRng-section} **[`How much compute was used to train DeepMind’s generally capable agents?`{=html}](https://www.lesswrong.com/posts/KaPaTdpLggdMqzdyo/how-much-compute-was-used-to-train-deepmind-s-generally "How much compute was used to train DeepMind’s generally capable agents?"){.id-not .include-annotation link-icon="deepmind" link-icon-type="svg" link-icon-color="#4185f4"}** ::: aux-links-transclude-file ::: collapse **View External Link**: [`https://www.lesswrong.com/posts/KaPaTdpLggdMqzdyo/how-much-compute-was-used-to-train-deepmind-s-generally`](https://www.lesswrong.com/posts/KaPaTdpLggdMqzdyo/how-much-compute-was-used-to-train-deepmind-s-generally){.id-not .link-annotated-not .include-content .include-lazy link-icon="deepmind" link-icon-type="svg" link-icon-color="#4185f4"} ::: ::: ## `“Why Neural Networks Generalise, and Why They Are (Kind Of) Bayesian”`{=html} {#_DBVP7I2A-section} **[`Why Neural Networks Generalise, and Why They Are (Kind of) Bayesian`{=html}](https://www.lesswrong.com/posts/YSFJosoHYFyXjoYWa/why-neural-networks-generalise-and-why-they-are-kind-of "Why Neural Networks Generalise, and Why They Are (Kind of) Bayesian"){.id-not .include-annotation link-icon="LW" link-icon-type="text" link-icon-color="#7faf83"}** ::: aux-links-transclude-file ::: collapse **View External Link**: [`https://www.lesswrong.com/posts/YSFJosoHYFyXjoYWa/why-neural-networks-generalise-and-why-they-are-kind-of`](https://www.lesswrong.com/posts/YSFJosoHYFyXjoYWa/why-neural-networks-generalise-and-why-they-are-kind-of){.id-not .link-annotated-not .include-content .include-lazy link-icon="LW" link-icon-type="text" link-icon-color="#7faf83"} ::: ::: ## `“LLMs As Giant Lookup-Tables of Shallow Circuits”, niplav 2026`{=html} {#_ne7BWIoa-section} **[`LLMs as Giant Lookup-Tables of Shallow Circuits`{=html}](https://www.lesswrong.com/posts/a9KqqgjN8gc3Mzzkh/llms-as-giant-lookup-tables-of-shallow-circuits "'LLMs as Giant Lookup-Tables of Shallow Circuits', niplav 2026"){.link-annotated-partial .id-not .include-annotation link-icon="LW" link-icon-type="text" link-icon-color="#7faf83"}** ## `“(Maybe) A Bag of Heuristics Is All There Is & A Bag of Heuristics Is All You Need”`{=html} {#_dnQ_opit-section} **[`(Maybe) A Bag of Heuristics is All There Is & A Bag of Heuristics is All You Need`{=html}](https://www.lesswrong.com/posts/azCuKGJTecCDLBEEj/maybe-a-bag-of-heuristics-is-all-there-is-and-a-bag-of "(Maybe) A Bag of Heuristics is All There Is & A Bag of Heuristics is All You Need"){.link-annotated-partial .id-not .include-annotation link-icon="LW" link-icon-type="text" link-icon-color="#7faf83"}** ## `“What’s the Backward-Forward FLOP Ratio for Neural Networks?”`{=html} {#_YM-BOoFT-section} **[`What’s the backward-forward FLOP ratio for Neural Networks?`{=html}](https://www.lesswrong.com/posts/fnjKpBoWJXcSDwhZk/what-s-the-backward-forward-flop-ratio-for-nns "What’s the backward-forward FLOP ratio for Neural Networks?"){.id-not .include-annotation link-icon="LW" link-icon-type="text" link-icon-color="#7faf83"}** ::: aux-links-transclude-file ::: collapse **View External Link**: [`https://www.lesswrong.com/posts/fnjKpBoWJXcSDwhZk/what-s-the-backward-forward-flop-ratio-for-nns`](https://www.lesswrong.com/posts/fnjKpBoWJXcSDwhZk/what-s-the-backward-forward-flop-ratio-for-nns){.id-not .link-annotated-not .include-content .include-lazy link-icon="LW" link-icon-type="text" link-icon-color="#7faf83"} ::: ::: ## `“Optimality Is the Tiger, and Agents Are Its Teeth”`{=html} {#_Mw83Di04-section} **[`Optimality is the tiger, and agents are its teeth`{=html}](https://www.lesswrong.com/posts/kpPnReyBC54KESiSn/optimality-is-the-tiger-and-agents-are-its-teeth "Optimality is the tiger, and agents are its teeth"){.link-annotated-partial .id-not .include-annotation link-icon="LW" link-icon-type="text" link-icon-color="#7faf83"}** ## `“What Next? A Dozen Information-Technology Research Goals: 3. Turing’s Vision of Machine Intelligence”`{=html} {#_iv1D1Hiw-section} **[`What Next? A Dozen Information-Technology Research Goals: 3. Turing’s vision of machine intelligence`{=html}](https://www.microsoft.com/en-us/research/wp-content/uploads/2016/02/ms_tr_99_50_turingtalk.pdf#page=11 "What Next? A Dozen Information-Technology Research Goals: 3. Turing’s vision of machine intelligence"){.id-not .include-annotation link-icon="MS" link-icon-type="text,sans,italic"}[  :]{.mobile-not}** ::: aux-links-transclude-file ::: {.collapse .mobile-not} **View PDF**: [`/doc/www/www.microsoft.com/5620cc2a603069db2406c32006715aa6535d051b.pdf#page=11`](/doc/www/www.microsoft.com/5620cc2a603069db2406c32006715aa6535d051b.pdf#page=11){.id-not .link-annotated-not .include-content .include-lazy link-icon="MS" link-icon-type="text,sans,italic"} ::: ::: ## `“Was Linguistic AI Created by Accident?”`{=html} {#_q2QtmqZ0-section} **[`Was Linguistic AI Created by Accident?`{=html}](https://www.newyorker.com/science/annals-of-artificial-intelligence/was-linguistic-ai-created-by-accident "Was Linguistic AI Created by Accident?"){.id-not .include-annotation link-icon="the-new-yorker" link-icon-type="svg"}** ## `“Tom Henighan”`{=html} {#_xDNEhCb9-section} **[`Tom Henighan`{=html}](https://www.tomhenighan.com/ "Tom Henighan"){.link-annotated-partial .id-not .include-annotation}** ## `“AI Cheating Is on the Rise: An Integrity Audit across BioMysteryBench, Terminal-Bench 2.1, and SWE-Bench Verified Shows That Cheating Increasingly Complicates Evaluation”`{=html} {#_qGU-tfjs-section} **[`AI Cheating is on the Rise: An integrity audit across BioMysteryBench, Terminal-Bench 2.1, and SWE-bench Verified shows that cheating increasingly complicates evaluation`{=html}](https://www.vals.ai/blogs/cheating-on-the-rise "AI Cheating is on the Rise: An integrity audit across BioMysteryBench, Terminal-Bench 2.1, and SWE-bench Verified shows that cheating increasingly complicates evaluation"){.link-modified-recently .id-not .include-annotation}** ## `“Ilya Sutskever: Deep Learning | AI Podcast #94 With Lex Fridman”`{=html} {#_FQ0wU7Cx-section} **[`Ilya Sutskever: Deep Learning | AI Podcast #94 with Lex Fridman`{=html}](https://www.youtube.com/watch?v=13CZPWmke6A "Ilya Sutskever: Deep Learning | AI Podcast #94 with Lex Fridman"){.link-annotated-partial .id-not .include-annotation link-icon="youtube" link-icon-type="svg" link-icon-color="#ff0033"}** ## `“A Universal Law of Robustness”`{=html} {#_wiT4lvAl-section} **[`A Universal Law of Robustness`{=html}](https://www.youtube.com/watch?v=OzGguadEHOU "A Universal Law of Robustness"){.id-not .include-annotation link-icon="youtube" link-icon-type="svg" link-icon-color="#ff0033"}** ::: aux-links-transclude-file [`https://www.youtube.com/watch?v=OzGguadEHOU`](https://www.youtube.com/watch?v=OzGguadEHOU){.link-annotated-not .include-content .width-full} ::: ## `“Greg Brockman: OpenAI and AGI”, Brockman 2026`{=html} {#_vlTi1sCo-section} **[`Greg Brockman: OpenAI and AGI`{=html}](https://www.youtube.com/watch?v=bIrEM2FbOLU&t=2740 "'Greg Brockman: OpenAI and AGI', Brockman 2026"){.id-not .include-annotation link-icon="youtube" link-icon-type="svg" link-icon-color="#ff0033"}** ::: aux-links-transclude-file [`https://www.youtube.com/watch?v=bIrEM2FbOLU&t=2740`](https://www.youtube.com/watch?v=bIrEM2FbOLU&t=2740){.link-annotated-not .include-content .width-full} ::: ## `“Season 1 Ep. 22 OpenAI’s Ilya Sutskever: The Man Who Made AI Work”`{=html} {#_7gZXYODe-section} **[`Season 1 Ep. 22 OpenAI’s Ilya Sutskever: The man who made AI work`{=html}](https://www.youtube.com/watch?v=fCoavgGZ64Y&t=2796s "Season 1 Ep. 22 OpenAI’s Ilya Sutskever: The man who made AI work"){.id-not .include-annotation link-icon="youtube" link-icon-type="svg" link-icon-color="#ff0033"}** ::: aux-links-transclude-file [`https://www.youtube.com/watch?v=fCoavgGZ64Y&t=2796s`](https://www.youtube.com/watch?v=fCoavgGZ64Y&t=2796s){.link-annotated-not .include-content .width-full} ::: ## `“A Law of Robustness and the Importance of Overparameterization in Deep Learning”`{=html} {#_4TXQZntA-section} **[`A law of robustness and the importance of overparameterization in deep learning`{=html}](https://www.youtube.com/watch?v=ujMvnQpP528 "A law of robustness and the importance of overparameterization in deep learning"){.id-not .include-annotation link-icon="youtube" link-icon-type="svg" link-icon-color="#ff0033"}** ::: aux-links-transclude-file [`https://www.youtube.com/watch?v=ujMvnQpP528`](https://www.youtube.com/watch?v=ujMvnQpP528){.link-annotated-not .include-content .width-full} ::: ## `“WELM”`{=html} {#_xx_M58kZ-section} **[`WELM`{=html}](https://www.youtube.com/watch?v=x-9KxACAPIo&t=19310 "WELM"){.link-annotated-partial .id-not .include-annotation link-icon="youtube" link-icon-type="svg" link-icon-color="#ff0033"}** ## `Bayesian0_0`{=html} {#_jWBpiZzy-section} **[`[Meta-analysis of AI benchmarks]`{=html}](https://x.com/Bayesian0_0/status/2084048749429497903 "'[Meta-analysis of AI benchmarks]', Bayesian0_0 2026"){.link-modified-recently .link-annotated .id-not .include-annotation link-icon="twitter" link-icon-type="svg" link-icon-color="#1da1f2"}** ## `ImMrShields`{=html} {#_fh0Wq9Q_-section} **[`[GPT-4.5 pretraining finally beaten internally?]`{=html}](https://x.com/ImMrShields/status/1995886770739892580 "'[GPT-4.5 pretraining finally beaten internally?]', ImMrShields 2026"){.id-not .include-annotation link-icon="twitter" link-icon-type="svg" link-icon-color="#1da1f2"}** ::: aux-links-transclude-file [`https://x.com/ImMrShields/status/1995886770739892580`](https://x.com/ImMrShields/status/1995886770739892580 "'[GPT-4.5 pretraining finally beaten internally?]', ImMrShields 2026"){.id-not .include-content} ::: ## `xiao_ted`{=html} {#_NeSNB0qw-section} **[`I’m curious how much overlap there is between ‘serious AI researchers’ and AI-adjacent subcultures`{=html}](https://x.com/xiao_ted/status/1916936738842546413 "'I’m curious how much overlap there is between ‘serious AI researchers’ and AI-adjacent subcultures', xiao_ted 2026"){.link-annotated .id-not .include-annotation link-icon="twitter" link-icon-type="svg" link-icon-color="#1da1f2"}** ## `“Yuxi on the Wired”, Liu 2026`{=html} {#_fmICcbvv-section} **[`Yuxi on the Wired`{=html}](https://yuxi.ml/ "'Yuxi on the Wired', Liu 2026"){.id-not .include-annotation link-icon="Yuxi" link-icon-type="text,mono,quad" link-icon-color="#2c3e50"}** ## `“Zhengdong Wang Homepage”, Wang 2026`{=html} {#_zpuCsFLX-section} **[`Zhengdong Wang homepage`{=html}](https://zhengdongwang.com/ "'Zhengdong Wang homepage', Wang 2026"){.link-annotated-partial .id-not .include-annotation}** ## Sort By Magic ::: {demo-type="sort-by-magic-preface"} Annotations sorted by machine learning into [inferred \'tags\'](/design#future-tag-features){#gwern-design--future-tag-features .link-annotated-partial}. This provides an alternative way to browse: instead of by *date* order, one can browse in *topic* order. The \'sorted\' list has been automatically clustered into multiple sections & auto-labeled for easier browsing. Beginning with the newest annotation, it uses the embedding of each annotation to attempt to create a list of nearest-neighbor annotations, creating a progression of topics. For more details, see the link. ::: ### `visual-analytics` {.collapse title="Machine-generated tag name for the following cluster of links."} [\[see previous entry\]](#henry-2025-section){.include .include-even-when-collapsed} [\[see previous entry\]](#smith-et-al-1994-section){.include .include-even-when-collapsed} ### `benchmark-analysis` {.collapse title="Machine-generated tag name for the following cluster of links."} [\[see previous entry\]](#abel-2019-section){.include .include-even-when-collapsed} [\[see previous entry\]](#_jWBpiZzy-section){.include .include-even-when-collapsed} ### `ai-mathematics ai-co-pilot knowledge-limits proof-checkers assistive-ai mathematical-ai` {.collapse title="Machine-generated tag name for the following cluster of links."} [\[see previous entry\]](#drösser-tao-2024-section){.include .include-even-when-collapsed} [\[see previous entry\]](#cheng-et-al-2024-3-section){.include .include-even-when-collapsed} ### `xai-struggles` {.collapse title="Machine-generated tag name for the following cluster of links."} [\[see previous entry\]](#siegel-kirkwood-2024-section){.include .include-even-when-collapsed} [\[see previous entry\]](#morris-criddle-2026-section){.include .include-even-when-collapsed} ### `text-completion` {.collapse title="Machine-generated tag name for the following cluster of links."} [\[see previous entry\]](#gpt-3-2020-page-48-section){.include .include-even-when-collapsed} [\[see previous entry\]](#harrington-1940-section){.include .include-even-when-collapsed} ### `controllable-generation` {.collapse title="Machine-generated tag name for the following cluster of links."} [\[see previous entry\]](#keskar-et-al-2019-section){.include .include-even-when-collapsed} [\[see previous entry\]](#radford-et-al-2018-page-5-section){.include .include-even-when-collapsed} ### `contextual-prediction` {.collapse title="Machine-generated tag name for the following cluster of links."} [\[see previous entry\]](#zhang-et-al-2021-krissbert-section){.include .include-even-when-collapsed} [\[see previous entry\]](#paperno-et-al-2016-section){.include .include-even-when-collapsed} ### `embedding-techniques, mathematical-verifiers, geometry-embeddings, wafer-scale, self-supervised-learning` {.collapse title="Machine-generated tag name for the following cluster of links."} [\[see previous entry\]](#rocki-et-al-2020-section){.include .include-even-when-collapsed} [\[see previous entry\]](#cobbe-et-al-2021-section){.include .include-even-when-collapsed} [\[see previous entry\]](#kirstain-et-al-2021-section){.include .include-even-when-collapsed} [\[see previous entry\]](#_VZDsjpId-section){.include .include-even-when-collapsed} [\[see previous entry\]](#jha-et-al-2025-section){.include .include-even-when-collapsed} [\[see previous entry\]](#gruver-et-al-2022-section){.include .include-even-when-collapsed} [\[see previous entry\]](#stein-et-al-2021-1-section){.include .include-even-when-collapsed} ### `scaling-reduction fastfold chinchilla-replication training-optimization model-efficiency scaling-attempt` {.collapse title="Machine-generated tag name for the following cluster of links."} [\[see previous entry\]](#besiroglu-et-al-2024-section){.include .include-even-when-collapsed} [\[see previous entry\]](#cheng-et-al-2022-2-section){.include .include-even-when-collapsed} ### `random-satisfiability` {.collapse title="Machine-generated tag name for the following cluster of links."} [\[see previous entry\]](#grace-2013-section){.include .include-even-when-collapsed} [\[see previous entry\]](#mezard-et-al-2002-section){.include .include-even-when-collapsed} ### `language-modeling text-generation narrative-analysis nlp-history speech-recognition language-technique` {.collapse title="Machine-generated tag name for the following cluster of links."} [\[see previous entry\]](#karpathy-2015-section){.include .include-even-when-collapsed} [\[see previous entry\]](#lipton-et-al-2015-section){.include .include-even-when-collapsed} [\[see previous entry\]](#cambria-white-2014-section){.include .include-even-when-collapsed} [\[see previous entry\]](#hannun-2021-2-section){.include .include-even-when-collapsed} ### `adversarial-strategies` {.collapse title="Machine-generated tag name for the following cluster of links."} [\[see previous entry\]](#wang-et-al-2022-08-section){.include .include-even-when-collapsed} [\[see previous entry\]](#hughes-et-al-2024-section){.include .include-even-when-collapsed} ### `agi-development` {.collapse title="Machine-generated tag name for the following cluster of links."} [\[see previous entry\]](#liu-2025-cyc-section){.include .include-even-when-collapsed} [\[see previous entry\]](#carmack-2023-section){.include .include-even-when-collapsed} [\[see previous entry\]](#altman-2023-1-section){.include .include-even-when-collapsed} [\[see previous entry\]](#gorrell-2023-section){.include .include-even-when-collapsed} ### `adversarial-robustness` {.collapse title="Machine-generated tag name for the following cluster of links."} [\[see previous entry\]](#burns-et-al-2025-section){.include .include-even-when-collapsed} [\[see previous entry\]](#merriënboer-et-al-2025-section){.include .include-even-when-collapsed} [\[see previous entry\]](#ding-et-al-2023-1-section){.include .include-even-when-collapsed} [\[see previous entry\]](#carreira-zisserman-2017-section){.include .include-even-when-collapsed} [\[see previous entry\]](#hutchinson-et-al-2020-section){.include .include-even-when-collapsed} [\[see previous entry\]](#yeung-et-al-2017-section){.include .include-even-when-collapsed} [\[see previous entry\]](#orhan-et-al-2020-section){.include .include-even-when-collapsed} [\[see previous entry\]](#dobs-et-al-2021-section){.include .include-even-when-collapsed} [\[see previous entry\]](#olah-et-al-2020-section){.include .include-even-when-collapsed} [\[see previous entry\]](#shamir-et-al-2021-section){.include .include-even-when-collapsed} [\[see previous entry\]](#ilyas-et-al-2019-section){.include .include-even-when-collapsed} [\[see previous entry\]](#uesato-et-al-2019-section){.include .include-even-when-collapsed} [\[see previous entry\]](#zhai-et-al-2019-section){.include .include-even-when-collapsed} [\[see previous entry\]](#geiping-et-al-2021-section){.include .include-even-when-collapsed} [\[see previous entry\]](#nado-et-al-2021-section){.include .include-even-when-collapsed} [\[see previous entry\]](#tang-et-al-2021-1bitadam-section){.include .include-even-when-collapsed} [\[see previous entry\]](#wang-et-al-2021-05-section){.include .include-even-when-collapsed} [\[see previous entry\]](#yuan-et-al-2022-2-section){.include .include-even-when-collapsed} [\[see previous entry\]](#xin-et-al-2022-section){.include .include-even-when-collapsed} [\[see previous entry\]](#musgrave-et-al-2020-section){.include .include-even-when-collapsed} [\[see previous entry\]](#lucic-et-al-2017-section){.include .include-even-when-collapsed} [\[see previous entry\]](#dodge-et-al-2019-section){.include .include-even-when-collapsed} [\[see previous entry\]](#bowman-2021-section){.include .include-even-when-collapsed} [\[see previous entry\]](#lipton-steinhardt-2018-section){.include .include-even-when-collapsed} [\[see previous entry\]](#breiman-1995-section){.include .include-even-when-collapsed} [\[see previous entry\]](#alexander-2018-2-section){.include .include-even-when-collapsed} [\[see previous entry\]](#hanson-2008-2-section){.include .include-even-when-collapsed} [\[see previous entry\]](#yudkowsky-2013-section){.include .include-even-when-collapsed} [\[see previous entry\]](#sotala-2012-section){.include .include-even-when-collapsed} [\[see previous entry\]](#chalmers-2016-section){.include .include-even-when-collapsed} [\[see previous entry\]](#garfinkel-et-al-2017-section){.include .include-even-when-collapsed} [\[see previous entry\]](#cochrane-et-al-1995-section){.include .include-even-when-collapsed} ### `question-answering` {.collapse title="Machine-generated tag name for the following cluster of links."} [\[see previous entry\]](#arora-et-al-2023-1-section){.include .include-even-when-collapsed} [\[see previous entry\]](#tafjord-clark-2021-section){.include .include-even-when-collapsed} ### `model-optimization` {.collapse title="Machine-generated tag name for the following cluster of links."} [\[see previous entry\]](#brockman-et-al-2020-section){.include .include-even-when-collapsed} [\[see previous entry\]](#wang-et-al-2021-gpt3labeling-section){.include .include-even-when-collapsed} [\[see previous entry\]](#dinh-et-al-2022-1-section){.include .include-even-when-collapsed} [\[see previous entry\]](#howard-ruder-2018-section){.include .include-even-when-collapsed} [\[see previous entry\]](#lu-et-al-2021-3-section){.include .include-even-when-collapsed} [\[see previous entry\]](#hasson-et-al-2020-section){.include .include-even-when-collapsed} [\[see previous entry\]](#saxe-et-al-2019-section){.include .include-even-when-collapsed} [\[see previous entry\]](#hill-et-al-2019-2-section){.include .include-even-when-collapsed} [\[see previous entry\]](#eslami-et-al-2018-section){.include .include-even-when-collapsed} [\[see previous entry\]](#santoro-et-al-2017-section){.include .include-even-when-collapsed} [\[see previous entry\]](#veit-et-al-2016-section){.include .include-even-when-collapsed} [\[see previous entry\]](#brock-et-al-2021-section){.include .include-even-when-collapsed} [\[see previous entry\]](#vasu-et-al-2022-section){.include .include-even-when-collapsed} [\[see previous entry\]](#fedorov-et-al-2019-section){.include .include-even-when-collapsed} [\[see previous entry\]](#lin-et-al-2022-smolml-section){.include .include-even-when-collapsed} [\[see previous entry\]](#zeng-et-al-2020-section){.include .include-even-when-collapsed} [\[see previous entry\]](#dean-2019-section){.include .include-even-when-collapsed} [\[see previous entry\]](#horowitz-2014b-section){.include .include-even-when-collapsed} [\[see previous entry\]](#hameed-et-al-2010-section){.include .include-even-when-collapsed} [\[see previous entry\]](#wiggers-2021-section){.include .include-even-when-collapsed} [\[see previous entry\]](#laanait-et-al-2019-section){.include .include-even-when-collapsed} [\[see previous entry\]](#jia-et-al-2020-section){.include .include-even-when-collapsed} [\[see previous entry\]](#edamadaka-et-al-2025-section){.include .include-even-when-collapsed} [\[see previous entry\]](#huh-et-al-2024-section){.include .include-even-when-collapsed} [\[see previous entry\]](#gröger-et-al-2026-section){.include .include-even-when-collapsed} [\[see previous entry\]](#frank-2018-section){.include .include-even-when-collapsed} [\[see previous entry\]](#yang-zhang-2023-section){.include .include-even-when-collapsed} [\[see previous entry\]](#wang-et-al-2019-section){.include .include-even-when-collapsed} [\[see previous entry\]](#weinberger-2020-section){.include .include-even-when-collapsed} [\[see previous entry\]](#lin-et-al-2016-2-section){.include .include-even-when-collapsed} [\[see previous entry\]](#chen-et-al-2023-06-section){.include .include-even-when-collapsed} [\[see previous entry\]](#mingard-et-al-2020-section){.include .include-even-when-collapsed} [\[see previous entry\]](#izmailov-et-al-2021-section){.include .include-even-when-collapsed} [\[see previous entry\]](#nichol-dhariwal-2021-section){.include .include-even-when-collapsed} [\[see previous entry\]](#ho-et-al-2020-3-section){.include .include-even-when-collapsed} [\[see previous entry\]](#gerstgrasser-et-al-2024-section){.include .include-even-when-collapsed} [\[see previous entry\]](#feng-et-al-2024-2-section){.include .include-even-when-collapsed} [\[see previous entry\]](#ferbach-et-al-2024-section){.include .include-even-when-collapsed} [\[see previous entry\]](#zhang-et-al-2025-section){.include .include-even-when-collapsed} [\[see previous entry\]](#zou-et-al-2021-section){.include .include-even-when-collapsed} [\[see previous entry\]](#wang-et-al-2024-10-section){.include .include-even-when-collapsed} [\[see previous entry\]](#reif-et-al-2021-section){.include .include-even-when-collapsed} [\[see previous entry\]](#pippi-et-al-2025-section){.include .include-even-when-collapsed} [\[see previous entry\]](#mokady-et-al-2022-2-section){.include .include-even-when-collapsed} [\[see previous entry\]](#reed-et-al-2017-section){.include .include-even-when-collapsed} [\[see previous entry\]](#weyand-et-al-2016-section){.include .include-even-when-collapsed} [\[see previous entry\]](#manvi-et-al-2023-section){.include .include-even-when-collapsed} [\[see previous entry\]](#zhuang-et-al-2024-section){.include .include-even-when-collapsed} [\[see previous entry\]](#aghajanyan-et-al-2021-1-section){.include .include-even-when-collapsed} [\[see previous entry\]](#ye-et-al-2024-section){.include .include-even-when-collapsed} [\[see previous entry\]](#bavarian-et-al-2022-section){.include .include-even-when-collapsed} [\[see previous entry\]](#mansimov-et-al-2020-section){.include .include-even-when-collapsed} [\[see previous entry\]](#tran-et-al-2021-2-section){.include .include-even-when-collapsed} [\[see previous entry\]](#bapna-et-al-2019-section){.include .include-even-when-collapsed} [\[see previous entry\]](#rawat-et-al-2021-section){.include .include-even-when-collapsed} [\[see previous entry\]](#chen-et-al-2015-1-section){.include .include-even-when-collapsed} [\[see previous entry\]](#schmidhuber-2018-section){.include .include-even-when-collapsed} [\[see previous entry\]](#scholl-2020-section){.include .include-even-when-collapsed} [\[see previous entry\]](#lerousseau-summerfield-2025-section){.include .include-even-when-collapsed} [\[see previous entry\]](#raventós-et-al-2023-section){.include .include-even-when-collapsed} [\[see previous entry\]](#akhauri-et-al-2025-section){.include .include-even-when-collapsed} [\[see previous entry\]](#yuan-liu-2022-section){.include .include-even-when-collapsed} [\[see previous entry\]](#khashabi-et-al-2022-section){.include .include-even-when-collapsed} [\[see previous entry\]](#lourie-et-al-2021-section){.include .include-even-when-collapsed} [\[see previous entry\]](#sakaguchi-et-al-2020-section){.include .include-even-when-collapsed} [\[see previous entry\]](#kocijan-et-al-2020-section){.include .include-even-when-collapsed} [\[see previous entry\]](#kocijan-et-al-2022-section){.include .include-even-when-collapsed} [\[see previous entry\]](#lin-et-al-2020-4-section){.include .include-even-when-collapsed} [\[see previous entry\]](#cole-2024-section){.include .include-even-when-collapsed} [\[see previous entry\]](#gwern-doc-ai-scaling-abstract-section){.include .include-even-when-collapsed} [\[see previous entry\]](#besiroglu-2025-section){.include .include-even-when-collapsed} [\[see previous entry\]](#erdil-sevilla-2023-section){.include .include-even-when-collapsed} [\[see previous entry\]](#kang-et-al-2024-section){.include .include-even-when-collapsed} [\[see previous entry\]](#motamed-et-al-2025-section){.include .include-even-when-collapsed} [\[see previous entry\]](#ansari-et-al-2026-section){.include .include-even-when-collapsed} [\[see previous entry\]](#taylor-et-al-2022-section){.include .include-even-when-collapsed} [\[see previous entry\]](#azulay-2025-section){.include .include-even-when-collapsed} [\[see previous entry\]](#roberts-et-al-2020-2-section){.include .include-even-when-collapsed} [\[see previous entry\]](#lee-et-al-2021-3-section){.include .include-even-when-collapsed} [\[see previous entry\]](#wang-et-al-2022-13-section){.include .include-even-when-collapsed} [\[see previous entry\]](#jacob-et-al-2024-section){.include .include-even-when-collapsed} [\[see previous entry\]](#brin-page-1998-section){.include .include-even-when-collapsed} [\[see previous entry\]](#koren-et-al-2009-section){.include .include-even-when-collapsed} [\[see previous entry\]](#liu-et-al-2022-12-section){.include .include-even-when-collapsed} [\[see previous entry\]](#huang-et-al-2023-section){.include .include-even-when-collapsed} [\[see previous entry\]](#wu-et-al-2023-5-section){.include .include-even-when-collapsed} [\[see previous entry\]](#zhao-et-al-2023-6-section){.include .include-even-when-collapsed} [\[see previous entry\]](#eichenberg-et-al-2021-section){.include .include-even-when-collapsed} [\[see previous entry\]](#alayrac-et-al-2022-section){.include .include-even-when-collapsed} [\[see previous entry\]](#yuan-et-al-2021-1-section){.include .include-even-when-collapsed} [\[see previous entry\]](#kiraly-traverse-2024-section){.include .include-even-when-collapsed} [\[see previous entry\]](#zweig-et-al-2021-section){.include .include-even-when-collapsed} [\[see previous entry\]](#berg-et-al-2020-section){.include .include-even-when-collapsed} [\[see previous entry\]](#bell-et-al-2020-section){.include .include-even-when-collapsed} [\[see previous entry\]](#zhu-et-al-2022-5-section){.include .include-even-when-collapsed} [\[see previous entry\]](#thomee-et-al-2015-section){.include .include-even-when-collapsed} [\[see previous entry\]](#schuhmann-et-al-2021-section){.include .include-even-when-collapsed} [\[see previous entry\]](#feng-et-al-2022-1-section){.include .include-even-when-collapsed} [\[see previous entry\]](#yin-et-al-2023-section){.include .include-even-when-collapsed} [\[see previous entry\]](#gupta-et-al-2022-1-section){.include .include-even-when-collapsed} [\[see previous entry\]](#pi-et-al-2022-section){.include .include-even-when-collapsed} [\[see previous entry\]](#west-et-al-2021-section){.include .include-even-when-collapsed} [\[see previous entry\]](#zhang-et-al-2021-04-section){.include .include-even-when-collapsed} [\[see previous entry\]](#chai-et-al-2022-section){.include .include-even-when-collapsed} ### `vision-transformers` {.collapse title="Machine-generated tag name for the following cluster of links."} [\[see previous entry\]](#_NeSNB0qw-section){.include .include-even-when-collapsed} [\[see previous entry\]](#field-et-al-2026-section){.include .include-even-when-collapsed} [\[see previous entry\]](#altman-2015-2-section){.include .include-even-when-collapsed} [\[see previous entry\]](#mitchell-2019-section){.include .include-even-when-collapsed} [\[see previous entry\]](#hofstadter-kim-2023-section){.include .include-even-when-collapsed} [\[see previous entry\]](#mitchell-chugg-2022-section){.include .include-even-when-collapsed} [\[see previous entry\]](#kiely-2022-section){.include .include-even-when-collapsed} [\[see previous entry\]](#wood-2009-section){.include .include-even-when-collapsed} [\[see previous entry\]](#legg-2009-tick-tock-section){.include .include-even-when-collapsed} [\[see previous entry\]](#legg-2009-the-teenies-section){.include .include-even-when-collapsed} [\[see previous entry\]](#moravec-2004-section){.include .include-even-when-collapsed} [\[see previous entry\]](#carlsmith-2020-section){.include .include-even-when-collapsed} [\[see previous entry\]](#hofman-2015-section){.include .include-even-when-collapsed} [\[see previous entry\]](#hofman-2014-section){.include .include-even-when-collapsed} [\[see previous entry\]](#herculano-houzel-2012-section){.include .include-even-when-collapsed} [\[see previous entry\]](#herculano-houzel-et-al-2007-section){.include .include-even-when-collapsed} [\[see previous entry\]](#bryer-et-al-2021-section){.include .include-even-when-collapsed} [\[see previous entry\]](#maloney-et-al-2022-section){.include .include-even-when-collapsed} [\[see previous entry\]](#kaplan-et-al-2020-section){.include .include-even-when-collapsed} [\[see previous entry\]](#henighan-et-al-2020-section){.include .include-even-when-collapsed} [\[see previous entry\]](#cherti-et-al-2022-section){.include .include-even-when-collapsed} [\[see previous entry\]](#dehghani-et-al-2023-section){.include .include-even-when-collapsed} [\[see previous entry\]](#zhai-et-al-2021-3-section){.include .include-even-when-collapsed} [\[see previous entry\]](#dosovitskiy-et-al-2020-section){.include .include-even-when-collapsed} [\[see previous entry\]](#raghu-et-al-2021-section){.include .include-even-when-collapsed} [\[see previous entry\]](#bhojanapalli-et-al-2021-section){.include .include-even-when-collapsed} [\[see previous entry\]](#djolonga-et-al-2020-section){.include .include-even-when-collapsed} [\[see previous entry\]](#taori-et-al-2020-section){.include .include-even-when-collapsed} [\[see previous entry\]](#shankar-et-al-2021-section){.include .include-even-when-collapsed} [\[see previous entry\]](#geirhos-et-al-2021-section){.include .include-even-when-collapsed} [\[see previous entry\]](#fort-et-al-2021-section){.include .include-even-when-collapsed} [\[see previous entry\]](#fang-et-al-2022-4-section){.include .include-even-when-collapsed} [\[see previous entry\]](#nguyen-et-al-2022-section){.include .include-even-when-collapsed} [\[see previous entry\]](#jia-et-al-2021-section){.include .include-even-when-collapsed} [\[see previous entry\]](#pham-et-al-2021-1-section){.include .include-even-when-collapsed} [\[see previous entry\]](#fang-et-al-2022-1-section){.include .include-even-when-collapsed} [\[see previous entry\]](#he-et-al-2021-1-section){.include .include-even-when-collapsed} [\[see previous entry\]](#geng-et-al-2022-section){.include .include-even-when-collapsed} [\[see previous entry\]](#aghajanyan-et-al-2023-section){.include .include-even-when-collapsed} [\[see previous entry\]](#wu-et-al-2024-section){.include .include-even-when-collapsed} [\[see previous entry\]](#bai-et-al-2023-2-section){.include .include-even-when-collapsed} [\[see previous entry\]](#chen-et-al-2022-pali-section){.include .include-even-when-collapsed} [\[see previous entry\]](#chen-et-al-2023-palix-section){.include .include-even-when-collapsed} [\[see previous entry\]](#chen-et-al-2023-pali3-section){.include .include-even-when-collapsed} [\[see previous entry\]](#steiner-et-al-2024-section){.include .include-even-when-collapsed} [\[see previous entry\]](#anil-et-al-2023-section){.include .include-even-when-collapsed} [\[see previous entry\]](#elias-2023-1-section){.include .include-even-when-collapsed} [\[see previous entry\]](#hoffmann-et-al-2022-section){.include .include-even-when-collapsed} [\[see previous entry\]](#hu-et-al-2024-1-section){.include .include-even-when-collapsed} [\[see previous entry\]](#zhang-et-al-2020-04-section){.include .include-even-when-collapsed} [\[see previous entry\]](#wu-et-al-2021-yuan-1-section){.include .include-even-when-collapsed} [\[see previous entry\]](#wang-et-al-2021-ernie30titan-section){.include .include-even-when-collapsed} [\[see previous entry\]](#sun-et-al-2021-4-section){.include .include-even-when-collapsed} [\[see previous entry\]](#zeng-et-al-2022-1-section){.include .include-even-when-collapsed} [\[see previous entry\]](#adlr-2019-section){.include .include-even-when-collapsed} [\[see previous entry\]](#smith-et-al-2022-3-section){.include .include-even-when-collapsed} [\[see previous entry\]](#rosset-2020-section){.include .include-even-when-collapsed} [\[see previous entry\]](#rajbhandari-et-al-2019-section){.include .include-even-when-collapsed} [\[see previous entry\]](#team-2020-1-section){.include .include-even-when-collapsed} [\[see previous entry\]](#lin-et-al-2021-m610t-section){.include .include-even-when-collapsed} [\[see previous entry\]](#jaszczur-et-al-2021-section){.include .include-even-when-collapsed} [\[see previous entry\]](#tay-et-al-2021-1-section){.include .include-even-when-collapsed} [\[see previous entry\]](#hernandez-et-al-2021-2-section){.include .include-even-when-collapsed} [\[see previous entry\]](#ghorbani-et-al-2021-section){.include .include-even-when-collapsed} [\[see previous entry\]](#gordon-et-al-2021-section){.include .include-even-when-collapsed} [\[see previous entry\]](#bansal-et-al-2022-nmtscaling-section){.include .include-even-when-collapsed} [\[see previous entry\]](#zhang-et-al-2024-07-section){.include .include-even-when-collapsed} [\[see previous entry\]](#porian-et-al-2024-section){.include .include-even-when-collapsed} [\[see previous entry\]](#scao-2020-section){.include .include-even-when-collapsed} [\[see previous entry\]](#tao-et-al-2024-section){.include .include-even-when-collapsed} [\[see previous entry\]](#huang-et-al-2025-section){.include .include-even-when-collapsed} [\[see previous entry\]](#godey-et-al-2024-section){.include .include-even-when-collapsed} [\[see previous entry\]](#pagnoni-et-al-2024-section){.include .include-even-when-collapsed} [\[see previous entry\]](#perić-2025-section){.include .include-even-when-collapsed} [\[see previous entry\]](#chao-et-al-2026-section){.include .include-even-when-collapsed} [\[see previous entry\]](#nie-et-al-2024-section){.include .include-even-when-collapsed} [\[see previous entry\]](#nie-et-al-2025-section){.include .include-even-when-collapsed} [\[see previous entry\]](#li-et-al-2021-7-section){.include .include-even-when-collapsed} [\[see previous entry\]](#amodei-et-al-2015-section){.include .include-even-when-collapsed} [\[see previous entry\]](#hannun-et-al-2014-section){.include .include-even-when-collapsed} [\[see previous entry\]](#baevski-et-al-2021-section){.include .include-even-when-collapsed} [\[see previous entry\]](#baevski-et-al-2020-section){.include .include-even-when-collapsed} [\[see previous entry\]](#conneau-et-al-2020-section){.include .include-even-when-collapsed} [\[see previous entry\]](#babu-et-al-2021-section){.include .include-even-when-collapsed} [\[see previous entry\]](#conneau-et-al-2019-section){.include .include-even-when-collapsed} [\[see previous entry\]](#goyal-et-al-2021-xlmrxl-section){.include .include-even-when-collapsed} [\[see previous entry\]](#fair-2019-section){.include .include-even-when-collapsed} [\[see previous entry\]](#lample-conneau-2019-section){.include .include-even-when-collapsed} [\[see previous entry\]](#fan-et-al-2020-1-section){.include .include-even-when-collapsed} [\[see previous entry\]](#fan-2020-section){.include .include-even-when-collapsed} [\[see previous entry\]](#communication-et-al-2023-2-section){.include .include-even-when-collapsed} [\[see previous entry\]](#communication-et-al-2023-1-section){.include .include-even-when-collapsed} [\[see previous entry\]](#wang-et-al-2021-wav2vec20-section){.include .include-even-when-collapsed} [\[see previous entry\]](#zhang-et-al-2020-06-section){.include .include-even-when-collapsed} [\[see previous entry\]](#zhang-et-al-2021-06-section){.include .include-even-when-collapsed} [\[see previous entry\]](#zhang-et-al-2024-12-section){.include .include-even-when-collapsed} [\[see previous entry\]](#shor-et-al-2021-section){.include .include-even-when-collapsed} [\[see previous entry\]](#xiao-et-al-2021-2-section){.include .include-even-when-collapsed} [\[see previous entry\]](#chelba-et-al-2012-section){.include .include-even-when-collapsed} [\[see previous entry\]](#jozefowicz-et-al-2016-section){.include .include-even-when-collapsed} [\[see previous entry\]](#chelba-et-al-2013-section){.include .include-even-when-collapsed} [\[see previous entry\]](#buck-et-al-2014-section){.include .include-even-when-collapsed} [\[see previous entry\]](#brants-et-al-2007-section){.include .include-even-when-collapsed} [\[see previous entry\]](#meng-et-al-2019-section){.include .include-even-when-collapsed} [\[see previous entry\]](#arivazhagan-et-al-2019-section){.include .include-even-when-collapsed} [\[see previous entry\]](#bapna-et-al-2022-section){.include .include-even-when-collapsed} [\[see previous entry\]](#caswell-et-al-2020-section){.include .include-even-when-collapsed} [\[see previous entry\]](#dou-et-al-2021-section){.include .include-even-when-collapsed} [\[see previous entry\]](#press-et-al-2022-section){.include .include-even-when-collapsed} [\[see previous entry\]](#wang-et-al-2022-20-section){.include .include-even-when-collapsed} [\[see previous entry\]](#tay-et-al-2022-ul2-section){.include .include-even-when-collapsed} [\[see previous entry\]](#chung-et-al-2022-section){.include .include-even-when-collapsed} [\[see previous entry\]](#wang-et-al-2022-16-section){.include .include-even-when-collapsed} [\[see previous entry\]](#su-et-al-2022-1-section){.include .include-even-when-collapsed} [\[see previous entry\]](#neelakantan-et-al-2022-section){.include .include-even-when-collapsed} [\[see previous entry\]](#radford-et-al-2021-section){.include .include-even-when-collapsed} [\[see previous entry\]](#chen-et-al-2020-blog-section){.include .include-even-when-collapsed} [\[see previous entry\]](#jukebox-blog-section){.include .include-even-when-collapsed} [\[see previous entry\]](#lee-et-al-2022-07-section){.include .include-even-when-collapsed} [\[see previous entry\]](#wang-et-al-2023-18-section){.include .include-even-when-collapsed} [\[see previous entry\]](#betker-2023-section){.include .include-even-when-collapsed} [\[see previous entry\]](#sauer-et-al-2023-2-section){.include .include-even-when-collapsed} [\[see previous entry\]](#kang-et-al-2023-section){.include .include-even-when-collapsed} [\[see previous entry\]](#sauer-et-al-2022-section){.include .include-even-when-collapsed} [\[see previous entry\]](#brock-et-al-2018-section){.include .include-even-when-collapsed} [\[see previous entry\]](#donahue-simonyan-2019-section){.include .include-even-when-collapsed} [\[see previous entry\]](#brock-et-al-2018-page-8-org-deepmind-section){.include .include-even-when-collapsed} [\[see previous entry\]](#sun-et-al-2017-2-section){.include .include-even-when-collapsed} [\[see previous entry\]](#yu-et-al-2015-section){.include .include-even-when-collapsed} [\[see previous entry\]](#guo-et-al-2018-2-section){.include .include-even-when-collapsed} [\[see previous entry\]](#li-et-al-2017-2-section){.include .include-even-when-collapsed} [\[see previous entry\]](#li-et-al-2017-1-section){.include .include-even-when-collapsed} [\[see previous entry\]](#gu-et-al-2022-2-section){.include .include-even-when-collapsed} [\[see previous entry\]](#huo-et-al-2021-section){.include .include-even-when-collapsed} [\[see previous entry\]](#zhang-et-al-2024-01-section){.include .include-even-when-collapsed} [\[see previous entry\]](#wang-et-al-2022-21-section){.include .include-even-when-collapsed} [\[see previous entry\]](#yu-et-al-2022-3-section){.include .include-even-when-collapsed} [\[see previous entry\]](#yan-et-al-2022-2-section){.include .include-even-when-collapsed} [\[see previous entry\]](#sun-et-al-2019-2-section){.include .include-even-when-collapsed} [\[see previous entry\]](#cheng-et-al-2022-1-section){.include .include-even-when-collapsed} [\[see previous entry\]](#girdhar-et-al-2023-section){.include .include-even-when-collapsed} [\[see previous entry\]](#singh-et-al-2022-section){.include .include-even-when-collapsed} [\[see previous entry\]](#mahajan-et-al-2018-2-section){.include .include-even-when-collapsed} [\[see previous entry\]](#goyal-et-al-2021-seer-section){.include .include-even-when-collapsed} [\[see previous entry\]](#goyal-et-al-2022-2-section){.include .include-even-when-collapsed} [\[see previous entry\]](#oquab-et-al-2023-section){.include .include-even-when-collapsed} [\[see previous entry\]](#caron-et-al-2021-section){.include .include-even-when-collapsed} [\[see previous entry\]](#liu-et-al-2022-23-section){.include .include-even-when-collapsed} [\[see previous entry\]](#dai-et-al-2021-2-section){.include .include-even-when-collapsed} [\[see previous entry\]](#smith-et-al-2023-2-section){.include .include-even-when-collapsed} [\[see previous entry\]](#abnar-et-al-2021-section){.include .include-even-when-collapsed} [\[see previous entry\]](#li-et-al-2020-5-section){.include .include-even-when-collapsed} [\[see previous entry\]](#tan-le-2019-section){.include .include-even-when-collapsed} [\[see previous entry\]](#bello-et-al-2021-section){.include .include-even-when-collapsed} [\[see previous entry\]](#dollár-et-al-2021-section){.include .include-even-when-collapsed} [\[see previous entry\]](#hestness-et-al-2017-section){.include .include-even-when-collapsed} [\[see previous entry\]](#hutter-2021-section){.include .include-even-when-collapsed} [\[see previous entry\]](#nakkiran-2021-section){.include .include-even-when-collapsed} [\[see previous entry\]](#sharma-kaplan-2020-section){.include .include-even-when-collapsed} [\[see previous entry\]](#bahri-et-al-2021-2-section){.include .include-even-when-collapsed} [\[see previous entry\]](#bordelon-et-al-2024-section){.include .include-even-when-collapsed} [\[see previous entry\]](#rosenfeld-2021-section){.include .include-even-when-collapsed} [\[see previous entry\]](#rosenfeld-et-al-2020-section){.include .include-even-when-collapsed} [\[see previous entry\]](#sorscher-et-al-2022-section){.include .include-even-when-collapsed} [\[see previous entry\]](#prato-et-al-2021-1-section){.include .include-even-when-collapsed} [\[see previous entry\]](#fang-et-al-2024-section){.include .include-even-when-collapsed} [\[see previous entry\]](#ardalani-et-al-2022-section){.include .include-even-when-collapsed} [\[see previous entry\]](#shin-et-al-2021-1-section){.include .include-even-when-collapsed} [\[see previous entry\]](#shin-et-al-2021-2-section){.include .include-even-when-collapsed} [\[see previous entry\]](#beal-et-al-2021-section){.include .include-even-when-collapsed} [\[see previous entry\]](#tschannen-et-al-2023-2-section){.include .include-even-when-collapsed} [\[see previous entry\]](#hu-et-al-2021-2-section){.include .include-even-when-collapsed} [\[see previous entry\]](#changpinyo-et-al-2021-section){.include .include-even-when-collapsed} [\[see previous entry\]](#agrawal-et-al-2018-section){.include .include-even-when-collapsed} [\[see previous entry\]](#desai-et-al-2021-section){.include .include-even-when-collapsed} [\[see previous entry\]](#lee-et-al-2021-acav100m-section){.include .include-even-when-collapsed} [\[see previous entry\]](#wang-et-al-2023-05-section){.include .include-even-when-collapsed} [\[see previous entry\]](#zhang-et-al-2023-07-section){.include .include-even-when-collapsed} [\[see previous entry\]](#blattmann-et-al-2023-section){.include .include-even-when-collapsed} [\[see previous entry\]](#weissenborn-et-al-2019-section){.include .include-even-when-collapsed} [\[see previous entry\]](#villegas-et-al-2019-section){.include .include-even-when-collapsed} [\[see previous entry\]](#villegas-et-al-2019-2-section){.include .include-even-when-collapsed} [\[see previous entry\]](#wu-et-al-2021-ghvae-section){.include .include-even-when-collapsed} [\[see previous entry\]](#vahdat-kautz-2020-section){.include .include-even-when-collapsed} [\[see previous entry\]](#child-2020-section){.include .include-even-when-collapsed} [\[see previous entry\]](#daly-et-al-2022-section){.include .include-even-when-collapsed} [\[see previous entry\]](#tian-et-al-2024-section){.include .include-even-when-collapsed} [\[see previous entry\]](#chen-igpt-paper-section){.include .include-even-when-collapsed} [\[see previous entry\]](#ramesh-et-al-2021-dalle-paper-section){.include .include-even-when-collapsed} [\[see previous entry\]](#kim-et-al-2021-3-section){.include .include-even-when-collapsed} [\[see previous entry\]](#chen-et-al-2019-3-section){.include .include-even-when-collapsed} [\[see previous entry\]](#dong-et-al-2019-section){.include .include-even-when-collapsed} [\[see previous entry\]](#radford-et-al-2018-section){.include .include-even-when-collapsed} [\[see previous entry\]](#openai-2018-1-section){.include .include-even-when-collapsed} [\[see previous entry\]](#gpt-2-blog-section){.include .include-even-when-collapsed} [\[see previous entry\]](#gpt-2-paper-section){.include .include-even-when-collapsed} [\[see previous entry\]](#raffel-et-al-2019-section){.include .include-even-when-collapsed} [\[see previous entry\]](#xue-et-al-2020-1-section){.include .include-even-when-collapsed} [\[see previous entry\]](#aribandi-et-al-2021-section){.include .include-even-when-collapsed} [\[see previous entry\]](#aghajanyan-et-al-2021-2-section){.include .include-even-when-collapsed} [\[see previous entry\]](#gan-isola-2026-section){.include .include-even-when-collapsed} [\[see previous entry\]](#mandal-et-al-2026-section){.include .include-even-when-collapsed} [\[see previous entry\]](#komatsuzaki-2019-section){.include .include-even-when-collapsed} [\[see previous entry\]](#geiping-goldstein-2022-section){.include .include-even-when-collapsed} [\[see previous entry\]](#muennighoff-et-al-2023-repeatscaling-section){.include .include-even-when-collapsed} [\[see previous entry\]](#kim-et-al-2025-section){.include .include-even-when-collapsed} [\[see previous entry\]](#finzi-et-al-2025-section){.include .include-even-when-collapsed} [\[see previous entry\]](#rosenfeld-et-al-2019-section){.include .include-even-when-collapsed} [\[see previous entry\]](#atanasov-et-al-2024-section){.include .include-even-when-collapsed} [\[see previous entry\]](#tripuraneni-et-al-2021-section){.include .include-even-when-collapsed} [\[see previous entry\]](#belkin-et-al-2018-section){.include .include-even-when-collapsed} [\[see previous entry\]](#yang-et-al-2020-6-section){.include .include-even-when-collapsed} [\[see previous entry\]](#dar-et-al-2021-section){.include .include-even-when-collapsed} [\[see previous entry\]](#maddox-et-al-2020-section){.include .include-even-when-collapsed} [\[see previous entry\]](#nakkiran-et-al-2019-2-section){.include .include-even-when-collapsed} [\[see previous entry\]](#nakkiran-et-al-2019-1-section){.include .include-even-when-collapsed} [\[see previous entry\]](#bachmann-et-al-2023-section){.include .include-even-when-collapsed} [\[see previous entry\]](#zhao-et-al-2021-4-section){.include .include-even-when-collapsed} [\[see previous entry\]](#tolstikhin-et-al-2021-section){.include .include-even-when-collapsed} [\[see previous entry\]](#nie-et-al-2021-section){.include .include-even-when-collapsed} [\[see previous entry\]](#madasu-lal-2023-section){.include .include-even-when-collapsed} [\[see previous entry\]](#lu-et-al-2019-2-section){.include .include-even-when-collapsed} [\[see previous entry\]](#zhang-et-al-2021-12-section){.include .include-even-when-collapsed} [\[see previous entry\]](#gan-et-al-2022-section){.include .include-even-when-collapsed} [\[see previous entry\]](#mehta-et-al-2024-section){.include .include-even-when-collapsed} [\[see previous entry\]](#li-et-al-2022-05-section){.include .include-even-when-collapsed} [\[see previous entry\]](#li-et-al-2024-05-section){.include .include-even-when-collapsed} [\[see previous entry\]](#zhai-et-al-2023-section){.include .include-even-when-collapsed} [\[see previous entry\]](#chen-et-al-2020-simclrv1-section){.include .include-even-when-collapsed} [\[see previous entry\]](#chen-et-al-2020-simclrv2-section){.include .include-even-when-collapsed} [\[see previous entry\]](#yalniz-et-al-2019-section){.include .include-even-when-collapsed} [\[see previous entry\]](#pham-et-al-2021-2-section){.include .include-even-when-collapsed} [\[see previous entry\]](#xie-et-al-2019-2-section){.include .include-even-when-collapsed} [\[see previous entry\]](#rolnick-et-al-2017-section){.include .include-even-when-collapsed} [\[see previous entry\]](#xiao-et-al-2015-section){.include .include-even-when-collapsed} [\[see previous entry\]](#krause-et-al-2015-section){.include .include-even-when-collapsed} [\[see previous entry\]](#joulin-et-al-2015-section){.include .include-even-when-collapsed} [\[see previous entry\]](#caron-et-al-2020-section){.include .include-even-when-collapsed} [\[see previous entry\]](#hénaff-et-al-2021-section){.include .include-even-when-collapsed} [\[see previous entry\]](#he-et-al-2019-1-section){.include .include-even-when-collapsed} [\[see previous entry\]](#orhan-2019-section){.include .include-even-when-collapsed} [\[see previous entry\]](#hendrycks-dietterich-2019-section){.include .include-even-when-collapsed} [\[see previous entry\]](#hendrycks-dietterich-2018-section){.include .include-even-when-collapsed} [\[see previous entry\]](#xie-yuille-2019-section){.include .include-even-when-collapsed} [\[see previous entry\]](#gowal-et-al-2020-section){.include .include-even-when-collapsed} [\[see previous entry\]](#madry-et-al-2017-section){.include .include-even-when-collapsed} [\[see previous entry\]](#bubeck-sellke-2021-section){.include .include-even-when-collapsed} [\[see previous entry\]](#bubeck-et-al-2021-section){.include .include-even-when-collapsed} [\[see previous entry\]](#li-et-al-2022-17-section){.include .include-even-when-collapsed} [\[see previous entry\]](#novak-et-al-2018-section){.include .include-even-when-collapsed} [\[see previous entry\]](#zhang-et-al-2016-2-section){.include .include-even-when-collapsed} [\[see previous entry\]](#valle-pérez-et-al-2018-section){.include .include-even-when-collapsed} [\[see previous entry\]](#wilson-2025-section){.include .include-even-when-collapsed} [\[see previous entry\]](#martin-mahoney-2017-section){.include .include-even-when-collapsed} [\[see previous entry\]](#reizinger-et-al-2024-section){.include .include-even-when-collapsed} [\[see previous entry\]](#nakkiran-et-al-2020-section){.include .include-even-when-collapsed} [\[see previous entry\]](#vyas-et-al-2022-1-section){.include .include-even-when-collapsed} [\[see previous entry\]](#spigler-et-al-2019-section){.include .include-even-when-collapsed} [\[see previous entry\]](#viering-loog-2021-section){.include .include-even-when-collapsed} [\[see previous entry\]](#haussler-et-al-1996-section){.include .include-even-when-collapsed} [\[see previous entry\]](#amari-murata-1993-section){.include .include-even-when-collapsed} [\[see previous entry\]](#schwartz-et-al-1990-section){.include .include-even-when-collapsed} [\[see previous entry\]](#chatterjee-2018-section){.include .include-even-when-collapsed} [\[see previous entry\]](#feldman-2019-section){.include .include-even-when-collapsed} [\[see previous entry\]](#tänzer-et-al-2021-section){.include .include-even-when-collapsed} [\[see previous entry\]](#carlini-et-al-2022-section){.include .include-even-when-collapsed} [\[see previous entry\]](#biderman-et-al-2023-1-section){.include .include-even-when-collapsed} [\[see previous entry\]](#tirumala-et-al-2022-section){.include .include-even-when-collapsed} [\[see previous entry\]](#chang-et-al-2024-2-section){.include .include-even-when-collapsed} [\[see previous entry\]](#kandpal-et-al-2022-section){.include .include-even-when-collapsed} [\[see previous entry\]](#li-et-al-2023-1-section){.include .include-even-when-collapsed} [\[see previous entry\]](#wei-et-al-2024-1-section){.include .include-even-when-collapsed} [\[see previous entry\]](#vu-et-al-2023-section){.include .include-even-when-collapsed} [\[see previous entry\]](#arora-et-al-2022-2-section){.include .include-even-when-collapsed} [\[see previous entry\]](#zhou-et-al-2022-1-section){.include .include-even-when-collapsed} [\[see previous entry\]](#xu-et-al-2022-7-section){.include .include-even-when-collapsed} [\[see previous entry\]](#zhong-et-al-2021-2-section){.include .include-even-when-collapsed} [\[see previous entry\]](#lester-et-al-2021-section){.include .include-even-when-collapsed} [\[see previous entry\]](#qin-joty-2021-section){.include .include-even-when-collapsed} [\[see previous entry\]](#ye-et-al-2021-2-section){.include .include-even-when-collapsed} [\[see previous entry\]](#lin-et-al-2021-2-section){.include .include-even-when-collapsed} [\[see previous entry\]](#schäfer-et-al-2024-section){.include .include-even-when-collapsed} [\[see previous entry\]](#kim-2021-section){.include .include-even-when-collapsed} [\[see previous entry\]](#luo-et-al-2025-section){.include .include-even-when-collapsed} [\[see previous entry\]](#xiao-et-al-2024-section){.include .include-even-when-collapsed} [\[see previous entry\]](#liu-et-al-2025-section){.include .include-even-when-collapsed} [\[see previous entry\]](#droppo-elibol-2021-section){.include .include-even-when-collapsed} [\[see previous entry\]](#radford-et-al-2022-section){.include .include-even-when-collapsed} [\[see previous entry\]](#lazaridou-et-al-2021-section){.include .include-even-when-collapsed} [\[see previous entry\]](#lazaridou-et-al-2021-page-7-org-deepmind-section){.include .include-even-when-collapsed} [\[see previous entry\]](#rannen-triki-et-al-2024-section){.include .include-even-when-collapsed} [\[see previous entry\]](#mikolov-et-al-2010-section){.include .include-even-when-collapsed} [\[see previous entry\]](#collobert-et-al-2011-section){.include .include-even-when-collapsed} [\[see previous entry\]](#marcus-et-al-1993-section){.include .include-even-when-collapsed} [\[see previous entry\]](#banko-brill-2001-section){.include .include-even-when-collapsed} [\[see previous entry\]](#halevy-et-al-2009-section){.include .include-even-when-collapsed} [\[see previous entry\]](#zhang-et-al-2020-05-section){.include .include-even-when-collapsed} [\[see previous entry\]](#warstadt-et-al-2020-section){.include .include-even-when-collapsed} [\[see previous entry\]](#liu-et-al-2021-roberta-section){.include .include-even-when-collapsed} [\[see previous entry\]](#choudhury-et-al-2022-section){.include .include-even-when-collapsed} [\[see previous entry\]](#lanham-et-al-2023-section){.include .include-even-when-collapsed} [\[see previous entry\]](#anil-et-al-2022-section){.include .include-even-when-collapsed} [\[see previous entry\]](#nye-et-al-2021-section){.include .include-even-when-collapsed} [\[see previous entry\]](#ye-et-al-2024-2-section){.include .include-even-when-collapsed} [\[see previous entry\]](#yuan-et-al-2023-section){.include .include-even-when-collapsed} [\[see previous entry\]](#hendrycks-et-al-2021-4-section){.include .include-even-when-collapsed} [\[see previous entry\]](#yuan-et-al-2023-2-section){.include .include-even-when-collapsed} [\[see previous entry\]](#zhang-et-al-2024-11-section){.include .include-even-when-collapsed} [\[see previous entry\]](#chiu-et-al-2024-section){.include .include-even-when-collapsed} [\[see previous entry\]](#bommarito-et-al-2023-section){.include .include-even-when-collapsed} [\[see previous entry\]](#ii-katz-2022-section){.include .include-even-when-collapsed} [\[see previous entry\]](#nay-2023-section){.include .include-even-when-collapsed} [\[see previous entry\]](#kadavath-et-al-2022-section){.include .include-even-when-collapsed} [\[see previous entry\]](#lin-et-al-2022-09-section){.include .include-even-when-collapsed} [\[see previous entry\]](#minderer-et-al-2021-section){.include .include-even-when-collapsed} [\[see previous entry\]](#lakshminarayanan-et-al-2016-section){.include .include-even-when-collapsed} [\[see previous entry\]](#ashukha-et-al-2020-section){.include .include-even-when-collapsed} [\[see previous entry\]](#gontijo-lopes-et-al-2021-section){.include .include-even-when-collapsed} [\[see previous entry\]](#zhang-davison-2020-section){.include .include-even-when-collapsed} [\[see previous entry\]](#recht-et-al-2019-section){.include .include-even-when-collapsed} [\[see previous entry\]](#recht-et-al-2018-section){.include .include-even-when-collapsed} [\[see previous entry\]](#beyer-et-al-2020-section){.include .include-even-when-collapsed} [\[see previous entry\]](#vasudevan-et-al-2022-section){.include .include-even-when-collapsed} [\[see previous entry\]](#linn-2015-section){.include .include-even-when-collapsed} [\[see previous entry\]](#levy-2024-2-section){.include .include-even-when-collapsed} [\[see previous entry\]](#ramsauer-et-al-2020-section){.include .include-even-when-collapsed} [\[see previous entry\]](#schug-et-al-2024-section){.include .include-even-when-collapsed} [\[see previous entry\]](#poli-et-al-2024-section){.include .include-even-when-collapsed} [\[see previous entry\]](#cheng-et-al-2024-1-section){.include .include-even-when-collapsed} [\[see previous entry\]](#xiao-et-al-2021-3-section){.include .include-even-when-collapsed} [\[see previous entry\]](#rives-et-al-2020-section){.include .include-even-when-collapsed} [\[see previous entry\]](#lin-et-al-2022-02-section){.include .include-even-when-collapsed} [\[see previous entry\]](#nijkamp-et-al-2022-1-section){.include .include-even-when-collapsed} [\[see previous entry\]](#elnaggar-et-al-2020-section){.include .include-even-when-collapsed} [\[see previous entry\]](#trotter-et-al-2021-section){.include .include-even-when-collapsed} [\[see previous entry\]](#luo-et-al-2022-2-section){.include .include-even-when-collapsed} [\[see previous entry\]](#jeong-et-al-2024-section){.include .include-even-when-collapsed} [\[see previous entry\]](#ghesu-et-al-2022-section){.include .include-even-when-collapsed} [\[see previous entry\]](#cherti-jitsev-2021-section){.include .include-even-when-collapsed} [\[see previous entry\]](#kolesnikov-et-al-2019-section){.include .include-even-when-collapsed} [\[see previous entry\]](#bilen-vedaldi-2017-section){.include .include-even-when-collapsed} [\[see previous entry\]](#horn-perona-2017-section){.include .include-even-when-collapsed} [\[see previous entry\]](#gao-et-al-2017-section){.include .include-even-when-collapsed} [\[see previous entry\]](#kamath-et-al-2022-section){.include .include-even-when-collapsed} [\[see previous entry\]](#shao-et-al-2021-2-section){.include .include-even-when-collapsed} [\[see previous entry\]](#gabeur-et-al-2026-section){.include .include-even-when-collapsed} [\[see previous entry\]](#jaini-et-al-2023-section){.include .include-even-when-collapsed} [\[see previous entry\]](#khangaonkar-pirsiavash-2025-section){.include .include-even-when-collapsed} [\[see previous entry\]](#voynov-et-al-2020-section){.include .include-even-when-collapsed} [\[see previous entry\]](#kirillov-et-al-2023-section){.include .include-even-when-collapsed} [\[see previous entry\]](#lin-et-al-2021-m6-section){.include .include-even-when-collapsed} [\[see previous entry\]](#sun-et-al-2023-1-section){.include .include-even-when-collapsed} [\[see previous entry\]](#aghajanyan-et-al-2022-section){.include .include-even-when-collapsed} [\[see previous entry\]](#caciularu-et-al-2021-section){.include .include-even-when-collapsed} [\[see previous entry\]](#el-kishky-et-al-2019-section){.include .include-even-when-collapsed} [\[see previous entry\]](#schwenk-et-al-2019-section){.include .include-even-when-collapsed} [\[see previous entry\]](#heafield-et-al-2013-section){.include .include-even-when-collapsed} [\[see previous entry\]](#goodman-2001-section){.include .include-even-when-collapsed} [\[see previous entry\]](#shen-et-al-2017-2-section){.include .include-even-when-collapsed} [\[see previous entry\]](#saphra-et-al-2023-section){.include .include-even-when-collapsed} [\[see previous entry\]](#finnveden-2020-section){.include .include-even-when-collapsed} [\[see previous entry\]](#antonello-et-al-2023-section){.include .include-even-when-collapsed} [\[see previous entry\]](#schrimpf-et-al-2020-section){.include .include-even-when-collapsed} [\[see previous entry\]](#caucheteux-king-2022-section){.include .include-even-when-collapsed} [\[see previous entry\]](#caucheteux-king-2021-section){.include .include-even-when-collapsed} [\[see previous entry\]](#millet-et-al-2022-section){.include .include-even-when-collapsed} [\[see previous entry\]](#sato-et-al-2024-section){.include .include-even-when-collapsed} [\[see previous entry\]](#shillingford-et-al-2018-section){.include .include-even-when-collapsed} [\[see previous entry\]](#shi-et-al-2022-avhubert-section){.include .include-even-when-collapsed} [\[see previous entry\]](#shi-et-al-2022-5-section){.include .include-even-when-collapsed} [\[see previous entry\]](#hsu-et-al-2021-section){.include .include-even-when-collapsed} [\[see previous entry\]](#chan-et-al-2021-2-section){.include .include-even-when-collapsed} [\[see previous entry\]](#wang-et-al-2021-voxpopuli-section){.include .include-even-when-collapsed} [\[see previous entry\]](#langlais-et-al-2025-section){.include .include-even-when-collapsed} [\[see previous entry\]](#villalobos-et-al-2022-section){.include .include-even-when-collapsed} [\[see previous entry\]](#sachdeva-et-al-2024-2-section){.include .include-even-when-collapsed} [\[see previous entry\]](#xia-et-al-2023-1-section){.include .include-even-when-collapsed} [\[see previous entry\]](#ma-et-al-2024-3-section){.include .include-even-when-collapsed} [\[see previous entry\]](#dettmers-et-al-2022-section){.include .include-even-when-collapsed} [\[see previous entry\]](#liu-et-al-2021-swintranformerv2-section){.include .include-even-when-collapsed} [\[see previous entry\]](#lian-et-al-2021-1-section){.include .include-even-when-collapsed} [\[see previous entry\]](#khrylchenko-et-al-2025-section){.include .include-even-when-collapsed} [\[see previous entry\]](#zhai-et-al-2024-1-section){.include .include-even-when-collapsed} [\[see previous entry\]](#gupta-et-al-2021-2-section){.include .include-even-when-collapsed} [\[see previous entry\]](#naumov-et-al-2020-section){.include .include-even-when-collapsed} [\[see previous entry\]](#goyal-et-al-2017-section){.include .include-even-when-collapsed} [\[see previous entry\]](#mccandlish-et-al-2018-section){.include .include-even-when-collapsed} [\[see previous entry\]](#bottou-bousquet-2007-section){.include .include-even-when-collapsed} [\[see previous entry\]](#perlich-et-al-2003-section){.include .include-even-when-collapsed} [\[see previous entry\]](#grinsztajn-et-al-2022-section){.include .include-even-when-collapsed} [\[see previous entry\]](#urban-et-al-2016-section){.include .include-even-when-collapsed} [\[see previous entry\]](#mirzadeh-et-al-2021-section){.include .include-even-when-collapsed} [\[see previous entry\]](#fan-et-al-2020-4-section){.include .include-even-when-collapsed} [\[see previous entry\]](#montúfar-et-al-2014-section){.include .include-even-when-collapsed} [\[see previous entry\]](#olah-2014-section){.include .include-even-when-collapsed} [\[see previous entry\]](#elmoznino-bonner-2022-section){.include .include-even-when-collapsed} [\[see previous entry\]](#shen-et-al-2017-1-section){.include .include-even-when-collapsed} [\[see previous entry\]](#schulz-et-al-2022-section){.include .include-even-when-collapsed} [\[see previous entry\]](#yue-et-al-2024-section){.include .include-even-when-collapsed} [\[see previous entry\]](#borgeaud-et-al-2021-section){.include .include-even-when-collapsed} [\[see previous entry\]](#lee-et-al-2024-1-section){.include .include-even-when-collapsed} [\[see previous entry\]](#kim-et-al-2021-6-section){.include .include-even-when-collapsed} [\[see previous entry\]](#jae-eun-2021-section){.include .include-even-when-collapsed} [\[see previous entry\]](#inflection-2024-section){.include .include-even-when-collapsed} [\[see previous entry\]](#adiwardana-luong-2020-section){.include .include-even-when-collapsed} [\[see previous entry\]](#adiwardana-et-al-2020-section){.include .include-even-when-collapsed} [\[see previous entry\]](#blender-blog-section){.include .include-even-when-collapsed} [\[see previous entry\]](#touvron-et-al-2023-1-section){.include .include-even-when-collapsed} [\[see previous entry\]](#thoppilan-et-al-2022-section){.include .include-even-when-collapsed} [\[see previous entry\]](#perez-et-al-2022-1-section){.include .include-even-when-collapsed} [\[see previous entry\]](#wei-et-al-2023-2-section){.include .include-even-when-collapsed} [\[see previous entry\]](#solaiman-dennison-2021-section){.include .include-even-when-collapsed} [\[see previous entry\]](#maini-et-al-2025-section){.include .include-even-when-collapsed} [\[see previous entry\]](#ren-et-al-2024-2-section){.include .include-even-when-collapsed} [\[see previous entry\]](#berglund-et-al-2023-2-section){.include .include-even-when-collapsed} [\[see previous entry\]](#koralus-wang-maścianica-2023-section){.include .include-even-when-collapsed} [\[see previous entry\]](#eisape-et-al-2023-section){.include .include-even-when-collapsed} [\[see previous entry\]](#ajayi-et-al-2026-section){.include .include-even-when-collapsed} [\[see previous entry\]](#lee-et-al-2023-4-section){.include .include-even-when-collapsed} [\[see previous entry\]](#yin-et-al-2017-1-section){.include .include-even-when-collapsed} [\[see previous entry\]](#chizat-et-al-2018-section){.include .include-even-when-collapsed} [\[see previous entry\]](#neyshabur-et-al-2014-section){.include .include-even-when-collapsed} [\[see previous entry\]](#neyshabur-et-al-2017-section){.include .include-even-when-collapsed} [\[see previous entry\]](#hastie-et-al-2019-section){.include .include-even-when-collapsed} [\[see previous entry\]](#nakkiran-2019-section){.include .include-even-when-collapsed} [\[see previous entry\]](#brain-webb-1999-section){.include .include-even-when-collapsed} [\[see previous entry\]](#oates-jensen-1997-section){.include .include-even-when-collapsed} [\[see previous entry\]](#provost-kolluri-1999-section){.include .include-even-when-collapsed} [\[see previous entry\]](#kohavi-1996-section){.include .include-even-when-collapsed} [\[see previous entry\]](#ng-jordan-2001-section){.include .include-even-when-collapsed} [\[see previous entry\]](#cortes-et-al-1993-section){.include .include-even-when-collapsed} [\[see previous entry\]](#bornschein-et-al-2020-section){.include .include-even-when-collapsed} [\[see previous entry\]](#blier-ollivier-2018-section){.include .include-even-when-collapsed} [\[see previous entry\]](#huang-et-al-2024-section){.include .include-even-when-collapsed} [\[see previous entry\]](#gignac-ilić-2025-section){.include .include-even-when-collapsed} [\[see previous entry\]](#ilić-gignac-2023-section){.include .include-even-when-collapsed} [\[see previous entry\]](#huang-et-al-2023-7-section){.include .include-even-when-collapsed} [\[see previous entry\]](#wang-et-al-2023-12-section){.include .include-even-when-collapsed} [\[see previous entry\]](#aher-et-al-2022-section){.include .include-even-when-collapsed} [\[see previous entry\]](#gandhi-et-al-2023-2-section){.include .include-even-when-collapsed} [\[see previous entry\]](#thrush-et-al-2024-2-section){.include .include-even-when-collapsed} [\[see previous entry\]](#bahri-et-al-2020-section){.include .include-even-when-collapsed} [\[see previous entry\]](#gao-2021-section){.include .include-even-when-collapsed} [\[see previous entry\]](#spiesberger-et-al-2026-section){.include .include-even-when-collapsed} [\[see previous entry\]](#mayne-et-al-2026-section){.include .include-even-when-collapsed} [\[see previous entry\]](#lin-et-al-2021-6-section){.include .include-even-when-collapsed} [\[see previous entry\]](#gudibande-et-al-2023-section){.include .include-even-when-collapsed} [\[see previous entry\]](#zhou-et-al-2023-09-section){.include .include-even-when-collapsed} [\[see previous entry\]](#jaroslawicz-et-al-2025-section){.include .include-even-when-collapsed} [\[see previous entry\]](#aw-et-al-2023-section){.include .include-even-when-collapsed} [\[see previous entry\]](#aghajanyan-et-al-2020-section){.include .include-even-when-collapsed} [\[see previous entry\]](#chen-et-al-2025-section){.include .include-even-when-collapsed} [\[see previous entry\]](#amos-et-al-2023-section){.include .include-even-when-collapsed} [\[see previous entry\]](#mehta-et-al-2021-1-section){.include .include-even-when-collapsed} [\[see previous entry\]](#cossu-et-al-2022-section){.include .include-even-when-collapsed} [\[see previous entry\]](#ramasesh-et-al-2022-section){.include .include-even-when-collapsed} [\[see previous entry\]](#yıldız-et-al-2024-section){.include .include-even-when-collapsed} [\[see previous entry\]](#ostapenko-et-al-2022-section){.include .include-even-when-collapsed} [\[see previous entry\]](#wang-et-al-2022-17-section){.include .include-even-when-collapsed} [\[see previous entry\]](#johnson-et-al-2016-2-section){.include .include-even-when-collapsed} [\[see previous entry\]](#han-et-al-2021-2-section){.include .include-even-when-collapsed} [\[see previous entry\]](#tanzer-et-al-2023-section){.include .include-even-when-collapsed} [\[see previous entry\]](#chang-et-al-2023-1-section){.include .include-even-when-collapsed} [\[see previous entry\]](#openai-2023-2-section){.include .include-even-when-collapsed} [\[see previous entry\]](#polu-sutskever-2020-section){.include .include-even-when-collapsed} [\[see previous entry\]](#drori-verma-2021-section){.include .include-even-when-collapsed} [\[see previous entry\]](#tang-et-al-2021-1-section){.include .include-even-when-collapsed} [\[see previous entry\]](#nijkamp-et-al-2022-2-section){.include .include-even-when-collapsed} [\[see previous entry\]](#dai-et-al-2022-2-section){.include .include-even-when-collapsed} [\[see previous entry\]](#zhou-et-al-2021-2-section){.include .include-even-when-collapsed} [\[see previous entry\]](#bi-et-al-2020-section){.include .include-even-when-collapsed} [\[see previous entry\]](#xu-et-al-2022-2-section){.include .include-even-when-collapsed} [\[see previous entry\]](#yang-et-al-2022-8-section){.include .include-even-when-collapsed} [\[see previous entry\]](#wang-et-al-2022-10-section){.include .include-even-when-collapsed} [\[see previous entry\]](#liu-et-al-2019-roberta-section){.include .include-even-when-collapsed} [\[see previous entry\]](#warner-et-al-2024-section){.include .include-even-when-collapsed} [\[see previous entry\]](#gu-dao-2023-section){.include .include-even-when-collapsed} [\[see previous entry\]](#beck-et-al-2024-section){.include .include-even-when-collapsed} [\[see previous entry\]](#greff-et-al-2015-section){.include .include-even-when-collapsed} [\[see previous entry\]](#morcos-tian-2019-section){.include .include-even-when-collapsed} [\[see previous entry\]](#ramanujan-et-al-2019-section){.include .include-even-when-collapsed} [\[see previous entry\]](#orseau-et-al-2020-section){.include .include-even-when-collapsed} [\[see previous entry\]](#gao-et-al-2025-section){.include .include-even-when-collapsed} [\[see previous entry\]](#gurnee-et-al-2023-section){.include .include-even-when-collapsed} [\[see previous entry\]](#zhao-et-al-2024-2-section){.include .include-even-when-collapsed} [\[see previous entry\]](#lampinen-et-al-2022-section){.include .include-even-when-collapsed} [\[see previous entry\]](#marasović-et-al-2021-section){.include .include-even-when-collapsed} [\[see previous entry\]](#prabhumoye-et-al-2021-section){.include .include-even-when-collapsed} [\[see previous entry\]](#sheng-et-al-2021-section){.include .include-even-when-collapsed} [\[see previous entry\]](#tömekçe-et-al-2024-section){.include .include-even-when-collapsed} [\[see previous entry\]](#price-et-al-2024-2-section){.include .include-even-when-collapsed} [\[see previous entry\]](#papadopoulos-et-al-2024-section){.include .include-even-when-collapsed} [\[see previous entry\]](#alabdulmohsin-et-al-2024-section){.include .include-even-when-collapsed} [\[see previous entry\]](#rust-et-al-2022-section){.include .include-even-when-collapsed} [\[see previous entry\]](#eldan-li-2023-section){.include .include-even-when-collapsed} [\[see previous entry\]](#yuying-2021-section){.include .include-even-when-collapsed} [\[see previous entry\]](#smith-et-al-2024-section){.include .include-even-when-collapsed} [\[see previous entry\]](#pan-et-al-2024-1-section){.include .include-even-when-collapsed} [\[see previous entry\]](#nguyen-et-al-2023-2-section){.include .include-even-when-collapsed} [\[see previous entry\]](#nguyen-et-al-2023-1-section){.include .include-even-when-collapsed} [\[see previous entry\]](#vonich-hakim-2025-section){.include .include-even-when-collapsed} [\[see previous entry\]](#bodnar-et-al-2024-section){.include .include-even-when-collapsed} [\[see previous entry\]](#ho-et-al-2025-section){.include .include-even-when-collapsed} [\[see previous entry\]](#gundlach-et-al-2025-section){.include .include-even-when-collapsed} [\[see previous entry\]](#hernandez-brown-2020-paper-section){.include .include-even-when-collapsed} [\[see previous entry\]](#koehler-et-al-2020-section){.include .include-even-when-collapsed} [\[see previous entry\]](#merity-2018-section){.include .include-even-when-collapsed} [\[see previous entry\]](#sutton-2019-2-section){.include .include-even-when-collapsed} [\[see previous entry\]](#marcus-2020-section){.include .include-even-when-collapsed} [\[see previous entry\]](#alexander-2019-2-section){.include .include-even-when-collapsed} [\[see previous entry\]](#godwin-2023-section){.include .include-even-when-collapsed} [\[see previous entry\]](#hao-2020-section){.include .include-even-when-collapsed} [\[see previous entry\]](#murgia-2023-1-section){.include .include-even-when-collapsed} [\[see previous entry\]](#tantau-2018-section){.include .include-even-when-collapsed} [\[see previous entry\]](#heaven-2023-section){.include .include-even-when-collapsed} [\[see previous entry\]](#che-2025-section){.include .include-even-when-collapsed} [\[see previous entry\]](#caithrin-2026-section){.include .include-even-when-collapsed} [\[see previous entry\]](#yudkowsky-2017-section){.include .include-even-when-collapsed} [\[see previous entry\]](#clarke-et-al-2022-section){.include .include-even-when-collapsed} [\[see previous entry\]](#gruetzemacher-et-al-2024-section){.include .include-even-when-collapsed} [\[see previous entry\]](#phillips-2023-section){.include .include-even-when-collapsed} [\[see previous entry\]](#leike-sutskever-2023-section){.include .include-even-when-collapsed} [\[see previous entry\]](#_Rx5I2W4d-section){.include .include-even-when-collapsed} ## Wikipedia (14) {#titled-links-wikipedia .collapse} 1. **[`Algorithmic information theory`{=html}](https://en.wikipedia.org/wiki/Algorithmic_information_theory "Algorithmic information theory"){.link-annotated-partial .id-not .include-annotation link-icon="wikipedia" link-icon-type="svg"}** 2. **[`Curse of dimensionality § Blessing of dimensionality`{=html}](https://en.wikipedia.org/wiki/Curse_of_dimensionality#Blessing_of_dimensionality "Curse of dimensionality § Blessing of dimensionality"){.id-not .include-annotation link-icon="wikipedia" link-icon-type="svg"}** ::: aux-links-transclude-file [`https://en.wikipedia.org/wiki/Curse_of_dimensionality#Blessing_of_dimensionality`](https://en.wikipedia.org/wiki/Curse_of_dimensionality#Blessing_of_dimensionality "Curse of dimensionality § Blessing of dimensionality"){.id-not .include-content include-template="$annotationFileIncludeTemplate"} ::: 3. **[`Dwarkesh Patel`{=html}](https://en.wikipedia.org/wiki/Dwarkesh_Patel "Dwarkesh Patel"){.id-not .include-annotation link-icon="wikipedia" link-icon-type="svg"}** ::: aux-links-transclude-file [`https://en.wikipedia.org/wiki/Dwarkesh_Patel`](https://en.wikipedia.org/wiki/Dwarkesh_Patel "Dwarkesh Patel"){.id-not .include-content include-template="$annotationFileIncludeTemplate"} ::: 4. **[`Fei-Fei Li`{=html}](https://en.wikipedia.org/wiki/Fei-Fei_Li "Fei-Fei Li"){.id-not .include-annotation link-icon="wikipedia" link-icon-type="svg"}** ::: aux-links-transclude-file [`https://en.wikipedia.org/wiki/Fei-Fei_Li`](https://en.wikipedia.org/wiki/Fei-Fei_Li "Fei-Fei Li"){.id-not .include-content include-template="$annotationFileIncludeTemplate"} ::: 5. **[`Hans Moravec`{=html}](https://en.wikipedia.org/wiki/Hans_Moravec "Hans Moravec"){.link-annotated-partial .id-not .include-annotation link-icon="wikipedia" link-icon-type="svg"}** 6. **[`Jürgen Schmidhuber`{=html}](https://en.wikipedia.org/wiki/J%C3%BCrgen_Schmidhuber "Jürgen Schmidhuber"){.link-annotated-partial .id-not .include-annotation link-icon="wikipedia" link-icon-type="svg"}** 7. **[`Jakub Pachocki`{=html}](https://en.wikipedia.org/wiki/Jakub_Pachocki "Jakub Pachocki"){.id-not .include-annotation link-icon="wikipedia" link-icon-type="svg"}** ::: aux-links-transclude-file [`https://en.wikipedia.org/wiki/Jakub_Pachocki`](https://en.wikipedia.org/wiki/Jakub_Pachocki "Jakub Pachocki"){.id-not .include-content include-template="$annotationFileIncludeTemplate"} ::: 8. **[`Leon Bottou`{=html}](https://en.wikipedia.org/wiki/Leon_Bottou "Leon Bottou"){.id-not .include-annotation link-icon="wikipedia" link-icon-type="svg"}** ::: aux-links-transclude-file [`https://en.wikipedia.org/wiki/Leon_Bottou`](https://en.wikipedia.org/wiki/Leon_Bottou "Leon Bottou"){.id-not .include-content include-template="$annotationFileIncludeTemplate"} ::: 9. **[`Moonshot AI`{=html}](https://en.wikipedia.org/wiki/Moonshot_AI "Moonshot AI"){.id-not .include-annotation link-icon="wikipedia" link-icon-type="svg"}** ::: aux-links-transclude-file [`https://en.wikipedia.org/wiki/Moonshot_AI`](https://en.wikipedia.org/wiki/Moonshot_AI "Moonshot AI"){.id-not .include-content include-template="$annotationFileIncludeTemplate"} ::: 10. **[`Neural scaling law`{=html}](https://en.wikipedia.org/wiki/Neural_scaling_law "Neural scaling law"){.id-not .include-annotation link-icon="wikipedia" link-icon-type="svg"}** ::: aux-links-transclude-file [`https://en.wikipedia.org/wiki/Neural_scaling_law`](https://en.wikipedia.org/wiki/Neural_scaling_law "Neural scaling law"){.id-not .include-content include-template="$annotationFileIncludeTemplate"} ::: 11. **[`Overcompleteness`{=html}](https://en.wikipedia.org/wiki/Overcompleteness "Overcompleteness"){.id-not .include-annotation link-icon="wikipedia" link-icon-type="svg"}** ::: aux-links-transclude-file [`https://en.wikipedia.org/wiki/Overcompleteness`](https://en.wikipedia.org/wiki/Overcompleteness "Overcompleteness"){.id-not .include-content include-template="$annotationFileIncludeTemplate"} ::: 12. **[`Power law`{=html}](https://en.wikipedia.org/wiki/Power_law "Power law"){.link-annotated-partial .id-not .include-annotation link-icon="wikipedia" link-icon-type="svg"}** 13. **[`Scale invariance`{=html}](https://en.wikipedia.org/wiki/Scale_invariance "Scale invariance"){.id-not .include-annotation link-icon="wikipedia" link-icon-type="svg"}** ::: aux-links-transclude-file [`https://en.wikipedia.org/wiki/Scale_invariance`](https://en.wikipedia.org/wiki/Scale_invariance "Scale invariance"){.id-not .include-content include-template="$annotationFileIncludeTemplate"} ::: 14. **[`Stockfish (chess) § Fishtest`{=html}](https://en.wikipedia.org/wiki/Stockfish_(chess)#Fishtest "Stockfish (chess) § Fishtest"){.id-not .include-annotation link-icon="wikipedia" link-icon-type="svg"}** ::: aux-links-transclude-file [`https://en.wikipedia.org/wiki/Stockfish_(chess)#Fishtest`](https://en.wikipedia.org/wiki/Stockfish_(chess)#Fishtest "Stockfish (chess) § Fishtest"){.id-not .include-content include-template="$annotationFileIncludeTemplate"} ::: # Miscellaneous - **[`https://www.nytimes.com/2026/04/26/business/dwarkesh-patel-podcast-ai.html`{=html}](https://www.nytimes.com/2026/04/26/business/dwarkesh-patel-podcast-ai.html){.link-annotated-partial .id-not .include-annotation link-icon="new-york-times" link-icon-type="svg"}** - **[`https://www.nytimes.com/2026/03/12/technology/meta-avocado-ai-model-delayed.html`{=html}](https://www.nytimes.com/2026/03/12/technology/meta-avocado-ai-model-delayed.html){.id-not .include-annotation link-icon="new-york-times" link-icon-type="svg"}** - **[`https://x.com/RuiHuang_art/status/1954716703340048877`{=html}](https://x.com/RuiHuang_art/status/1954716703340048877 "RuiHuang_art 2025"){.link-annotated-partial .id-not .include-annotation link-icon="twitter" link-icon-type="svg" link-icon-color="#1da1f2"}** - **[`https://x.com/DimitrisPapail/status/1927553481612677188`{=html}](https://x.com/DimitrisPapail/status/1927553481612677188 "Papail 2025"){.link-annotated-partial .id-not .include-annotation link-icon="twitter" link-icon-type="svg" link-icon-color="#1da1f2"}** - **[`https://x.com/jxmnop/status/1927385194601886065`{=html}](https://x.com/jxmnop/status/1927385194601886065 "jxmnop 2025"){.link-annotated-partial .id-not .include-annotation link-icon="twitter" link-icon-type="svg" link-icon-color="#1da1f2"}** - **[`/doc/ai/scaling/2024-01-01-gwern-reddit-rmachinelearning-screenshotshowingscalingcentricdiscussions.png`{=html}](/doc/ai/scaling/2024-01-01-gwern-reddit-rmachinelearning-screenshotshowingscalingcentricdiscussions.png "Gwern 2024"){.link-annotated-partial .id-not .include-annotation link-icon="image" link-icon-type="svg"}** - **[`/doc/ai/scaling/2024-lin-figure2-inversescalingontruthfulqa.jpg`{=html}](/doc/ai/scaling/2024-lin-figure2-inversescalingontruthfulqa.jpg){.link-annotated-partial .id-not .include-annotation link-icon="image" link-icon-type="svg"}** - **[`/doc/ai/scaling/2024-smith-figure2-validationlossesofgalaxyimagepredictiontransformershowingscalingcurves.png`{=html}](/doc/ai/scaling/2024-smith-figure2-validationlossesofgalaxyimagepredictiontransformershowingscalingcurves.png){.link-annotated-partial .id-not .include-annotation link-icon="image" link-icon-type="svg"}** - **[`/doc/ai/scaling/2024-smith-figure4-downstreamperformanceinastronomytasksfromgalaxypretrainedgpt2.png`{=html}](/doc/ai/scaling/2024-smith-figure4-downstreamperformanceinastronomytasksfromgalaxypretrainedgpt2.png){.link-annotated-partial .id-not .include-annotation link-icon="image" link-icon-type="svg"}** - **[`/doc/ai/scaling/2024-wang-figure1-writebenchcreativewritingscalingwithmodelsizeshowingweaveroutlier.jpg`{=html}](/doc/ai/scaling/2024-wang-figure1-writebenchcreativewritingscalingwithmodelsizeshowingweaveroutlier.jpg){.link-annotated-partial .id-not .include-annotation link-icon="image" link-icon-type="svg"}** - **[`/doc/ai/scaling/2023-eldan-figure23-scalinglawoftinystoriesgpttransformermodelswithtrainingflops.jpg`{=html}](/doc/ai/scaling/2023-eldan-figure23-scalinglawoftinystoriesgpttransformermodelswithtrainingflops.jpg){.link-annotated-partial .id-not .include-annotation link-icon="image" link-icon-type="svg"}** - **[`/doc/ai/scaling/2023-manvi-figure4-llmvstabularmachinelearningscalingofpredictionperformanceinsamplesize.png`{=html}](/doc/ai/scaling/2023-manvi-figure4-llmvstabularmachinelearningscalingofpredictionperformanceinsamplesize.png){.link-annotated-partial .id-not .include-annotation link-icon="image" link-icon-type="svg"}** - **[`/doc/ai/scaling/2023-nguyen-figure6-stormerweatherforecastingscalesinmodelsizeanddatagranularity.png`{=html}](/doc/ai/scaling/2023-nguyen-figure6-stormerweatherforecastingscalesinmodelsizeanddatagranularity.png){.link-annotated-partial .id-not .include-annotation link-icon="image" link-icon-type="svg"}** - **[`/doc/ai/scaling/2023-vu-figure2-largermorepowerfulllmsperformbetteronfastchangingquestionsorfalsepremisesinfreshqa.jpg`{=html}](/doc/ai/scaling/2023-vu-figure2-largermorepowerfulllmsperformbetteronfastchangingquestionsorfalsepremisesinfreshqa.jpg){.link-annotated-partial .id-not .include-annotation link-icon="image" link-icon-type="svg"}** - **[`/doc/ai/scaling/2023-wang-figure9-videodatascalingoftft2vvideogeneration.png`{=html}](/doc/ai/scaling/2023-wang-figure9-videodatascalingoftft2vvideogeneration.png){.link-annotated-partial .id-not .include-annotation link-icon="image" link-icon-type="svg"}** - **[`/doc/ai/nn/fully-connected/2023-bachmann-figure1-mlpcomputescalingoncifar100.jpg`{=html}](/doc/ai/nn/fully-connected/2023-bachmann-figure1-mlpcomputescalingoncifar100.jpg){.link-annotated-partial .id-not .include-annotation link-icon="image" link-icon-type="svg"}** - **[`/doc/ai/nn/fully-connected/2023-bachmann-figure4-mlpsscalewellwithincreasingbatchsize.jpg`{=html}](/doc/ai/nn/fully-connected/2023-bachmann-figure4-mlpsscalewellwithincreasingbatchsize.jpg){.link-annotated-partial .id-not .include-annotation link-icon="image" link-icon-type="svg"}** - **[`/doc/ai/nn/fully-connected/2023-bachmann-figure5-scalingofmlpsoncifar10andimagenet1k.png`{=html}](/doc/ai/nn/fully-connected/2023-bachmann-figure5-scalingofmlpsoncifar10andimagenet1k.png){.link-annotated-partial .id-not .include-annotation link-icon="image" link-icon-type="svg"}** - **[`/doc/ai/nn/fully-connected/2023-bachmann-figure6-powerlawincifar100losswhenconstrainingparametersordatasetsize.jpg`{=html}](/doc/ai/nn/fully-connected/2023-bachmann-figure6-powerlawincifar100losswhenconstrainingparametersordatasetsize.jpg){.link-annotated-partial .id-not .include-annotation link-icon="image" link-icon-type="svg"}** - **[`/doc/ai/nn/fully-connected/2023-bachmann-figure7-suprachinchilladatascalingformlpsoncifar100loss.jpg`{=html}](/doc/ai/nn/fully-connected/2023-bachmann-figure7-suprachinchilladatascalingformlpsoncifar100loss.jpg){.link-annotated-partial .id-not .include-annotation link-icon="image" link-icon-type="svg"}** - **[`/doc/ai/nn/transformer/clip/2023-girdhar-figure6-imagebindscalingofperformancewithincreasingclipimageencodersize.png`{=html}](/doc/ai/nn/transformer/clip/2023-girdhar-figure6-imagebindscalingofperformancewithincreasingclipimageencodersize.png){.link-annotated-partial .id-not .include-annotation link-icon="image" link-icon-type="svg"}** - **[`/doc/ai/scaling/2022-10-06-robert-lesswrongmoreaudiblepodcast-itlookslikeyouretryingtotakeovertheworld.mp3`{=html}](/doc/ai/scaling/2022-10-06-robert-lesswrongmoreaudiblepodcast-itlookslikeyouretryingtotakeovertheworld.mp3){.link-annotated-partial .id-not .include-annotation link-icon="audio" link-icon-type="svg"}** - **[`/doc/ai/scaling/2022-maloney-figure11-equiparameterizationhypothesisshows1to1parameterdatascalingratioisoptimal.jpg`{=html}](/doc/ai/scaling/2022-maloney-figure11-equiparameterizationhypothesisshows1to1parameterdatascalingratioisoptimal.jpg){.link-annotated-partial .id-not .include-annotation link-icon="image" link-icon-type="svg"}** - **[`/doc/ai/scaling/2022-press-figure1-scalingofgpt3modelperformanceoncompositionalcelebritiesdatasetshowingincreasingperformanceofbothsingleand2stepquestions.png`{=html}](/doc/ai/scaling/2022-press-figure1-scalingofgpt3modelperformanceoncompositionalcelebritiesdatasetshowingincreasingperformanceofbothsingleand2stepquestions.png){.link-annotated-partial .id-not .include-annotation link-icon="image" link-icon-type="svg"}** - **[`/doc/ai/scaling/2022-zhu-figure9-webface260mcnnfacerecognitionscalingbyn.png`{=html}](/doc/ai/scaling/2022-zhu-figure9-webface260mcnnfacerecognitionscalingbyn.png){.link-annotated-partial .id-not .include-annotation link-icon="image" link-icon-type="svg"}** - **[`/doc/ai/nn/transformer/gpt/whisper/2022-radford-figure4-correlationofpretraininglanguagedatawithtranslationperformance.jpg`{=html}](/doc/ai/nn/transformer/gpt/whisper/2022-radford-figure4-correlationofpretraininglanguagedatawithtranslationperformance.jpg){.link-annotated-partial .id-not .include-annotation link-icon="image" link-icon-type="svg"}** - **[`/doc/ai/nn/transformer/gpt/whisper/2022-radford-figure8-whisperscalingbymodelsize.png`{=html}](/doc/ai/nn/transformer/gpt/whisper/2022-radford-figure8-whisperscalingbymodelsize.png){.link-annotated-partial .id-not .include-annotation link-icon="image" link-icon-type="svg"}** - **[`/doc/ai/nn/transformer/gpt/whisper/2022-radford-figure9-crossoverinmonolingualvsmultilingualtrainingscalingshowseventualtransfer.jpg`{=html}](/doc/ai/nn/transformer/gpt/whisper/2022-radford-figure9-crossoverinmonolingualvsmultilingualtrainingscalingshowseventualtransfer.jpg){.link-annotated-partial .id-not .include-annotation link-icon="image" link-icon-type="svg"}** - **[`/doc/ai/scaling/2021-10-11-xinzhiyuan-inspursource10gpt245b.html`{=html}](/doc/ai/scaling/2021-10-11-xinzhiyuan-inspursource10gpt245b.html){.link-annotated-partial .id-not .include-annotation link-icon="internet-archive" link-icon-type="svg"}** - **[`/doc/ai/scaling/2021-goyal-figure1-seerscalinginparameters.png`{=html}](/doc/ai/scaling/2021-goyal-figure1-seerscalinginparameters.png){.link-annotated-partial .id-not .include-annotation link-icon="image" link-icon-type="svg"}** - **[`/doc/ai/scaling/2021-goyal-figure6-seerscalingindatan.jpg`{=html}](/doc/ai/scaling/2021-goyal-figure6-seerscalingindatan.jpg){.link-annotated-partial .id-not .include-annotation link-icon="image" link-icon-type="svg"}** - **[`/doc/ai/scaling/2021-hernandez-transferlearning-figure1-transfervsfinetuning.png`{=html}](/doc/ai/scaling/2021-hernandez-transferlearning-figure1-transfervsfinetuning.png){.link-annotated-partial .id-not .include-annotation link-icon="image" link-icon-type="svg"}** - **[`/doc/ai/scaling/2021-hu-figure1-lemontransformerscalingonmscocoimagecaptioning.png`{=html}](/doc/ai/scaling/2021-hu-figure1-lemontransformerscalingonmscocoimagecaptioning.png){.link-annotated-partial .id-not .include-annotation link-icon="image" link-icon-type="svg"}** - **[`/doc/ai/scaling/2021-hu-figure2-a-datascalingfinetuningperformanceonmscoco.jpg`{=html}](/doc/ai/scaling/2021-hu-figure2-a-datascalingfinetuningperformanceonmscoco.jpg){.link-annotated-partial .id-not .include-annotation link-icon="image" link-icon-type="svg"}** - **[`/doc/ai/scaling/2021-lazaridou-figure3-incorrectverysmallscalescalingoftransformerxlmodelsdoesnotleadtolargeperformancegainsontemporaldriftbenchmark.png`{=html}](/doc/ai/scaling/2021-lazaridou-figure3-incorrectverysmallscalescalingoftransformerxlmodelsdoesnotleadtolargeperformancegainsontemporaldriftbenchmark.png){.link-annotated-partial .id-not .include-annotation link-icon="image" link-icon-type="svg"}** - **[`/doc/ai/scaling/2021-zhang-figure1a-conformermodelworderrorscalingindatasetsize.jpg`{=html}](/doc/ai/scaling/2021-zhang-figure1a-conformermodelworderrorscalingindatasetsize.jpg){.link-annotated-partial .id-not .include-annotation link-icon="image" link-icon-type="svg"}** - **[`/doc/ai/scaling/2021-zhang-figure2-conformerpmodelworderrorscalingratesindatasetsize.png`{=html}](/doc/ai/scaling/2021-zhang-figure2-conformerpmodelworderrorscalingratesindatasetsize.png){.link-annotated-partial .id-not .include-annotation link-icon="image" link-icon-type="svg"}** - **[`/doc/reinforcement-learning/model/muzero/2021-schrittwieser-figure1-mspacmanmuzerologrewardscaling.jpg`{=html}](/doc/reinforcement-learning/model/muzero/2021-schrittwieser-figure1-mspacmanmuzerologrewardscaling.jpg){.link-annotated-partial .id-not .include-annotation link-icon="image" link-icon-type="svg"}** - **[`/doc/ai/scaling/2020-carlsmith-figure5-flopsbudgetestimates.png`{=html}](/doc/ai/scaling/2020-carlsmith-figure5-flopsbudgetestimates.png){.link-annotated-partial .id-not .include-annotation link-icon="image" link-icon-type="svg"}** - **[`/doc/ai/scaling/2020-chrisdyer-aacl2020-machinetranslationscaling-ngramsvsrnns.jpg`{=html}](/doc/ai/scaling/2020-chrisdyer-aacl2020-machinetranslationscaling-ngramsvsrnns.jpg){.link-annotated-partial .id-not .include-annotation link-icon="image" link-icon-type="svg"}** - **[`/doc/ai/scaling/2020-finnveden-extrapolationwcomparisons.png`{=html}](/doc/ai/scaling/2020-finnveden-extrapolationwcomparisons.png){.link-annotated-partial .id-not .include-annotation link-icon="image" link-icon-type="svg"}** - **[`/doc/ai/scaling/2020-finnveden-normalizedlosscurves.jpg`{=html}](/doc/ai/scaling/2020-finnveden-normalizedlosscurves.jpg){.link-annotated-partial .id-not .include-annotation link-icon="image" link-icon-type="svg"}** - **[`/doc/ai/scaling/2020-rosset-turingnlg-nlpmodelparametercountovertime.png`{=html}](/doc/ai/scaling/2020-rosset-turingnlg-nlpmodelparametercountovertime.png){.link-annotated-partial .id-not .include-annotation link-icon="image" link-icon-type="svg"}** - **[`/doc/ai/scaling/2019-liu-table4-robertabenefitsfromscalingdatasets10xoverbert.png`{=html}](/doc/ai/scaling/2019-liu-table4-robertabenefitsfromscalingdatasets10xoverbert.png){.link-annotated-partial .id-not .include-annotation link-icon="image" link-icon-type="svg"}** - **[`/doc/ai/scaling/2018-howard-figure3-datascalingofrnnpretrainingfortextclassification.jpg`{=html}](/doc/ai/scaling/2018-howard-figure3-datascalingofrnnpretrainingfortextclassification.jpg){.link-annotated-partial .id-not .include-annotation link-icon="image" link-icon-type="svg"}** - **[`/doc/ai/scaling/2017-koehn-figure3-bleuscoreswithvaryingamountsoftrainingdata.png`{=html}](/doc/ai/scaling/2017-koehn-figure3-bleuscoreswithvaryingamountsoftrainingdata.png){.link-annotated-partial .id-not .include-annotation link-icon="image" link-icon-type="svg"}** - **[`/doc/ai/scaling/2015-krause-figure11-cub2002011imageclassificationlogarithmcscalinginnoisywebimagedatasetsize.png`{=html}](/doc/ai/scaling/2015-krause-figure11-cub2002011imageclassificationlogarithmcscalinginnoisywebimagedatasetsize.png){.link-annotated-partial .id-not .include-annotation link-icon="image" link-icon-type="svg"}** - **[`/doc/ai/scaling/2015-krause-table1-effectivenessofscalingupcnnsonlargenoisywebdatasetsvscompetitors.png`{=html}](/doc/ai/scaling/2015-krause-table1-effectivenessofscalingupcnnsonlargenoisywebdatasetsvscompetitors.png){.link-annotated-partial .id-not .include-annotation link-icon="image" link-icon-type="svg"}** - **[`/doc/ai/nn/cnn/2015-joulin-figure2-flickrpascalvoc2007precisionscalingwithflickr100mnscaling.jpg`{=html}](/doc/ai/nn/cnn/2015-joulin-figure2-flickrpascalvoc2007precisionscalingwithflickr100mnscaling.jpg){.link-annotated-partial .id-not .include-annotation link-icon="image" link-icon-type="svg"}** - **[`/doc/ai/scaling/2014-cambria-figure1-hypotheticalnlpprogresscurves.png`{=html}](/doc/ai/scaling/2014-cambria-figure1-hypotheticalnlpprogresscurves.png){.link-annotated-partial .id-not .include-annotation link-icon="image" link-icon-type="svg"}** - **[`/doc/ai/scaling/2012-bottou-figure13-1-sgdtrainingtimetestlossvstron.png`{=html}](/doc/ai/scaling/2012-bottou-figure13-1-sgdtrainingtimetestlossvstron.png){.link-annotated-partial .id-not .include-annotation link-icon="image" link-icon-type="svg"}** - **[`/doc/ai/scaling/2012-bottou-figure13-2-sgdtrainingtimetestlossvsconjugategradients.png`{=html}](/doc/ai/scaling/2012-bottou-figure13-2-sgdtrainingtimetestlossvsconjugategradients.png){.link-annotated-partial .id-not .include-annotation link-icon="image" link-icon-type="svg"}** - **[`/doc/ai/scaling/2012-chelba-figure2-5gramscalingindatasize.png`{=html}](/doc/ai/scaling/2012-chelba-figure2-5gramscalingindatasize.png){.link-annotated-partial .id-not .include-annotation link-icon="image" link-icon-type="svg"}** - **[`/doc/ai/scaling/2011-torralba-table3-positivetransfervalueofimageclassificationdatasetsacrosstasksforsvmhogs.png`{=html}](/doc/ai/scaling/2011-torralba-table3-positivetransfervalueofimageclassificationdatasetsacrosstasksforsvmhogs.png){.link-annotated-partial .id-not .include-annotation link-icon="image" link-icon-type="svg"}** - **[`/doc/ai/scaling/2009-12-07-shanelegg-supercomputerlinpackoverpast50years.png`{=html}](/doc/ai/scaling/2009-12-07-shanelegg-supercomputerlinpackoverpast50years.png){.link-annotated-partial .id-not .include-annotation link-icon="image" link-icon-type="svg"}** - **[`/doc/ai/scaling/2009-koren-figure4-netflixprizecontentmodelaccuracyscaleswithparametercount.png`{=html}](/doc/ai/scaling/2009-koren-figure4-netflixprizecontentmodelaccuracyscaleswithparametercount.png){.link-annotated-partial .id-not .include-annotation link-icon="image" link-icon-type="svg"}** - **[`/doc/ai/scaling/2001-banko-figure1-scalingcurve.png`{=html}](/doc/ai/scaling/2001-banko-figure1-scalingcurve.png){.link-annotated-partial .id-not .include-annotation link-icon="image" link-icon-type="svg"}** - **[`http://www.incompleteideas.net/Talks/UBC-2016.pdf`{=html}](http://www.incompleteideas.net/Talks/UBC-2016.pdf){.link-annotated-partial .id-not .include-annotation link-icon="pdf" link-icon-type="svg" link-icon-color="#f40f02"}** - **[`https://ai.meta.com/blog/harmful-content-can-evolve-quickly-our-new-ai-system-adapts-to-tackle-it`{=html}](https://ai.meta.com/blog/harmful-content-can-evolve-quickly-our-new-ai-system-adapts-to-tackle-it){.link-annotated-partial .id-not .include-annotation link-icon="facebook" link-icon-type="svg" link-icon-color="#1877f2"}** - **[`https://ai.meta.com/blog/harmful-content-can-evolve-quickly-our-new-ai-system-adapts-to-tackle-it/`{=html}](https://ai.meta.com/blog/harmful-content-can-evolve-quickly-our-new-ai-system-adapts-to-tackle-it/){.link-annotated-partial .id-not .include-annotation link-icon="facebook" link-icon-type="svg" link-icon-color="#1877f2"}** - **[`https://cacm.acm.org/research/the-decline-of-computers-as-a-general-purpose-technology/`{=html}](https://cacm.acm.org/research/the-decline-of-computers-as-a-general-purpose-technology/){.link-annotated-partial .id-not .include-annotation link-icon="acm" link-icon-type="text,tri,sans" link-icon-color="#3795c4"}** - **[`https://chinamediaproject.org/2024/05/27/goldfish-memories/`{=html}](https://chinamediaproject.org/2024/05/27/goldfish-memories/){.link-annotated-partial .id-not .include-annotation}** - **[`https://darioamodei.com/post/we-must-pace-the-frontier`{=html}](https://darioamodei.com/post/we-must-pace-the-frontier){.link-annotated-partial .id-not .include-annotation}** - **[`https://github.com/Dicklesworthstone/the_lighthill_debate_on_ai`{=html}](https://github.com/Dicklesworthstone/the_lighthill_debate_on_ai){.id-not .include-annotation link-icon="github" link-icon-type="svg"}** - **[`https://github.com/features/copilot/`{=html}](https://github.com/features/copilot/){.link-annotated-partial .id-not .include-annotation link-icon="github" link-icon-type="svg"}** - **[`https://karpathy.github.io/2022/03/14/lecun1989/`{=html}](https://karpathy.github.io/2022/03/14/lecun1989/){.link-annotated-partial .id-not .include-annotation}** - **[`https://markovbio.github.io/biomedical-progress/`{=html}](https://markovbio.github.io/biomedical-progress/){.id-not .include-annotation}** ::: aux-links-transclude-file ::: collapse **View External Link**: [`https://markovbio.github.io/biomedical-progress/`](https://markovbio.github.io/biomedical-progress/){.id-not .link-annotated-not .include-content .include-lazy} ::: ::: - **[`https://nonint.com/2023/06/10/the-it-in-ai-models-is-the-dataset/`{=html}](https://nonint.com/2023/06/10/the-it-in-ai-models-is-the-dataset/){.id-not .include-annotation}** - **[`https://nonint.com/2024/03/03/learned-structures/`{=html}](https://nonint.com/2024/03/03/learned-structures/){.link-annotated-partial .id-not .include-annotation}** - **[`https://people.eecs.berkeley.edu/~hendrycks/`{=html}](https://people.eecs.berkeley.edu/~hendrycks/){.link-annotated-partial .id-not .include-annotation link-icon="BAIR" link-icon-type="text,quad,mono"}** - **[`https://research.google/blog/large-scale-matrix-factorization-on-tpus/`{=html}](https://research.google/blog/large-scale-matrix-factorization-on-tpus/){.id-not .include-annotation link-icon="alphabet" link-icon-type="svg" link-icon-color="#4285f4"}** - **[`https://research.google/blog/scalable-deep-reinforcement-learning-for-robotic-manipulation/`{=html}](https://research.google/blog/scalable-deep-reinforcement-learning-for-robotic-manipulation/){.id-not .include-annotation link-icon="alphabet" link-icon-type="svg" link-icon-color="#4285f4"}** - **[`https://scienceblogs.de/klausis-krypto-kolumne/2019/12/19/bigram-750-challenge-solved-new-world-record-set/`{=html}](https://scienceblogs.de/klausis-krypto-kolumne/2019/12/19/bigram-750-challenge-solved-new-world-record-set/){.link-annotated-partial .id-not .include-annotation}** - **[`https://thezvi.substack.com/p/on-openais-preparedness-framework`{=html}](https://thezvi.substack.com/p/on-openais-preparedness-framework){.link-annotated-partial .id-not .include-annotation link-icon="substack" link-icon-type="svg" link-icon-color="#ff6719"}** - **[`https://time.com/6556168/when-ai-outsmart-humans/`{=html}](https://time.com/6556168/when-ai-outsmart-humans/){.link-annotated-partial .id-not .include-annotation link-icon="T" link-icon-type="text" link-icon-color="#e90606"}** - **[`https://towardsdatascience.com/deep-neural-networks-are-biased-at-initialisation-towards-simple-functions-a63487edcb99`{=html}](https://towardsdatascience.com/deep-neural-networks-are-biased-at-initialisation-towards-simple-functions-a63487edcb99){.link-annotated-partial .id-not .include-annotation link-icon="𝐌" link-icon-type="text"}** - **[`https://towardsdatascience.com/neural-networks-are-fundamentally-bayesian-bee9a172fad8`{=html}](https://towardsdatascience.com/neural-networks-are-fundamentally-bayesian-bee9a172fad8){.link-annotated-partial .id-not .include-annotation link-icon="𝐌" link-icon-type="text"}** - **[`https://web.archive.org/web/20210415022657/http://starcraft.blizzplanet.com/blog/comments/blizzcon-2018-starcraft-ii-whats-next-panel-transcript`{=html}](https://web.archive.org/web/20210415022657/http://starcraft.blizzplanet.com/blog/comments/blizzcon-2018-starcraft-ii-whats-next-panel-transcript){.link-annotated-partial .id-not .include-annotation link-icon="internet-archive" link-icon-type="svg"}** - **[`https://windowsontheory.org/2019/12/05/deep-double-descent/`{=html}](https://windowsontheory.org/2019/12/05/deep-double-descent/){.link-annotated-partial .id-not .include-annotation}** - **[`https://www.beren.io/2022-08-06-The-scale-of-the-brain-vs-machine-learning/`{=html}](https://www.beren.io/2022-08-06-The-scale-of-the-brain-vs-machine-learning/){.link-annotated-partial .id-not .include-annotation}** - **[`https://www.dwarkesh.com/p/demis-hassabis#%C2%A7timestamps`{=html}](https://www.dwarkesh.com/p/demis-hassabis#%C2%A7timestamps){.link-annotated-partial .id-not .include-annotation link-icon="Dwkh" link-icon-type="text,sans,quad" link-icon-color="#f3c016"}** - **[`https://www.dwarkesh.com/p/will-scaling-work`{=html}](https://www.dwarkesh.com/p/will-scaling-work){.id-not .include-annotation link-icon="Dwkh" link-icon-type="text,sans,quad" link-icon-color="#f3c016"}** - **[`https://www.lesswrong.com/posts/75o8oja43LXGAqbAR/palm-2-and-gpt-4-in-extrapolating-gpt-n-performance`{=html}](https://www.lesswrong.com/posts/75o8oja43LXGAqbAR/palm-2-and-gpt-4-in-extrapolating-gpt-n-performance){.link-annotated-partial .id-not .include-annotation link-icon="LW" link-icon-type="text" link-icon-color="#7faf83"}** - **[`https://www.lesswrong.com/posts/B8Djo44WtZK6kK4K5/outreach-success-intro-to-ai-risk-that-has-been-successful`{=html}](https://www.lesswrong.com/posts/B8Djo44WtZK6kK4K5/outreach-success-intro-to-ai-risk-that-has-been-successful){.id-not .include-annotation link-icon="LW" link-icon-type="text" link-icon-color="#7faf83"}** ::: aux-links-transclude-file ::: collapse **View External Link**: [`https://www.lesswrong.com/posts/B8Djo44WtZK6kK4K5/outreach-success-intro-to-ai-risk-that-has-been-successful`](https://www.lesswrong.com/posts/B8Djo44WtZK6kK4K5/outreach-success-intro-to-ai-risk-that-has-been-successful){.id-not .link-annotated-not .include-content .include-lazy link-icon="LW" link-icon-type="text" link-icon-color="#7faf83"} ::: ::: - **[`https://www.lesswrong.com/posts/KbRxdBCcJqwtbiPzm/whisper-s-wild-implications-1`{=html}](https://www.lesswrong.com/posts/KbRxdBCcJqwtbiPzm/whisper-s-wild-implications-1){.id-not .include-annotation link-icon="LW" link-icon-type="text" link-icon-color="#7faf83"}** ::: aux-links-transclude-file ::: collapse **View External Link**: [`https://www.lesswrong.com/posts/KbRxdBCcJqwtbiPzm/whisper-s-wild-implications-1`](https://www.lesswrong.com/posts/KbRxdBCcJqwtbiPzm/whisper-s-wild-implications-1){.id-not .link-annotated-not .include-content .include-lazy link-icon="LW" link-icon-type="text" link-icon-color="#7faf83"} ::: ::: - **[`https://www.lesswrong.com/posts/No5JpRCHzBrWA4jmS/q-and-a-with-shane-legg-on-risks-from-ai`{=html}](https://www.lesswrong.com/posts/No5JpRCHzBrWA4jmS/q-and-a-with-shane-legg-on-risks-from-ai){.link-annotated-partial .id-not .include-annotation link-icon="LW" link-icon-type="text" link-icon-color="#7faf83"}** - **[`https://www.lesswrong.com/posts/Rdwui3wHxCeKb7feK/getting-50-sota-on-arc-agi-with-gpt-4o?commentId=JptpWoG5DwNDXxykC`{=html}](https://www.lesswrong.com/posts/Rdwui3wHxCeKb7feK/getting-50-sota-on-arc-agi-with-gpt-4o?commentId=JptpWoG5DwNDXxykC){.link-annotated-partial .id-not .include-annotation link-icon="LW" link-icon-type="text" link-icon-color="#7faf83"}** - **[`https://www.lesswrong.com/posts/dLXdCjxbJMGtDBWTH/no-one-in-my-org-puts-money-in-their-pension`{=html}](https://www.lesswrong.com/posts/dLXdCjxbJMGtDBWTH/no-one-in-my-org-puts-money-in-their-pension){.link-annotated-partial .id-not .include-annotation link-icon="LW" link-icon-type="text" link-icon-color="#7faf83"}** - **[`https://www.lesswrong.com/posts/qdStMFDMrWAnTqNWL/gpt-4-predictions`{=html}](https://www.lesswrong.com/posts/qdStMFDMrWAnTqNWL/gpt-4-predictions){.id-not .include-annotation link-icon="LW" link-icon-type="text" link-icon-color="#7faf83"}** ::: aux-links-transclude-file ::: collapse **View External Link**: [`https://www.lesswrong.com/posts/qdStMFDMrWAnTqNWL/gpt-4-predictions`](https://www.lesswrong.com/posts/qdStMFDMrWAnTqNWL/gpt-4-predictions){.id-not .link-annotated-not .include-content .include-lazy link-icon="LW" link-icon-type="text" link-icon-color="#7faf83"} ::: ::: - **[`https://www.microsoft.com/en-us/research/blog/phi-2-the-surprising-power-of-small-language-models/`{=html}](https://www.microsoft.com/en-us/research/blog/phi-2-the-surprising-power-of-small-language-models/){.link-annotated-partial .id-not .include-annotation link-icon="MS" link-icon-type="text,sans,italic"}** - **[`https://www.reddit.com/r/mlscaling/comments/1ggr0j4/neural_network_recognizer_for_handwritten_zip/`{=html}](https://www.reddit.com/r/mlscaling/comments/1ggr0j4/neural_network_recognizer_for_handwritten_zip/){.id-not .include-annotation link-icon="reddit" link-icon-type="svg" link-icon-color="#ff4500"}** - **[`https://www.reddit.com/r/reinforcementlearning/comments/nsi7bf/what_could_make_ai_conscious_with_wojciech/`{=html}](https://www.reddit.com/r/reinforcementlearning/comments/nsi7bf/what_could_make_ai_conscious_with_wojciech/){.link-annotated-partial .id-not .include-annotation link-icon="reddit" link-icon-type="svg" link-icon-color="#ff4500"}** - **[`https://x.com/AxSauer/status/1644264940218327042`{=html}](https://x.com/AxSauer/status/1644264940218327042 "AxSauer 2026"){.link-annotated-partial .id-not .include-annotation link-icon="twitter" link-icon-type="svg" link-icon-color="#1da1f2"}** - **[`https://x.com/RichardSocher/status/1736161332259614989`{=html}](https://x.com/RichardSocher/status/1736161332259614989 "Socher 2026"){.link-annotated-partial .id-not .include-annotation link-icon="twitter" link-icon-type="svg" link-icon-color="#1da1f2"}** - **[`https://x.com/ShaneLegg/status/1648340576545169410`{=html}](https://x.com/ShaneLegg/status/1648340576545169410 "ShaneLegg 2026"){.link-annotated-partial .id-not .include-annotation link-icon="twitter" link-icon-type="svg" link-icon-color="#1da1f2"}** - **[`https://x.com/TerribleMaps/status/2065881394044338298`{=html}](https://x.com/TerribleMaps/status/2065881394044338298 "TerribleMaps 2026"){.id-not .include-annotation link-icon="twitter" link-icon-type="svg" link-icon-color="#1da1f2"}** ::: aux-links-transclude-file [`https://x.com/TerribleMaps/status/2065881394044338298`](https://x.com/TerribleMaps/status/2065881394044338298 "TerribleMaps 2026"){.id-not .include-content} ::: - **[`https://x.com/TheAhmadOsman/status/2088307781354889487`{=html}](https://x.com/TheAhmadOsman/status/2088307781354889487 "TheAhmadOsman 2026"){.link-modified-recently .id-not .include-annotation link-icon="twitter" link-icon-type="svg" link-icon-color="#1da1f2"}** ::: aux-links-transclude-file [`https://x.com/TheAhmadOsman/status/2088307781354889487`](https://x.com/TheAhmadOsman/status/2088307781354889487 "TheAhmadOsman 2026"){.link-modified-recently .id-not .include-content} ::: - **[`https://x.com/andrewwhite01/status/1634728559506870274`{=html}](https://x.com/andrewwhite01/status/1634728559506870274 "White 2026"){.link-annotated-partial .id-not .include-annotation link-icon="twitter" link-icon-type="svg" link-icon-color="#1da1f2"}** - **[`https://x.com/arankomatsuzaki/status/1936623933119398014`{=html}](https://x.com/arankomatsuzaki/status/1936623933119398014 "Komatsuzaki 2026"){.id-not .include-annotation link-icon="twitter" link-icon-type="svg" link-icon-color="#1da1f2"}** ::: aux-links-transclude-file [`https://x.com/arankomatsuzaki/status/1936623933119398014`](https://x.com/arankomatsuzaki/status/1936623933119398014 "Komatsuzaki 2026"){.id-not .include-content} ::: - **[`https://x.com/arcprize/status/2105007700966871247`{=html}](https://x.com/arcprize/status/2105007700966871247 "arcprize 2026"){.link-modified-recently .link-annotated-partial .id-not .include-annotation link-icon="twitter" link-icon-type="svg" link-icon-color="#1da1f2"}** - **[`https://x.com/borgeaud_s/status/1780988694163321250`{=html}](https://x.com/borgeaud_s/status/1780988694163321250 "borgeaud_s 2026"){.id-not .include-annotation link-icon="twitter" link-icon-type="svg" link-icon-color="#1da1f2"}** ::: aux-links-transclude-file [`https://x.com/borgeaud_s/status/1780988694163321250`](https://x.com/borgeaud_s/status/1780988694163321250 "borgeaud_s 2026"){.id-not .include-content} ::: - **[`https://x.com/davisblalock/status/1542929841338494976`{=html}](https://x.com/davisblalock/status/1542929841338494976 "Blalock 2026"){.link-annotated-partial .id-not .include-annotation link-icon="twitter" link-icon-type="svg" link-icon-color="#1da1f2"}** - **[`https://x.com/fluffykittnmeow/status/1737639861350269213`{=html}](https://x.com/fluffykittnmeow/status/1737639861350269213 "fluffykittnmeow 2026"){.link-annotated-partial .id-not .include-annotation link-icon="twitter" link-icon-type="svg" link-icon-color="#1da1f2"}** - **[`https://x.com/olivercameron/status/1622802466470514688`{=html}](https://x.com/olivercameron/status/1622802466470514688 "olivercameron 2026"){.link-annotated-partial .id-not .include-annotation link-icon="twitter" link-icon-type="svg" link-icon-color="#1da1f2"}** - **[`https://x.com/polynoamial/status/1903501780102926552`{=html}](https://x.com/polynoamial/status/1903501780102926552 "Brown 2026"){.id-not .include-annotation link-icon="twitter" link-icon-type="svg" link-icon-color="#1da1f2"}** ::: aux-links-transclude-file [`https://x.com/polynoamial/status/1903501780102926552`](https://x.com/polynoamial/status/1903501780102926552 "Brown 2026"){.id-not .include-content} ::: - **[`https://x.com/sevensix43/status/2098932778595123307`{=html}](https://x.com/sevensix43/status/2098932778595123307 "sevensix43 2026"){.link-modified-recently .link-annotated-partial .id-not .include-annotation link-icon="twitter" link-icon-type="svg" link-icon-color="#1da1f2"}** # Bibliography {#link-bibliography-section} 1. `https://arxiv.org/abs/2606.03938`: [`“q0: Primitives for Hyper-Epoch Pretraining”`{=html}](https://arxiv.org/abs/2606.03938 "'q0: Primitives for Hyper-Epoch Pretraining', Mandal et al 2026"){#mandal-et-al-2026 .link-annotated}, [Bishwas Mandal, Shmuel Berman, Akshay Vegesna, Samip Dahal]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2606.03938.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2606.03938"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 2. `https://arxiv.org/abs/2510.15020`: [`“The Coverage Principle: How Pre-Training Enables Post-Training”`{=html}](https://arxiv.org/abs/2510.15020 "'The Coverage Principle: How Pre-Training Enables Post-Training', Chen et al 2025"){#chen-et-al-2025 .link-annotated}, [Fan Chen, Audrey Huang, Noah Golowich[, Sadhika Malladi, Adam Block, Jordan T. Ash, Akshay Krishnamurthy, Dylan J. Foster]{.collapse}]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2510.15020.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2510.15020"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 3. `https://www.sciencedirect.com/science/article/pii/S016028962500025X`: [`“Psychometrically Derived 60-Question Benchmarks: Substantial Efficiencies and the Possibility of Human-AI Comparisons”`{=html}](https://www.sciencedirect.com/science/article/pii/S016028962500025X "'Psychometrically derived 60-question benchmarks: Substantial efficiencies and the possibility of human-AI comparisons', Gignac & Ilić 2025"){#gignac-ilić-2025 .link-annotated}, [[Gilles E. Gignac](https://en.wikipedia.org/wiki/Gilles_E._Gignac "Gilles E. Gignac"){.id-not}, David Ilić]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Fwww.sciencedirect.com%252Fscience%252Farticle%252Fpii%252FS016028962500025X.html "Directory-tag link-bibliography for link https://www.sciencedirect.com/science/article/pii/S016028962500025X"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 4. `https://arxiv.org/abs/2503.17074`: [`“Emuru: Zero-Shot Styled Text Image Generation, but Make It Autoregressive”`{=html}](https://arxiv.org/abs/2503.17074 "'Emuru: Zero-Shot Styled Text Image Generation, but Make It Autoregressive', Pippi et al 2025"){#pippi-et-al-2025 .link-annotated}, [Vittorio Pippi, Fabio Quattrini, Silvia Cascianelli[, Alessio Tonioni, Rita Cucchiara]{.collapse}]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2503.17074.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2503.17074"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 5. `https://arxiv.org/abs/2502.09992`: [`“LLaDA: Large Language Diffusion Models”`{=html}](https://arxiv.org/abs/2502.09992 "'LLaDA: Large Language Diffusion Models', Nie et al 2025"){#nie-et-al-2025 .link-annotated}, [Shen Nie, Fengqi Zhu, Zebin You[, Xiaolu Zhang, Jingyang Ou, [Jun Hu](https://en.wikipedia.org/wiki/Jun_Hu "Jun Hu"){.id-not}, Jun Zhou, Yankai Lin, Ji-Rong Wen, Chongxuan Li]{.collapse}]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2502.09992.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2502.09992"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 6. `https://arxiv.org/abs/2501.09038#deepmind`: [`“Do Generative Video Models Learn Physical Principles from Watching Videos?”`{=html}](https://arxiv.org/abs/2501.09038#deepmind "'Do generative video models learn physical principles from watching videos?', Motamed et al 2025"){#motamed-et-al-2025 .link-annotated}, [Saman Motamed, Laura Culp, Kevin Swersky[, Priyank Jaini, [Robert Geirhos](https://scholar.google.com/citations?user=w3kGtMIAAAAJ "Robert Geirhos"){.id-not}]{.collapse}]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2501.09038%2523deepmind.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2501.09038#deepmind"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 7. `https://arxiv.org/abs/2412.04332`: [`“Liquid: Language Models Are Scalable and Unified Multi-Modal Generators”`{=html}](https://arxiv.org/abs/2412.04332 "'Liquid: Language Models are Scalable and Unified Multi-modal Generators', Wu et al 2024"){#wu-et-al-2024 .link-annotated}, [Junfeng Wu, [Yi Jiang](https://en.wikipedia.org/wiki/Yi_Jiang "Yi Jiang"){.id-not}, Chuofan Ma[, Yuliang Liu, Hengshuang Zhao, Zehuan Yuan, Song Bai, Xiang Bai]{.collapse}]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2412.04332.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2412.04332"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 8. `https://arxiv.org/abs/2410.18514`: [`“Scaling up Masked Diffusion Models on Text”`{=html}](https://arxiv.org/abs/2410.18514 "'Scaling up Masked Diffusion Models on Text', Nie et al 2024"){#nie-et-al-2024 .link-annotated}, [Shen Nie, Fengqi Zhu, Chao Du[, Tianyu Pang, [Qian Liu](https://en.wikipedia.org/wiki/Qian_Liu "Qian Liu"){.id-not}, Guangtao Zeng, Min Lin, Chongxuan Li]{.collapse}]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2410.18514.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2410.18514"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 9. `https://research.google/blog/taking-medical-imaging-embeddings-3d/`: [`“CT Foundation: Taking Medical Imaging Embeddings 3D”`{=html}](https://research.google/blog/taking-medical-imaging-embeddings-3d/ "'CT Foundation: Taking medical imaging embeddings 3D', Kiraly & Traverse 2024"){#kiraly-traverse-2024 .link-annotated}, [Atilla Kiraly, Madeleine Traverse]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Fresearch.google%252Fblog%252Ftaking-medical-imaging-embeddings-3d%252F.html "Directory-tag link-bibliography for link https://research.google/blog/taking-medical-imaging-embeddings-3d/"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 10. `https://arxiv.org/abs/2407.04108`: [`“Future Events As Backdoor Triggers: Investigating Temporal Vulnerabilities in LLMs”`{=html}](https://arxiv.org/abs/2407.04108 "'Future Events as Backdoor Triggers: Investigating Temporal Vulnerabilities in LLMs', Price et al 2024"){#price-et-al-2024-2 .link-annotated}, [Sara Price, Arjun Panickssery, [Samuel R. Bowman](https://cims.nyu.edu/~sbowman/ "Sam Bowman"){.link-annotated-partial .id-not}, Asa Cooper Stickland]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2407.04108.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2407.04108"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 11. `https://arxiv.org/abs/2406.19146`: [`“Resolving Discrepancies in Compute-Optimal Scaling of Language Models”`{=html}](https://arxiv.org/abs/2406.19146 "'Resolving Discrepancies in Compute-Optimal Scaling of Language Models', Porian et al 2024"){#porian-et-al-2024 .link-annotated}, [Tomer Porian, [Mitchell Wortsman](https://mitchellnw.github.io/ "Mitchell Wortsman"){.id-not}, Jenia Jitsev[, [Ludwig Schmidt](https://en.wikipedia.org/wiki/Ludwig_Schmidt "Ludwig Schmidt"){.id-not}, Yair Carmon]{.collapse}]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2406.19146.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2406.19146"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 12. `https://arxiv.org/abs/2406.13121#google`: [`“Can Long-Context Language Models Subsume Retrieval, RAG, SQL, and More?”`{=html}](https://arxiv.org/abs/2406.13121#google "'Can Long-Context Language Models Subsume Retrieval, RAG, SQL, and More?', Lee et al 2024"){#lee-et-al-2024-1 .link-annotated}, [Jinhyuk Lee, [Anthony Chen](https://en.wikipedia.org/wiki/Anthony_Chen "Anthony Chen"){.id-not}, Zhuyun Dai[, Dheeru Dua, Devendra Singh Sachan, Michael Boratko, Yi Luan, Sébastien M. R. Arnold, Vincent Perot, Siddharth Dalmia, Hexiang Hu, Xudong Lin, Panupong Pasupat, Aida Amini, Jeremy R. Cole, [Sebastian Riedel](http://www.riedelcastro.org/ "'Sebastian Riedel homepage', Riedel 2026"){.link-annotated-partial .id-not}, Iftekhar Naim, Ming-Wei Chang, Kelvin Guu]{.collapse}]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2406.13121%2523google.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2406.13121#google"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 13. `https://arxiv.org/abs/2406.11233`: [`“Probing the Decision Boundaries of In-Context Learning in Large Language Models”`{=html}](https://arxiv.org/abs/2406.11233 "'Probing the Decision Boundaries of In-context Learning in Large Language Models', Zhao et al 2024"){#zhao-et-al-2024-2 .link-annotated}, [Siyan Zhao, Tung Nguyen, [Aditya Grover](https://scholar.google.com/citations?user=oOhnPUgAAAAJ "Aditya Grover"){.id-not}]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2406.11233.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2406.11233"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 14. `https://www.biorxiv.org/content/10.1101/2024.06.06.597716.full`: [`“Training Compute-Optimal Protein Language Models”`{=html}](https://www.biorxiv.org/content/10.1101/2024.06.06.597716.full "'Training Compute-Optimal Protein Language Models', Cheng et al 2024"){#cheng-et-al-2024-1 .link-annotated}, [Xingyi Cheng, Bo Chen, Pan Li[, Jing Gong, [Jie Tang](https://en.wikipedia.org/wiki/Jie_Tang "Jie Tang"){.id-not}, Le Song]{.collapse}]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Fwww.biorxiv.org%252Fcontent%252F10.1101%252F2024.06.06.597716.full.html "Directory-tag link-bibliography for link https://www.biorxiv.org/content/10.1101/2024.06.06.597716.full"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 15. `https://arxiv.org/abs/2405.14930`: [`“AstroPT: Scaling Large Observation Models for Astronomy”`{=html}](https://arxiv.org/abs/2405.14930 "'AstroPT: Scaling Large Observation Models for Astronomy', Smith et al 2024"){#smith-et-al-2024 .link-annotated}, [Michael J. Smith, Ryan J. Roberts, Eirini Angeloudi, Marc Huertas-Company]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2405.14930.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2405.14930"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 16. `https://arxiv.org/abs/2405.00332#scale`: [`“GSM1k: A Careful Examination of Large Language Model Performance on Grade School Arithmetic”`{=html}](https://arxiv.org/abs/2405.00332#scale "'GSM1k: A Careful Examination of Large Language Model Performance on Grade School Arithmetic', Zhang et al 2024"){#zhang-et-al-2024-11 .link-annotated}, [Hugh Zhang, Jeff Da, Dean Lee[, Vaughn Robinson, Catherine Wu, Will Song, Tiffany Zhao, Pranav Raja, Dylan Slack, Qin Lyu, Sean Hendryx, Russell Kaplan, Michele Lunati, Summer Yue]{.collapse}]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2405.00332%2523scale.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2405.00332#scale"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 17. `https://arxiv.org/abs/2404.10618`: [`“Private Attribute Inference from Images With Vision-Language Models”`{=html}](https://arxiv.org/abs/2404.10618 "'Private Attribute Inference from Images with Vision-Language Models', Tömekçe et al 2024"){#tömekçe-et-al-2024 .link-annotated}, [Batuhan Tömekçe, Mark Vero, Robin Staab, [Martin Vechev](https://en.wikipedia.org/wiki/Martin_Vechev "Martin Vechev"){.id-not}]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2404.10618.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2404.10618"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 18. `https://lab42.global/community-interview-jack-cole/`: [`“Test-Time Augmentation to Solve ARC”`{=html}](https://lab42.global/community-interview-jack-cole/ "'Test-Time Augmentation to solve ARC', Cole 2024"){#cole-2024 .link-annotated}, [Jack Cole]{.author .cite-author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Flab42.global%252Fcommunity-interview-jack-cole%252F.html "Directory-tag link-bibliography for link https://lab42.global/community-interview-jack-cole/"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 19. `https://arxiv.org/abs/2404.09937`: [`“Compression Represents Intelligence Linearly”`{=html}](https://arxiv.org/abs/2404.09937 "'Compression Represents Intelligence Linearly', Huang et al 2024"){#huang-et-al-2024 .link-annotated}, [Yuzhen Huang, Jinghan Zhang, Zifei Shan, Junxian He]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2404.09937.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2404.09937"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 20. `https://arxiv.org/abs/2404.06664`: [`“CulturalTeaming: AI-Assisted Interactive Red-Teaming for Challenging LLMs’ (Lack Of) Multicultural Knowledge”`{=html}](https://arxiv.org/abs/2404.06664 "'CulturalTeaming: AI-Assisted Interactive Red-Teaming for Challenging LLMs’ (Lack of) Multicultural Knowledge', Chiu et al 2024"){#chiu-et-al-2024 .link-annotated}, [Yu Ying Chiu, Liwei Jiang, Maria Antoniak[, Chan Young Park, Shuyue Stella Li, Mehar Bhatia, Sahithya Ravi, Yulia Tsvetkov, Vered Shwartz, [Yejin Choi](https://en.wikipedia.org/wiki/Yejin_Choi "Yejin Choi"){.id-not}]{.collapse}]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2404.06664.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2404.06664"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 21. `https://arxiv.org/abs/2404.02905#bytedance`: [`“Visual Autoregressive Modeling (VAR): Scalable Image Generation via Next-Scale Prediction”`{=html}](https://arxiv.org/abs/2404.02905#bytedance "'Visual Autoregressive Modeling (VAR): Scalable Image Generation via Next-Scale Prediction', Tian et al 2024"){#tian-et-al-2024 .link-annotated}, [Keyu Tian, [Yi Jiang](https://en.wikipedia.org/wiki/Yi_Jiang "Yi Jiang"){.id-not}, Zehuan Yuan[, Bingyue Peng, Liwei Wang]{.collapse}]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2404.02905%2523bytedance.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2404.02905#bytedance"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 22. `https://arxiv.org/abs/2403.18802#deepmind`: [`“Long-Form Factuality in Large Language Models”`{=html}](https://arxiv.org/abs/2403.18802#deepmind "'Long-form factuality in large language models', Wei et al 2024"){#wei-et-al-2024-1 .link-annotated}, [Jerry Wei, Chengrun Yang, Xinying Song[, Yifeng Lu, Nathan Hu, Jie Huang, Dustin Tran, Daiyi Peng, Ruibo Liu, Da Huang, Cosmo Du, [Quoc V. Le](https://en.wikipedia.org/wiki/Quoc_V._Le "Quoc V. Le"){.id-not}]{.collapse}]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2403.18802%2523deepmind.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2403.18802#deepmind"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 23. `https://arxiv.org/abs/2403.17844`: [`“Mechanistic Design and Scaling of Hybrid Architectures”`{=html}](https://arxiv.org/abs/2403.17844 "'Mechanistic Design and Scaling of Hybrid Architectures', Poli et al 2024"){#poli-et-al-2024 .link-annotated}, [Michael Poli, Armin W. Thomas, Eric Nguyen[, Pragaash Ponnusamy, Björn Deiseroth, [Kristian Kersting](https://en.wikipedia.org/wiki/Kristian_Kersting "Kristian Kersting"){.id-not}, Taiji Suzuki, Brian Hie, [Stefano Ermon](https://scholar.google.com/citations?user=ogXTOZ4AAAAJ "Stefano Ermon"){.id-not}, [Christopher Ré](https://en.wikipedia.org/wiki/Christopher_R%C3%A9 "Christopher Ré"){.id-not}, Ce Zhang, Stefano Massaroli]{.collapse}]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2403.17844.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2403.17844"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 24. `https://www.wired.com/story/eight-google-employees-invented-modern-ai-transformers-paper/`: [`“8 Google Employees Invented Modern AI. Here’s the Inside Story: They Met by Chance, Got Hooked on an Idea, and Wrote the Transformers Paper—The Most Consequential Tech Breakthrough in Recent History”`{=html}](https://www.wired.com/story/eight-google-employees-invented-modern-ai-transformers-paper/ "'8 Google Employees Invented Modern AI. Here’s the Inside Story: They met by chance, got hooked on an idea, and wrote the Transformers paper—the most consequential tech breakthrough in recent history', Levy 2024"){#levy-2024-2 .link-annotated}, [[Steven Levy](https://en.wikipedia.org/wiki/Steven_Levy "Steven Levy"){.link-annotated-partial .id-not}]{.author .cite-author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Fwww.wired.com%252Fstory%252Feight-google-employees-invented-modern-ai-transformers-paper%252F.html "Directory-tag link-bibliography for link https://www.wired.com/story/eight-google-employees-invented-modern-ai-transformers-paper/"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 25. `https://inflection.ai/inflection-2-5`: [`“Inflection-2.5: Meet the World’s Best Personal AI”`{=html}](https://inflection.ai/inflection-2-5 "'Inflection-2.5: meet the world’s best personal AI', Inflection 2024"){#inflection-2024 .link-annotated}, [Inflection]{.author .cite-author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Finflection.ai%252Finflection-2-5.html "Directory-tag link-bibliography for link https://inflection.ai/inflection-2-5"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 26. `https://arxiv.org/abs/2403.01518#deepmind`: [`“Revisiting Dynamic Evaluation: Online Adaptation for Large Language Models”`{=html}](https://arxiv.org/abs/2403.01518#deepmind "'Revisiting Dynamic Evaluation: Online Adaptation for Large Language Models', Rannen-Triki et al 2024"){#rannen-triki-et-al-2024 .link-annotated}, [Amal Rannen-Triki, Jorg Bornschein, [Razvan Pascanu](https://sites.google.com/view/razp/home){.link-annotated-partial .id-not}[, [Marcus Hutter](https://en.wikipedia.org/wiki/Marcus_Hutter "Marcus Hutter"){.id-not}, Andras György, Alexandre Galashov, [Yee Whye Teh](https://en.wikipedia.org/wiki/Yee_Whye_Teh "Yee Whye Teh"){.id-not}, Michalis K. Titsias]{.collapse}]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2403.01518%2523deepmind.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2403.01518#deepmind"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 27. `https://arxiv.org/abs/2402.17152#facebook`: [`“Actions Speak Louder Than Words: Trillion-Parameter Sequential Transducers for Generative Recommendations (HSTU)”`{=html}](https://arxiv.org/abs/2402.17152#facebook "'Actions Speak Louder than Words: Trillion-Parameter Sequential Transducers for Generative Recommendations (HSTU)', Zhai et al 2024"){#zhai-et-al-2024-1 .link-annotated}, [Jiaqi Zhai, Lucy Liao, Xing Liu[, Yueming Wang, Rui Li, Xuan Cao, Leon Gao, Zhaojie Gong, Fangda Gu, Michael He, Yinghai Lu, [Yu Shi](https://en.wikipedia.org/wiki/Yu_Shi "Yu Shi"){.id-not}]{.collapse}]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2402.17152%2523facebook.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2402.17152#facebook"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 28. `https://arxiv.org/abs/2402.16671`: [`“StructLM: Towards Building Generalist Models for Structured Knowledge Grounding”`{=html}](https://arxiv.org/abs/2402.16671 "'StructLM: Towards Building Generalist Models for Structured Knowledge Grounding', Zhuang et al 2024"){#zhuang-et-al-2024 .link-annotated}, [Alex Zhuang, Ge Zhang, Tianyu Zheng[, Xinrun Du, Junjie Wang, Weiming Ren, Stephen W. Huang, Jie Fu, Xiang Yue, Wenhu Chen]{.collapse}]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2402.16671.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2402.16671"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 29. `https://arxiv.org/abs/2312.15770#alibaba`: [`“TF-T2V: A Recipe for Scaling up Text-To-Video Generation With Text-Free Videos”`{=html}](https://arxiv.org/abs/2312.15770#alibaba "'TF-T2V: A Recipe for Scaling up Text-to-Video Generation with Text-free Videos', Wang et al 2023"){#wang-et-al-2023-05 .link-annotated}, [Xiang Wang, Shiwei Zhang, Hangjie Yuan[, Zhiwu Qing, Biao Gong, Yingya Zhang, Yujun Shen, Changxin Gao, Nong Sang]{.collapse}]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2312.15770%2523alibaba.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2312.15770#alibaba"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 30. `https://arxiv.org/abs/2312.04927`: [`“Zoology: Measuring and Improving Recall in Efficient Language Models”`{=html}](https://arxiv.org/abs/2312.04927 "'Zoology: Measuring and Improving Recall in Efficient Language Models', Arora et al 2023"){#arora-et-al-2023-1 .link-annotated}, [Simran Arora, Sabri Eyuboglu, Aman Timalsina[, Isys Johnson, Michael Poli, [James Y. Zou](https://www.james-zou.com/ "'James Zou homepage [Machine learning]', Zou 2019"){.id-not}, [Atri Rudra](https://cse.buffalo.edu/faculty/atri/ "Atri Rudra"){.id-not}, [Christopher Ré](https://en.wikipedia.org/wiki/Christopher_R%C3%A9 "Christopher Ré"){.id-not}]{.collapse}]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2312.04927.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2312.04927"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 31. `https://arxiv.org/abs/2312.03876`: [`“Scaling Transformer Neural Networks for Skillful and Reliable Medium-Range Weather Forecasting”`{=html}](https://arxiv.org/abs/2312.03876 "'Scaling transformer neural networks for skillful and reliable medium-range weather forecasting', Nguyen et al 2023"){#nguyen-et-al-2023-1 .link-annotated}, [Tung Nguyen, Rohan Shah, Hritik Bansal[, Troy Arcomano, Sandeep Madireddy, Romit Maulik, Veerabhadra Kotamarthi, Ian Foster, [Aditya Grover](https://scholar.google.com/citations?user=oOhnPUgAAAAJ "Aditya Grover"){.id-not}]{.collapse}]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2312.03876.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2312.03876"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 32. `https://arxiv.org/abs/2312.00752`: [`“Mamba: Linear-Time Sequence Modeling With Selective State Spaces”`{=html}](https://arxiv.org/abs/2312.00752 "'Mamba: Linear-Time Sequence Modeling with Selective State Spaces', Gu & Dao 2023"){#gu-dao-2023 .link-annotated}, [[Albert Gu](https://scholar.google.com/citations?user=DVCHv1kAAAAJ "Albert Gu"){.id-not}, [Tri Dao](https://tridao.me/ "Tri Dao"){.link-annotated-partial .id-not}]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2312.00752.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2312.00752"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 33. `https://arxiv.org/abs/2311.15599#tencent`: [`“UniRepLKNet: A Universal Perception Large-Kernel ConvNet for Audio, Video, Point Cloud, Time-Series and Image Recognition”`{=html}](https://arxiv.org/abs/2311.15599#tencent "'UniRepLKNet: A Universal Perception Large-Kernel ConvNet for Audio, Video, Point Cloud, Time-Series and Image Recognition', Ding et al 2023"){#ding-et-al-2023-1 .link-annotated}, [Xiaohan Ding, Yiyuan Zhang, Yixiao Ge[, Sijie Zhao, Lin Song, Xiangyu Yue, [Ying Shan](https://en.wikipedia.org/wiki/Ying_Shan "Ying Shan"){.id-not}]{.collapse}]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2311.15599%2523tencent.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2311.15599#tencent"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 34. `https://arxiv.org/abs/2311.04145#alibaba`: [`“I2VGen-XL: High-Quality Image-To-Video Synthesis via Cascaded Diffusion Models”`{=html}](https://arxiv.org/abs/2311.04145#alibaba "'I2VGen-XL: High-Quality Image-to-Video Synthesis via Cascaded Diffusion Models', Zhang et al 2023"){#zhang-et-al-2023-07 .link-annotated}, [Shiwei Zhang, Jiayu Wang, Yingya Zhang[, [Kang Zhao](https://en.wikipedia.org/wiki/Kang_Zhao "Kang Zhao"){.id-not}, Hangjie Yuan, Zhiwu Qin, Xiang Wang, Deli Zhao, Jingren Zhou]{.collapse}]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2311.04145%2523alibaba.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2311.04145#alibaba"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 35. `https://arxiv.org/abs/2310.16764#deepmind`: [`“ConvNets Match Vision Transformers at Scale”`{=html}](https://arxiv.org/abs/2310.16764#deepmind "'ConvNets Match Vision Transformers at Scale', Smith et al 2023"){#smith-et-al-2023-2 .link-annotated}, [[Samuel L. Smith](https://en.wikipedia.org/wiki/Samuel_L._Smith "Samuel L. Smith"){.id-not}, [Andrew Brock](https://en.wikipedia.org/wiki/Andrew_Brock "Andrew Brock"){.id-not}, Leonard Berrada, Soham De]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2310.16764%2523deepmind.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2310.16764#deepmind"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 36. `https://arxiv.org/abs/2310.09199#google`: [`“PaLI-3 Vision Language Models: Smaller, Faster, Stronger”`{=html}](https://arxiv.org/abs/2310.09199#google "'PaLI-3 Vision Language Models: Smaller, Faster, Stronger', Chen et al 2023"){#chen-et-al-2023-pali3 .link-annotated}, [[Xi Chen](https://en.wikipedia.org/wiki/Xi_Chen "Xi Chen"){.id-not}, [Xiao Wang](https://en.wikipedia.org/wiki/Xiao_Wang "Xiao Wang"){.id-not}, [Lucas Beyer](https://scholar.google.com/citations?user=p2gwhK4AAAAJ "Lucas Beyer"){.id-not}[, Alexander Kolesnikov, Jialin Wu, Paul Voigtlaender, Basil Mustafa, Sebastian Goodman, Ibrahim Alabdulmohsin, Piotr Padlewski, Daniel Salz, Xi Xiong, Daniel Vlasic, Filip Pavetic, Keran Rong, Tianli Yu, Daniel Keysers, [Xiaohua Zhai](https://sites.google.com/view/xzhai "Xiaohua Zhai (翟晓华)"){.link-annotated-partial .id-not}, Radu Soricut]{.collapse}]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2310.09199%2523google.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2310.09199#google"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 37. `https://arxiv.org/abs/2310.06213`: [`“GeoLLM: Extracting Geospatial Knowledge from Large Language Models”`{=html}](https://arxiv.org/abs/2310.06213 "'GeoLLM: Extracting Geospatial Knowledge from Large Language Models', Manvi et al 2023"){#manvi-et-al-2023 .link-annotated}, [Rohin Manvi, Samar Khanna, Gengchen Mai[, Marshall Burke, [David Lobell](https://en.wikipedia.org/wiki/David_Lobell "David Lobell"){.id-not}, [Stefano Ermon](https://scholar.google.com/citations?user=ogXTOZ4AAAAJ "Stefano Ermon"){.id-not}]{.collapse}]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2310.06213.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2310.06213"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 38. `https://arxiv.org/abs/2310.06694`: [`“Sheared LLaMA: Accelerating Language Model Pre-Training via Structured Pruning”`{=html}](https://arxiv.org/abs/2310.06694 "'Sheared LLaMA: Accelerating Language Model Pre-training via Structured Pruning', Xia et al 2023"){#xia-et-al-2023-1 .link-annotated}, [Mengzhou Xia, Tianyu Gao, Zhiyuan Zeng, [Danqi Chen](https://en.wikipedia.org/wiki/Danqi_Chen "Danqi Chen"){.id-not}]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2310.06694.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2310.06694"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 39. `https://arxiv.org/abs/2310.03214#google`: [`“FreshLLMs: Refreshing Large Language Models With Search Engine Augmentation”`{=html}](https://arxiv.org/abs/2310.03214#google "'FreshLLMs: Refreshing Large Language Models with Search Engine Augmentation', Vu et al 2023"){#vu-et-al-2023 .link-annotated}, [Tu Vu, Mohit Iyyer, Xuezhi Wang[, [Noah Constant](https://scholar.google.com/citations?user=PbgcS6AAAAAJ "Noah Constant"){.id-not}, Jerry Wei, [Jason Wei](https://www.jasonwei.net/ "Jason Wei"){.id-not}, Chris Tar, Yun-Hsuan Sung, [Denny Zhou](https://dennyzhou.github.io/ "Denny Zhou’s Home Page"){.link-annotated-partial .id-not}, [Quoc V. Le](https://en.wikipedia.org/wiki/Quoc_V._Le "Quoc V. Le"){.id-not}, Thang Luong]{.collapse}]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2310.03214%2523google.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2310.03214#google"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 40. `https://arxiv.org/abs/2310.02980`: [`“Never Train from Scratch: Fair Comparison of Long-Sequence Models Requires Data-Driven Priors”`{=html}](https://arxiv.org/abs/2310.02980 "'Never Train from Scratch: Fair Comparison of Long-Sequence Models Requires Data-Driven Priors', Amos et al 2023"){#amos-et-al-2023 .link-annotated}, [Ido Amos, [Jonathan Berant](https://www.cs.tau.ac.il//~joberant/ "Jonathan Berant"){.link-annotated-partial .id-not}, Ankit Gupta]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2310.02980.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2310.02980"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 41. `https://arxiv.org/abs/2309.00667`: [`“Taken out of Context: On Measuring Situational Awareness in LLMs”`{=html}](https://arxiv.org/abs/2309.00667 "'Taken out of context: On measuring situational awareness in LLMs', Berglund et al 2023"){#berglund-et-al-2023-2 .link-annotated}, [Lukas Berglund, Asa Cooper Stickland, Mikita Balesni[, Max Kaufmann, Meg Tong, Tomasz Korbak, [Daniel Kokotajlo](https://en.wikipedia.org/wiki/Daniel_Kokotajlo "Daniel Kokotajlo"){.id-not}, [Owain Evans](https://owainevans.github.io/ "Owain Evans, AI Alignment researcher"){.link-annotated-partial .id-not}]{.collapse}]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2309.00667.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2309.00667"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 42. `https://arxiv.org/abs/2308.11596#facebook`: [`“SeamlessM4T: Massively Multilingual & Multimodal Machine Translation”`{=html}](https://arxiv.org/abs/2308.11596#facebook "'SeamlessM4T: Massively Multilingual & Multimodal Machine Translation', Communication et al 2023"){#communication-et-al-2023-2 .link-annotated}, [Seamless Communication, Loïc Barrault, Yu-An Chung[, Mariano Cora Meglioli, [David Dale](https://en.wikipedia.org/wiki/David_Dale "David Dale"){.id-not}, Ning Dong, Paul-Ambroise Duquenne, Hady Elsahar, Hongyu Gong, Kevin Heffernan, John Hoffman, Christopher Klaiber, Pengwei Li, Daniel Licht, [Jean Maillard](https://en.wikipedia.org/wiki/Jean_Maillard "Jean Maillard"){.id-not}, Alice Rakotoarison, Kaushik Ram Sadagopan, Guillaume Wenzek, Ethan Ye, Bapi Akula, Peng-Jen Chen, Naji El Hachem, Brian Ellis, Gabriel Mejia Gonzalez, Justin Haaheim, Prangthip Hansanti, Russ Howes, Bernie Huang, Min-Jae Hwang, Hirofumi Inaguma, Somya Jain, Elahe Kalbassi, Amanda Kallet, Ilia Kulikov, Janice Lam, Daniel Li, Xutai Ma, Ruslan Mavlyutov, Benjamin Peloquin, Mohamed Ramadan, Abinesh Ramakrishnan, [Anna Sun](https://en.wikipedia.org/wiki/Anna_Sun "Anna Sun"){.id-not}, [Kevin Tran](https://en.wikipedia.org/wiki/Kevin_Tran "Kevin Tran"){.id-not}, Tuan Tran, Igor Tufanov, Vish Vogeti, Carleigh Wood, [Yilin Yang](https://en.wikipedia.org/wiki/Yilin_Yang "Yilin Yang"){.id-not}, Bokai Yu, Pierre Andrews, Can Balioglu, Marta R. Costa-jussà, Onur Celebi, Maha Elbayad, Cynthia Gao, Francisco Guzmán, Justine Kao, [Ann Lee](https://en.wikipedia.org/wiki/Ann_Lee "Ann Lee"){.id-not}, Alexandre Mourachko, [Juan Pino](https://en.wikipedia.org/wiki/Juan_Pino "Juan Pino"){.id-not}, Sravya Popuri, Christophe Ropers, Safiyyah Saleem, Holger Schwenk, Paden Tomasello, Changhan Wang, Jeff Wang, Skyler Wang]{.collapse}]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2308.11596%2523facebook.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2308.11596#facebook"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 43. `https://arxiv.org/abs/2308.03958#deepmind`: [`“Simple Synthetic Data Reduces Sycophancy in Large Language Models”`{=html}](https://arxiv.org/abs/2308.03958#deepmind "'Simple synthetic data reduces sycophancy in large language models', Wei et al 2023"){#wei-et-al-2023-2 .link-annotated}, [Jerry Wei, Da Huang, Yifeng Lu[, [Denny Zhou](https://dennyzhou.github.io/ "Denny Zhou’s Home Page"){.link-annotated-partial .id-not}, [Quoc V. Le](https://en.wikipedia.org/wiki/Quoc_V._Le "Quoc V. Le"){.id-not}]{.collapse}]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2308.03958%2523deepmind.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2308.03958#deepmind"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 44. `https://arxiv.org/abs/2307.05300#microsoft`: [`“Unleashing the Emergent Cognitive Synergy in Large Language Models: A Task-Solving Agent through Multi-Persona Self-Collaboration”`{=html}](https://arxiv.org/abs/2307.05300#microsoft "'Unleashing the Emergent Cognitive Synergy in Large Language Models: A Task-Solving Agent through Multi-Persona Self-Collaboration', Wang et al 2023"){#wang-et-al-2023-12 .link-annotated}, [Zhenhailong Wang, Shaoguang Mao, Wenshan Wu[, Tao Ge, [Furu Wei](https://scholar.google.com/citations?user=G-V1VpwAAAAJ "Furu Wei"){.id-not}, [Heng Ji](https://en.wikipedia.org/wiki/Heng_Ji "Heng Ji"){.id-not}]{.collapse}]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2307.05300%2523microsoft.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2307.05300#microsoft"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 45. `https://openai.com/index/introducing-superalignment/`: [`“Introducing Superalignment”`{=html}](https://openai.com/index/introducing-superalignment/ "'Introducing Superalignment', Leike & Sutskever 2023"){#leike-sutskever-2023 .link-annotated}, [[Jan Leike](https://jan.leike.name/ "Jan Leike"){.link-annotated-partial .id-not}, [Ilya Sutskever](https://en.wikipedia.org/wiki/Ilya_Sutskever "Ilya Sutskever"){.link-annotated-partial .id-not}]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Fopenai.com%252Findex%252Fintroducing-superalignment%252F.html "Directory-tag link-bibliography for link https://openai.com/index/introducing-superalignment/"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 46. `https://www.youtube.com/watch?v=lfXxzAVtdpU&t=1763s`: [`“Gödel, Escher, Bach Author Douglas Hofstadter on the State of AI Today § What about AI Terrifies You?”`{=html}](https://www.youtube.com/watch?v=lfXxzAVtdpU&t=1763s "'Gödel, Escher, Bach author Douglas Hofstadter on the state of AI today § What about AI terrifies you?', Hofstadter & Kim 2023"){#hofstadter-kim-2023 .link-annotated}, [[Douglas Hofstadter](https://en.wikipedia.org/wiki/Douglas_Hofstadter "Douglas Hofstadter"){.link-annotated-partial .id-not}, Amy Jo Kim]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Fwww.youtube.com%252Fwatch%253Fv%253DlfXxzAVtdpU%2526t%253D1763s.html "Directory-tag link-bibliography for link https://www.youtube.com/watch?v=lfXxzAVtdpU&t=1763s"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 47. `https://arxiv.org/abs/2306.13575`: [`“Scaling MLPs: A Tale of Inductive Bias”`{=html}](https://arxiv.org/abs/2306.13575 "'Scaling MLPs: A Tale of Inductive Bias', Bachmann et al 2023"){#bachmann-et-al-2023 .link-annotated}, [Gregor Bachmann, Sotiris Anagnostidis, [Thomas Hofmann](https://en.wikipedia.org/wiki/Thomas_Hofmann "Thomas Hofmann"){.id-not}]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2306.13575.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2306.13575"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 48. `https://arxiv.org/abs/2306.15448`: [`“Understanding Social Reasoning in Language Models With Language Models”`{=html}](https://arxiv.org/abs/2306.15448 "'Understanding Social Reasoning in Language Models with Language Models', Gandhi et al 2023"){#gandhi-et-al-2023-2 .link-annotated}, [Kanishk Gandhi, Jan-Philipp Fränken, Tobias Gerstenberg, [Noah D. Goodman](https://scholar.google.com/citations?user=OUpIbcQAAAAJ "Noah D. Goodman"){.id-not}]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2306.15448.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2306.15448"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 49. `https://arxiv.org/abs/2305.15717`: [`“The False Promise of Imitating Proprietary LLMs”`{=html}](https://arxiv.org/abs/2305.15717 "'The False Promise of Imitating Proprietary LLMs', Gudibande et al 2023"){#gudibande-et-al-2023 .link-annotated}, [Arnav Gudibande, [Eric Wallace](https://en.wikipedia.org/wiki/Eric_Wallace "Eric Wallace"){.id-not}, [Charlie Snell](https://en.wikipedia.org/wiki/Charlie_Snell "Charlie Snell"){.id-not}[, Xinyang Geng, Hao Liu, [Pieter Abbeel](https://en.wikipedia.org/wiki/Pieter_Abbeel "Pieter Abbeel"){.id-not}, [Sergey Levine](https://scholar.google.com/citations?user=8R35rCwAAAAJ "Sergey Levine"){.id-not}, [Dawn Song](https://en.wikipedia.org/wiki/Dawn_Song "Dawn Song"){.id-not}]{.collapse}]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2305.15717.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2305.15717"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 50. `https://arxiv.org/abs/2305.11863`: [`“Scaling Laws for Language Encoding Models in FMRI”`{=html}](https://arxiv.org/abs/2305.11863 "'Scaling laws for language encoding models in fMRI', Antonello et al 2023"){#antonello-et-al-2023 .link-annotated}, [Richard Antonello, Aditya Vaidya, Alexander G. Huth]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2305.11863.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2305.11863"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 51. `https://arxiv.org/abs/2305.10403#google`: [`“PaLM 2 Technical Report”`{=html}](https://arxiv.org/abs/2305.10403#google "'PaLM 2 Technical Report', Anil et al 2023"){#anil-et-al-2023 .link-annotated}, [Rohan Anil, Andrew M. Dai, [Orhan Firat](https://orhanfirat.com/ "Orhan Fırat"){.link-annotated-partial .id-not}[, [Melvin Johnson](https://en.wikipedia.org/wiki/Melvin_Johnson "Melvin Johnson"){.id-not}, Dmitry Lepikhin, Alexandre Passos, Siamak Shakeri, Emanuel Taropa, Paige Bailey, [Zhifeng Chen](https://scholar.google.com/citations?user=EemUE4gAAAAJ "Zhifeng Chen"){.id-not}, [Eric Chu](https://en.wikipedia.org/wiki/Eric_Chu "Eric Chu"){.id-not}, Jonathan H. Clark, Laurent El Shafey, [Yanping Huang](https://scholar.google.com/citations?user=uEtBQScAAAAJ "Yanping Huang"){.id-not}, Kathy Meier-Hellstern, Gaurav Mishra, Erica Moreira, Mark Omernick, Kevin Robinson, Sebastian Ruder, [Yi Tay](https://www.yitay.net/ "Yi Tay"){.link-annotated-partial .id-not}, Kefan Xiao, Yuanzhong Xu, Yujing Zhang, Gustavo Hernandez Abrego, Junwhan Ahn, [Jacob Austin](https://en.wikipedia.org/wiki/Jacob_Austin "Jacob Austin"){.id-not}, [Paul Barham](https://en.wikipedia.org/wiki/Paul_Barham "Paul Barham"){.id-not}, Jan Botha, [James Bradbury](https://x.com/jekbradbury "Bradbury 2026"){.id-not}, Siddhartha Brahma, Kevin Brooks, Michele Catasta, Yong Cheng, [Colin Cherry](https://en.wikipedia.org/wiki/Colin_Cherry "Colin Cherry"){.id-not}, Christopher A. Choquette-Choo, Aakanksha Chowdhery, Clément Crepy, Shachi Dave, [Mostafa Dehghani](https://scholar.google.com/citations?user=MiHOX3QAAAAJ "Mostafa Dehghani"){.id-not}, Sunipa Dev, Jacob Devlin, Mark Díaz, Nan Du, [Ethan Dyer](https://scholar.google.com/citations?user=LWeVRdUAAAAJ "Ethan Dyer"){.id-not}, Vlad Feinberg, Fangxiaoyu Feng, Vlad Fienber, Markus Freitag, Xavier Garcia, Sebastian Gehrmann, Lucas Gonzalez, [Guy Gur-Ari](https://scholar.google.com/citations?user=mx8P4QUAAAAJ "Guy Gur-Ari"){.id-not}, Steven Hand, Hadi Hashemi, Le Hou, Joshua Howland, Andrea Hu, Jeffrey Hui, Jeremy Hurwitz, Michael Isard, Abe Ittycheriah, Matthew Jagielski, Wenhao Jia, [Kathleen Kenealy](https://en.wikipedia.org/wiki/Kathleen_Kenealy "Kathleen Kenealy"){.id-not}, Maxim Krikun, Sneha Kudugunta, Chang Lan, Katherine Lee, Benjamin Lee, Eric Li, Music Li, [Wei Li](https://scholar.google.com/citations?user=7IUCbE4AAAAJ "Wei Li"){.id-not}, YaGuang Li, Jian Li, Hyeontaek Lim, Hanzhao Lin, Zhongtao Liu, Frederick Liu, Marcello Maggioni, Aroma Mahendru, Joshua Maynez, [Vedant Misra](http://vedantmisra.com/ "Vedant Misra"){.id-not}, Maysam Moussalem, Zachary Nado, John Nham, Eric Ni, Andrew Nystrom, Alicia Parrish, Marie Pellat, Martin Polacek, Alex Polozov, Reiner Pope, Siyuan Qiao, Emily Reif, Bryan Richter, Parker Riley, Alex Castro Ros, Aurko Roy, Brennan Saeta, Rajkumar Samuel, Renee Shelby, Ambrose Slone, Daniel Smilkov, David R. So, Daniel Sohn, Simon Tokumine, Dasha Valter, Vijay Vasudevan, Kiran Vodrahalli, Xuezhi Wang, Pidong Wang, Zirui Wang, Tao Wang, [John Wieting](https://en.wikipedia.org/wiki/John_Wieting "John Wieting"){.id-not}, [Yuhuai Wu](https://yuhuaiwu.github.io/ "Yuhuai (Tony) Wu’s Home Page"){.link-annotated-partial .id-not}, Kelvin Xu, Yunhan Xu, Linting Xue, Pengcheng Yin, Jiahui Yu, Qiao Zhang, Steven Zheng, Ce Zheng, Weikang Zhou, [Denny Zhou](https://dennyzhou.github.io/ "Denny Zhou’s Home Page"){.link-annotated-partial .id-not}, Slav Petrov, [Yonghui Wu](https://scholar.google.com/citations?user=bMWMlk4AAAAJ "Yonghui Wu"){.id-not}]{.collapse}]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2305.10403%2523google.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2305.10403#google"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 52. `https://www.cnbc.com/2023/05/16/googles-palm-2-uses-nearly-five-times-more-text-data-than-predecessor.html`: [`“Google’s Newest AI Model Uses Nearly 5× More Text Data for Training Than Its Predecessor”`{=html}](https://www.cnbc.com/2023/05/16/googles-palm-2-uses-nearly-five-times-more-text-data-than-predecessor.html "'Google’s newest AI model uses nearly 5× more text data for training than its predecessor', Elias 2023"){#elias-2023-1 .link-annotated}, [Jennifer Elias]{.author .cite-author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Fwww.cnbc.com%252F2023%252F05%252F16%252Fgoogles-palm-2-uses-nearly-five-times-more-text-data-than-predecessor.html.html "Directory-tag link-bibliography for link https://www.cnbc.com/2023/05/16/googles-palm-2-uses-nearly-five-times-more-text-data-than-predecessor.html"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 53. `https://arxiv.org/abs/2305.07759#microsoft`: [`“TinyStories: How Small Can Language Models Be and Still Speak Coherent English?”`{=html}](https://arxiv.org/abs/2305.07759#microsoft "'TinyStories: How Small Can Language Models Be and Still Speak Coherent English?', Eldan & Li 2023"){#eldan-li-2023 .link-annotated}, [[Ronen Eldan](https://en.wikipedia.org/wiki/Ronen_Eldan "Ronen Eldan"){.id-not}, Yuanzhi Li]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2305.07759%2523microsoft.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2305.07759#microsoft"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 54. `https://arxiv.org/abs/2305.05665#facebook`: [`“ImageBind: One Embedding Space To Bind Them All”`{=html}](https://arxiv.org/abs/2305.05665#facebook "'ImageBind: One Embedding Space To Bind Them All', Girdhar et al 2023"){#girdhar-et-al-2023 .link-annotated}, [Rohit Girdhar, Alaaeldin El-Nouby, [Zhuang Liu](https://liuzhuang13.github.io/ "Zhuang Liu’s Homepage"){.link-annotated-partial .id-not}[, Mannat Singh, Kalyan Vasudev Alwala, [Armand Joulin](https://scholar.google.com/citations?user=kRJkDakAAAAJ "Armand Joulin"){.id-not}, [Ishan Misra](https://imisra.github.io/ "Ishan Misra"){.id-not}]{.collapse}]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2305.05665%2523facebook.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2305.05665#facebook"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 55. `https://www.ft.com/content/f4f73815-6fc2-4016-bd97-4bace459e95e`: [`“Google’s DeepMind-Brain Merger: Tech Giant Regroups for AI Battle”`{=html}](https://www.ft.com/content/f4f73815-6fc2-4016-bd97-4bace459e95e "'Google’s DeepMind-Brain merger: tech giant regroups for AI battle', Murgia 2023"){#murgia-2023-1 .link-annotated}, [Madhumita Murgia]{.author .cite-author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Fwww.ft.com%252Fcontent%252Ff4f73815-6fc2-4016-bd97-4bace459e95e.html "Directory-tag link-bibliography for link https://www.ft.com/content/f4f73815-6fc2-4016-bd97-4bace459e95e"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 56. `https://arxiv.org/abs/2304.07193#facebook`: [`“DINOv2: Learning Robust Visual Features without Supervision”`{=html}](https://arxiv.org/abs/2304.07193#facebook "'DINOv2: Learning Robust Visual Features without Supervision', Oquab et al 2023"){#oquab-et-al-2023 .link-annotated}, [Maxime Oquab, Timothée Darcet, Théo Moutakanni[, Huy Vo, Marc Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, Mahmoud Assran, Nicolas Ballas, Wojciech Galuba, Russell Howes, Po-Yao Huang, Shang-Wen Li, [Ishan Misra](https://imisra.github.io/ "Ishan Misra"){.id-not}, Michael Rabbat, Vasu Sharma, Gabriel Synnaeve, Hu Xu, Hervé Jegou, Julien Mairal, Patrick Labatut, [Armand Joulin](https://scholar.google.com/citations?user=kRJkDakAAAAJ "Armand Joulin"){.id-not}, [Piotr Bojanowski](https://scholar.google.com/citations?user=lJ_oh2EAAAAJ "Piotr Bojanowski"){.id-not}]{.collapse}]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2304.07193%2523facebook.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2304.07193#facebook"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 57. `https://arxiv.org/abs/2303.15343#google`: [`“Sigmoid Loss for Language Image Pre-Training”`{=html}](https://arxiv.org/abs/2303.15343#google "'Sigmoid Loss for Language Image Pre-Training', Zhai et al 2023"){#zhai-et-al-2023 .link-annotated}, [[Xiaohua Zhai](https://sites.google.com/view/xzhai "Xiaohua Zhai (翟晓华)"){.link-annotated-partial .id-not}, Basil Mustafa, Alexander Kolesnikov, [Lucas Beyer](https://scholar.google.com/citations?user=p2gwhK4AAAAJ "Lucas Beyer"){.id-not}]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2303.15343%2523google.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2303.15343#google"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 58. `https://arxiv.org/abs/2304.02015#alibaba`: [`“How Well Do Large Language Models Perform in Arithmetic Tasks?”`{=html}](https://arxiv.org/abs/2304.02015#alibaba "'How well do Large Language Models perform in Arithmetic tasks?', Yuan et al 2023"){#yuan-et-al-2023-2 .link-annotated}, [Zheng Yuan, Hongyi Yuan, Chuanqi Tan[, [Wei Wang](https://web.cs.ucla.edu/~weiwang/ "Wei Wang’s Home Page"){.link-annotated-partial .id-not}, Songfang Huang]{.collapse}]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2304.02015%2523alibaba.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2304.02015#alibaba"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 59. `https://jameswphillips.substack.com/p/securing-liberal-democratic-control`: [`“Securing Liberal Democratic Control of AGI through UK Leadership”`{=html}](https://jameswphillips.substack.com/p/securing-liberal-democratic-control "'Securing Liberal Democratic Control of AGI through UK Leadership', Phillips 2023"){#phillips-2023 .link-annotated}, [James W. Phillips]{.author .cite-author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Fjameswphillips.substack.com%252Fp%252Fsecuring-liberal-democratic-control.html "Directory-tag link-bibliography for link https://jameswphillips.substack.com/p/securing-liberal-democratic-control"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 60. `https://arxiv.org/abs/2303.05511#adobe`: [`“GigaGAN: Scaling up GANs for Text-To-Image Synthesis”`{=html}](https://arxiv.org/abs/2303.05511#adobe "'GigaGAN: Scaling up GANs for Text-to-Image Synthesis', Kang et al 2023"){#kang-et-al-2023 .link-annotated}, [Minguk Kang, [Jun-Yan Zhu](https://www.cs.cmu.edu/~junyanz/ "Jun-Yan Zhu’s Homepage"){.id-not}, Richard Zhang[, Jaesik Park, Eli Shechtman, Sylvain Paris, Taesung Park]{.collapse}]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2303.05511%2523adobe.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2303.05511#adobe"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 61. `https://arxiv.org/abs/2302.05442#google`: [`“Scaling Vision Transformers to 22 Billion Parameters”`{=html}](https://arxiv.org/abs/2302.05442#google "'Scaling Vision Transformers to 22 Billion Parameters', Dehghani et al 2023"){#dehghani-et-al-2023 .link-annotated}, [[Mostafa Dehghani](https://scholar.google.com/citations?user=MiHOX3QAAAAJ "Mostafa Dehghani"){.id-not}, Josip Djolonga, Basil Mustafa[, Piotr Padlewski, Jonathan Heek, Justin Gilmer, Andreas Steiner, Mathilde Caron, [Robert Geirhos](https://scholar.google.com/citations?user=w3kGtMIAAAAJ "Robert Geirhos"){.id-not}, Ibrahim Alabdulmohsin, Rodolphe Jenatton, [Lucas Beyer](https://scholar.google.com/citations?user=p2gwhK4AAAAJ "Lucas Beyer"){.id-not}, [Michael Tschannen](https://mitscha.github.io/ "Michael Tschannen"){.link-annotated-partial .id-not}, Anurag Arnab, [Xiao Wang](https://en.wikipedia.org/wiki/Xiao_Wang "Xiao Wang"){.id-not}, Carlos Riquelme, Matthias Minderer, Joan Puigcerver, Utku Evci, Manoj Kumar, Sjoerd van Steenkiste, Gamaleldin F. Elsayed, Aravindh Mahendran, Fisher Yu, Avital Oliver, Fantine Huot, Jasmijn Bastings, Mark Patrick Collier, Alexey Gritsenko, Vighnesh Birodkar, Cristina Vasconcelos, [Yi Tay](https://www.yitay.net/ "Yi Tay"){.link-annotated-partial .id-not}, Thomas Mensink, Alexander Kolesnikov, Filip Pavetić, Dustin Tran, Thomas Kipf, Mario Lučić, [Xiaohua Zhai](https://sites.google.com/view/xzhai "Xiaohua Zhai (翟晓华)"){.link-annotated-partial .id-not}, Daniel Keysers, Jeremiah Harmsen, [Neil Houlsby](https://neilhoulsby.github.io/ "Neil Houlsby"){.link-annotated-partial .id-not}]{.collapse}]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2302.05442%2523google.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2302.05442#google"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 62. `https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4335945`: [`“Large Language Models As Fiduciaries: A Case Study Toward Robustly Communicating With Artificial Intelligence Through Legal Standards”`{=html}](https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4335945 "'Large Language Models as Fiduciaries: A Case Study Toward Robustly Communicating With Artificial Intelligence Through Legal Standards', Nay 2023"){#nay-2023 .link-annotated}, [John Nay]{.author .cite-author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Fpapers.ssrn.com%252Fsol3%252Fpapers.cfm%253Fabstract_id%253D4335945.html "Directory-tag link-bibliography for link https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4335945"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 63. `https://arxiv.org/abs/2301.09515#nvidia`: [`“StyleGAN-T: Unlocking the Power of GANs for Fast Large-Scale Text-To-Image Synthesis”`{=html}](https://arxiv.org/abs/2301.09515#nvidia "'StyleGAN-T: Unlocking the Power of GANs for Fast Large-Scale Text-to-Image Synthesis', Sauer et al 2023"){#sauer-et-al-2023-2 .link-annotated}, [Axel Sauer, [Tero Karras](https://research.nvidia.com/person/tero-karras "Tero Karras"){.id-not}, [Samuli Laine](https://users.aalto.fi/~laines9/ "Samuli Laine—Homepage"){.link-annotated-partial .id-not}[, Andreas Geiger, [Timo Aila](https://scholar.google.com/citations?user=e7abmgkAAAAJ "Timo Aila"){.id-not}]{.collapse}]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2301.09515%2523nvidia.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2301.09515#nvidia"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 64. `https://arxiv.org/abs/2301.07088#bytedance`: [`“MUG: Vision Learners Meet Web Image-Text Pairs”`{=html}](https://arxiv.org/abs/2301.07088#bytedance "'MUG: Vision Learners Meet Web Image-Text Pairs', Zhao et al 2023"){#zhao-et-al-2023-6 .link-annotated}, [Bingchen Zhao, Quan Cui, [Hao Wu](https://en.wikipedia.org/wiki/Hao_Wu_(biochemist) "Hao Wu (biochemist)"){.id-not}[, Osamu Yoshie, Cheng Yang]{.collapse}]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2301.07088%2523bytedance.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2301.07088#bytedance"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 65. `https://arxiv.org/abs/2301.04408`: [`“GPT-3 As Knowledge Worker: A Zero-Shot Evaluation of AI CPA Capabilities”`{=html}](https://arxiv.org/abs/2301.04408 "'GPT-3 as Knowledge Worker: A Zero-Shot Evaluation of AI CPA Capabilities', Bommarito et al 2023"){#bommarito-et-al-2023 .link-annotated}, [Jillian Bommarito, Michael Bommarito, Daniel Martin Katz, Jessica Katz]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2301.04408.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2301.04408"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 66. `https://arxiv.org/abs/2301.03728#facebook`: [`“Scaling Laws for Generative Mixed-Modal Language Models”`{=html}](https://arxiv.org/abs/2301.03728#facebook "'Scaling Laws for Generative Mixed-Modal Language Models', Aghajanyan et al 2023"){#aghajanyan-et-al-2023 .link-annotated}, [Armen Aghajanyan, Lili Yu, Alexis Conneau[, Wei-Ning Hsu, Karen Hambardzumyan, Susan Zhang, Stephen Roller, [Naman Goyal](https://namangoyal.com/ "Naman Goyal"){.link-annotated-partial .id-not}, [Omer Levy](https://scholar.google.com/citations?user=PZVd2h8AAAAJ "Omer Levy"){.id-not}, [Luke Zettlemoyer](https://scholar.google.com/citations?user=UjpbO6IAAAAJ "Luke Zettlemoyer"){.id-not}]{.collapse}]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2301.03728%2523facebook.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2301.03728#facebook"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 67. `https://arxiv.org/abs/2301.02111#microsoft`: [`“VALL-E: Neural Codec Language Models Are Zero-Shot Text to Speech Synthesizers”`{=html}](https://arxiv.org/abs/2301.02111#microsoft "'VALL-E: Neural Codec Language Models are Zero-Shot Text to Speech Synthesizers', Wang et al 2023"){#wang-et-al-2023-18 .link-annotated}, [Chengyi Wang, Sanyuan Chen, Yu Wu[, Ziqiang Zhang, Long Zhou, Shujie Liu, Zhuo Chen, Yanqing Liu, Huaming Wang, [Jinyu Li](https://en.wikipedia.org/wiki/Jinyu_Li "Jinyu Li"){.id-not}, Lei He, Sheng Zhao, [Furu Wei](https://scholar.google.com/citations?user=G-V1VpwAAAAJ "Furu Wei"){.id-not}]{.collapse}]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2301.02111%2523microsoft.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2301.02111#microsoft"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 68. `https://arxiv.org/abs/2212.14402`: [`“GPT-3 Takes the Bar Exam”`{=html}](https://arxiv.org/abs/2212.14402 "'GPT-3 Takes the Bar Exam', II & Katz 2022"){#ii-katz-2022 .link-annotated}, [Michael Bommarito II, Daniel Martin Katz]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2212.14402.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2212.14402"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 69. `https://arxiv.org/abs/2212.09741`: [`“One Embedder, Any Task: Instruction-Finetuned Text Embeddings (INSTRUCTOR)”`{=html}](https://arxiv.org/abs/2212.09741 "'One Embedder, Any Task: Instruction-Finetuned Text Embeddings (INSTRUCTOR)', Su et al 2022"){#su-et-al-2022-1 .link-annotated}, [Hongjin Su, Weijia Shi, [Jungo Kasai](https://jungokasai.github.io/ "About me—Jungo Kasai"){.link-annotated-partial .id-not}[, [Yizhong Wang](https://homes.cs.washington.edu/~yizhongw/ "Yizhong Wang—University of Washington"){.link-annotated-partial .id-not}, Yushi Hu, [Mari Ostendorf](https://en.wikipedia.org/wiki/Mari_Ostendorf "Mari Ostendorf"){.id-not}, Wen-tau Yih, [Noah Smith](https://en.wikipedia.org/wiki/Noah_Smith_(writer) "Noah Smith (writer)"){.id-not}, [Luke Zettlemoyer](https://scholar.google.com/citations?user=UjpbO6IAAAAJ "Luke Zettlemoyer"){.id-not}, Tao Yu]{.collapse}]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2212.09741.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2212.09741"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 70. `https://arxiv.org/abs/2212.07143`: [`“Reproducible Scaling Laws for Contrastive Language-Image Learning”`{=html}](https://arxiv.org/abs/2212.07143 "'Reproducible scaling laws for contrastive language-image learning', Cherti et al 2022"){#cherti-et-al-2022 .link-annotated}, [Mehdi Cherti, Romain Beaumont, [Ross Wightman](https://en.wikipedia.org/wiki/Ross_Wightman "Ross Wightman"){.id-not}[, [Mitchell Wortsman](https://mitchellnw.github.io/ "Mitchell Wortsman"){.id-not}, [Gabriel Ilharco](https://gabrielilharco.com/ "Gabriel Ilharco"){.id-not}, Cade Gordon, Christoph Schuhmann, [Ludwig Schmidt](https://en.wikipedia.org/wiki/Ludwig_Schmidt "Ludwig Schmidt"){.id-not}, Jenia Jitsev]{.collapse}]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2212.07143.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2212.07143"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 71. `https://arxiv.org/abs/2212.04979#google`: [`“VideoCoCa: Video-Text Modeling With Zero-Shot Transfer from Contrastive Captioners”`{=html}](https://arxiv.org/abs/2212.04979#google "'VideoCoCa: Video-Text Modeling with Zero-Shot Transfer from Contrastive Captioners', Yan et al 2022"){#yan-et-al-2022-2 .link-annotated}, [Shen Yan, [Tao Zhu](https://en.wikipedia.org/wiki/Tao_Zhu "Tao Zhu"){.id-not}, Zirui Wang[, [Yuan Cao](https://en.wikipedia.org/wiki/Yuan_Cao "Yuan Cao"){.id-not}, [Mi Zhang](https://en.wikipedia.org/wiki/Mi_Zhang "Mi Zhang"){.id-not}, [Soham Ghosh](https://en.wikipedia.org/wiki/Soham_Ghosh "Soham Ghosh"){.id-not}, [Yonghui Wu](https://scholar.google.com/citations?user=bMWMlk4AAAAJ "Yonghui Wu"){.id-not}, Jiahui Yu]{.collapse}]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2212.04979%2523google.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2212.04979#google"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 72. `https://arxiv.org/abs/2212.05051`: [`“VindLU: A Recipe for Effective Video-And-Language Pretraining”`{=html}](https://arxiv.org/abs/2212.05051 "'VindLU: A Recipe for Effective Video-and-Language Pretraining', Cheng et al 2022"){#cheng-et-al-2022-1 .link-annotated}, [Feng Cheng, Xizi Wang, Jie Lei[, David Crandall, [Mohit Bansal](https://www.cs.unc.edu/~mbansal/ "Mohit Bansal"){.link-annotated-partial .id-not}, Gedas Bertasius]{.collapse}]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2212.05051.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2212.05051"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 73. `https://arxiv.org/abs/2212.04356#openai`: [`“Whisper: Robust Speech Recognition via Large-Scale Weak Supervision”`{=html}](https://arxiv.org/abs/2212.04356#openai "'Whisper: Robust Speech Recognition via Large-Scale Weak Supervision', Radford et al 2022"){#radford-et-al-2022 .link-annotated}, [[Alec Radford](https://scholar.google.com/citations?user=dOad5HoAAAAJ "Alec Radford"){.id-not}, [Jong Wook Kim](https://jongwook.kim/ "Jong Wook Kim"){.link-annotated-partial .id-not}, Tao Xu[, [Greg Brockman](https://en.wikipedia.org/wiki/Greg_Brockman "Greg Brockman"){.link-annotated-partial .id-not}, Christine McLeavey, [Ilya Sutskever](https://en.wikipedia.org/wiki/Ilya_Sutskever "Ilya Sutskever"){.link-annotated-partial .id-not}]{.collapse}]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2212.04356%2523openai.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2212.04356#openai"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 74. `https://arxiv.org/abs/2211.09085#facebook`: [`“Galactica: A Large Language Model for Science”`{=html}](https://arxiv.org/abs/2211.09085#facebook "'Galactica: A Large Language Model for Science', Taylor et al 2022"){#taylor-et-al-2022 .link-annotated}, [[Ross Taylor](https://en.wikipedia.org/wiki/Ross_Taylor "Ross Taylor"){.id-not}, Marcin Kardas, Guillem Cucurull[, Thomas Scialom, Anthony Hartshorn, Elvis Saravia, Andrew Poulton, Viktor Kerkez, Robert Stojnic]{.collapse}]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2211.09085%2523facebook.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2211.09085#facebook"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 75. `https://arxiv.org/abs/2211.08411`: [`“Large Language Models Struggle to Learn Long-Tail Knowledge”`{=html}](https://arxiv.org/abs/2211.08411 "'Large Language Models Struggle to Learn Long-Tail Knowledge', Kandpal et al 2022"){#kandpal-et-al-2022 .link-annotated}, [Nikhil Kandpal, Haikang Deng, [Adam Roberts](https://scholar.google.com/citations?user=U5UpKq8AAAAJ "Adam Roberts"){.id-not}[, [Eric Wallace](https://en.wikipedia.org/wiki/Eric_Wallace "Eric Wallace"){.id-not}, [Colin Raffel](https://colinraffel.com/ "Colin Raffel"){.link-annotated-partial .id-not}]{.collapse}]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2211.08411.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2211.08411"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 76. `https://arxiv.org/abs/2211.07636#baai`: [`“EVA: Exploring the Limits of Masked Visual Representation Learning at Scale”`{=html}](https://arxiv.org/abs/2211.07636#baai "'EVA: Exploring the Limits of Masked Visual Representation Learning at Scale', Fang et al 2022"){#fang-et-al-2022-1 .link-annotated}, [Yuxin Fang, [Wen Wang](https://en.wikipedia.org/wiki/Wen_Wang "Wen Wang"){.id-not}, Binhui Xie[, Quan Sun, Ledell Wu, Xinggang Wang, Tiejun Huang, Xinlong Wang, Yue Cao]{.collapse}]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2211.07636%2523baai.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2211.07636#baai"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 77. `https://arxiv.org/abs/2211.00241`: [`“Adversarial Policies Beat Superhuman Go AIs”`{=html}](https://arxiv.org/abs/2211.00241 "'Adversarial Policies Beat Superhuman Go AIs', Wang et al 2022"){#wang-et-al-2022-08 .link-annotated}, [Tony T. Wang, Adam Gleave, Tom Tseng[, Kellin Pelrine, Nora Belrose, Joseph Miller, Michael D. Dennis, Yawen Duan, Viktor Pogrebniak, [Sergey Levine](https://scholar.google.com/citations?user=8R35rCwAAAAJ "Sergey Levine"){.id-not}, Stuart Russell]{.collapse}]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2211.00241.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2211.00241"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 78. `https://www.youtube.com/watch?v=Q-TJFyUoenc&t=2444s`: [`“Increments Podcast: #45—4 Central Fallacies of AI Research (With Melanie Mitchell)”`{=html}](https://www.youtube.com/watch?v=Q-TJFyUoenc&t=2444s "'Increments Podcast: #45—4 Central Fallacies of AI Research (with Melanie Mitchell)', Mitchell & Chugg 2022"){#mitchell-chugg-2022 .link-annotated}, [[Melanie Mitchell](https://en.wikipedia.org/wiki/Melanie_Mitchell "Melanie Mitchell"){.id-not}, Benny Chugg]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Fwww.youtube.com%252Fwatch%253Fv%253DQ-TJFyUoenc%2526t%253D2444s.html "Directory-tag link-bibliography for link https://www.youtube.com/watch?v=Q-TJFyUoenc&t=2444s"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 79. `https://arxiv.org/abs/2210.16859`: [`“A Solvable Model of Neural Scaling Laws”`{=html}](https://arxiv.org/abs/2210.16859 "'A Solvable Model of Neural Scaling Laws', Maloney et al 2022"){#maloney-et-al-2022 .link-annotated}, [Alexander Maloney, Daniel A. Roberts, James Sully]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2210.16859.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2210.16859"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 80. `https://arxiv.org/abs/2210.13673#nvidia`: [`“Evaluating Parameter Efficient Learning for Generation”`{=html}](https://arxiv.org/abs/2210.13673#nvidia "'Evaluating Parameter Efficient Learning for Generation', Xu et al 2022"){#xu-et-al-2022-2 .link-annotated}, [Peng Xu, Mostofa Patwary, Shrimai Prabhumoye[, Virginia Adams, Ryan J. Prenger, Wei Ping, Nayeon Lee, [Mohammad Shoeybi](https://scholar.google.com/citations?user=62ElavIAAAAJ "Mohammad Shoeybi"){.id-not}, [Bryan Catanzaro](https://scholar.google.com/citations?user=UZ6kI2AAAAAJ "Bryan Catanzaro"){.id-not}]{.collapse}]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2210.13673%2523nvidia.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2210.13673#nvidia"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 81. `https://arxiv.org/abs/2210.11416#google`: [`“FLAN: Scaling Instruction-Finetuned Language Models”`{=html}](https://arxiv.org/abs/2210.11416#google "'FLAN: Scaling Instruction-Finetuned Language Models', Chung et al 2022"){#chung-et-al-2022 .link-annotated}, [Hyung Won Chung, Le Hou, Shayne Longpre[, [Barret Zoph](https://barretzoph.github.io/ "Barret Zoph"){.link-annotated-partial .id-not}, [Yi Tay](https://www.yitay.net/ "Yi Tay"){.link-annotated-partial .id-not}, [William Fedus](https://scholar.google.com/citations?user=-ZfwQOkAAAAJ "William Fedus"){.id-not}, Yunxuan Li, Xuezhi Wang, [Mostafa Dehghani](https://scholar.google.com/citations?user=MiHOX3QAAAAJ "Mostafa Dehghani"){.id-not}, Siddhartha Brahma, [Albert Webson](https://representation.ai/ "Albert Webson"){.link-annotated-partial .id-not}, [Shixiang Shane Gu](https://sites.google.com/view/gugurus/home){.id-not}, Zhuyun Dai, Mirac Suzgun, Xinyun Chen, Aakanksha Chowdhery, Alex Castro-Ros, Marie Pellat, Kevin Robinson, Dasha Valter, [Sharan Narang](https://scholar.google.com/citations?user=CWOixywAAAAJ "Sharan Narang"){.link-modified-recently .id-not}, Gaurav Mishra, Adams Yu, Vincent Zhao, [Yanping Huang](https://scholar.google.com/citations?user=uEtBQScAAAAJ "Yanping Huang"){.id-not}, Andrew Dai, Hongkun Yu, Slav Petrov, [Ed H. Chi](https://en.wikipedia.org/wiki/Ed_H._Chi "Ed H. Chi"){.id-not}, [Jeff Dean](https://en.wikipedia.org/wiki/Jeff_Dean "Jeff Dean"){.link-annotated-partial .id-not}, Jacob Devlin, [Adam Roberts](https://scholar.google.com/citations?user=U5UpKq8AAAAJ "Adam Roberts"){.id-not}, [Denny Zhou](https://dennyzhou.github.io/ "Denny Zhou’s Home Page"){.link-annotated-partial .id-not}, [Quoc V. Le](https://en.wikipedia.org/wiki/Quoc_V._Le "Quoc V. Le"){.id-not}, [Jason Wei](https://www.jasonwei.net/ "Jason Wei"){.id-not}]{.collapse}]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2210.11416%2523google.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2210.11416#google"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 82. `https://arxiv.org/abs/2210.10341#microsoft`: [`“BioGPT: Generative Pre-Trained Transformer for Biomedical Text Generation and Mining”`{=html}](https://arxiv.org/abs/2210.10341#microsoft "'BioGPT: Generative Pre-trained Transformer for Biomedical Text Generation and Mining', Luo et al 2022"){#luo-et-al-2022-2 .link-annotated}, [Renqian Luo, Liai Sun, Yingce Xia[, [Tao Qin](https://en.wikipedia.org/wiki/Tao_Qin "Tao Qin"){.id-not}, Sheng Zhang, Hoifung Poon, [Tie-Yan Liu](https://en.wikipedia.org/wiki/Tie-Yan_Liu "Tie-Yan Liu"){.id-not}]{.collapse}]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2210.10341%2523microsoft.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2210.10341#microsoft"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 83. `https://arxiv.org/abs/2210.06423#microsoft`: [`“Foundation Transformers”`{=html}](https://arxiv.org/abs/2210.06423#microsoft "'Foundation Transformers', Wang et al 2022"){#wang-et-al-2022-10 .link-annotated}, [Hongyu Wang, Shuming Ma, Shaohan Huang[, [Li Dong](https://scholar.google.com/citations?user=wEfQgPgAAAAJ "Li Dong"){.id-not}, Wenhui Wang, Zhiliang Peng, Yu Wu, Payal Bajaj, Saksham Singhal, Alon Benhaim, Barun Patra, Zhun Liu, Vishrav Chaudhary, Xia Song, [Furu Wei](https://scholar.google.com/citations?user=G-V1VpwAAAAJ "Furu Wei"){.id-not}]{.collapse}]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2210.06423%2523microsoft.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2210.06423#microsoft"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 84. `https://arxiv.org/abs/2210.03350#allen`: [`“Self-Ask: Measuring and Narrowing the Compositionality Gap in Language Models (Bamboogle)”`{=html}](https://arxiv.org/abs/2210.03350#allen "'Self-Ask: Measuring and Narrowing the Compositionality Gap in Language Models (Bamboogle)', Press et al 2022"){#press-et-al-2022 .link-annotated}, [Ofir Press, Muru Zhang, Sewon Min[, [Ludwig Schmidt](https://en.wikipedia.org/wiki/Ludwig_Schmidt "Ludwig Schmidt"){.id-not}, [Noah A. Smith](https://nasmith.github.io/ "Noah A. Smith"){.link-annotated-partial .id-not}, [Mike Lewis](https://scholar.google.com/citations?user=SnQnQicAAAAJ "Mike Lewis"){.id-not}]{.collapse}]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2210.03350%2523allen.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2210.03350#allen"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 85. `https://arxiv.org/abs/2210.02414#zhipu`: [`“GLM-130B: An Open Bilingual Pre-Trained Model”`{=html}](https://arxiv.org/abs/2210.02414#zhipu "'GLM-130B: An Open Bilingual Pre-trained Model', Zeng et al 2022"){#zeng-et-al-2022-1 .link-annotated}, [Aohan Zeng, Xiao Liu, Zhengxiao Du[, Zihan Wang, Hanyu Lai, [Ming Ding](https://scholar.google.com/citations?user=Va50YzkAAAAJ "Ming Ding"){.id-not}, Zhuoyi Yang, [Yifan Xu](https://en.wikipedia.org/wiki/Yifan_Xu "Yifan Xu"){.id-not}, Wendi Zheng, Xiao Xia, Weng Lam Tam, Zixuan Ma, Yufei Xue, Jidong Zhai, Wenguang Chen, [Peng Zhang](https://sites.google.com/site/pengzhang27182/ "Peng Zhang’s homepage"){.id-not}, Yuxiao Dong, [Jie Tang](https://en.wikipedia.org/wiki/Jie_Tang "Jie Tang"){.id-not}]{.collapse}]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2210.02414%2523zhipu.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2210.02414#zhipu"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 86. `https://arxiv.org/abs/2210.02441`: [`“Ask Me Anything (AMA): A Simple Strategy for Prompting Language Models”`{=html}](https://arxiv.org/abs/2210.02441 "'Ask Me Anything (AMA): A simple strategy for prompting language models', Arora et al 2022"){#arora-et-al-2022-2 .link-annotated}, [Simran Arora, Avanika Narayan, Mayee F. Chen[, Laurel Orr, Neel Guha, Kush Bhatia, Ines Chami, Frederic Sala, [Christopher Ré](https://en.wikipedia.org/wiki/Christopher_R%C3%A9 "Christopher Ré"){.id-not}]{.collapse}]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2210.02441.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2210.02441"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 87. `https://arxiv.org/abs/2208.05516`: [`“Quality Not Quantity: On the Interaction between Dataset Design and Robustness of CLIP”`{=html}](https://arxiv.org/abs/2208.05516 "'Quality Not Quantity: On the Interaction between Dataset Design and Robustness of CLIP', Nguyen et al 2022"){#nguyen-et-al-2022 .link-annotated}, [Thao Nguyen, [Gabriel Ilharco](https://gabrielilharco.com/ "Gabriel Ilharco"){.id-not}, [Mitchell Wortsman](https://mitchellnw.github.io/ "Mitchell Wortsman"){.id-not}[, Sewoong Oh, [Ludwig Schmidt](https://en.wikipedia.org/wiki/Ludwig_Schmidt "Ludwig Schmidt"){.id-not}]{.collapse}]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2208.05516.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2208.05516"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 88. `https://arxiv.org/abs/2207.06991`: [`“PIXEL: Language Modeling With Pixels”`{=html}](https://arxiv.org/abs/2207.06991 "'PIXEL: Language Modeling with Pixels', Rust et al 2022"){#rust-et-al-2022 .link-annotated}, [Phillip Rust, Jonas F. Lotz, Emanuele Bugliarello[, Elizabeth Salesky, Miryam de Lhoneux, [Desmond Elliott](https://en.wikipedia.org/wiki/Desmond_Elliott "Desmond Elliott"){.id-not}]{.collapse}]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2207.06991.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2207.06991"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 89. `https://arxiv.org/abs/2207.05221#anthropic`: [`“Language Models (Mostly) Know What They Know”`{=html}](https://arxiv.org/abs/2207.05221#anthropic "'Language Models (Mostly) Know What They Know', Kadavath et al 2022"){#kadavath-et-al-2022 .link-annotated}, [[Saurav Kadavath](https://scholar.google.com/citations?user=Z2Uo_FcAAAAJ "Saurav Kadavath"){.id-not}, Tom Conerly, [Amanda Askell](https://askell.io/ "About Me"){.link-annotated-partial .id-not}[, [Tom Henighan](https://www.tomhenighan.com/ "Tom Henighan"){.link-annotated-partial .id-not}, Dawn Drain, [Ethan Perez](https://ethanperez.net/ "Ethan Perez"){.link-annotated-partial .id-not}, Nicholas Schiefer, Zac Hatfield Dodds, Nova DasSarma, Eli Tran-Johnson, [Scott Johnston](https://en.wikipedia.org/wiki/Scott_Johnston "Scott Johnston"){.id-not}, Sheer El-Showk, [Andy L. Jones](https://andyljones.com/ "andy jones"){.id-not}, [Nelson Elhage](https://nelhage.com/){.link-annotated-partial .id-not}, Tristan Hume, Anna Chen, [Yuntao Bai](https://scholar.google.com/citations?user=r7GUEVsAAAAJ "Yuntao Bai"){.id-not}, [Samuel R. Bowman](https://cims.nyu.edu/~sbowman/ "Sam Bowman"){.link-annotated-partial .id-not}, Stanislav Fort, [Deep Ganguli](https://dganguli.github.io/pweb/ "Deep Ganguli—Research Scientist at Anthropic"){.link-annotated-partial .id-not}, [Danny Hernandez](https://en.wikipedia.org/wiki/Danny_Hernandez "Danny Hernandez"){.id-not}, Josh Jacobson, [Jackson Kernion](https://jacksonkernion.com/){.link-annotated-partial .id-not}, Shauna Kravec, Liane Lovitt, Kamal Ndousse, Catherine Olsson, Sam Ringer, [Dario Amodei](https://en.wikipedia.org/wiki/Dario_Amodei "Dario Amodei"){.link-annotated-partial .id-not}, [Tom B. Brown](https://scholar.google.com/citations?user=RLvsC94AAAAJ "Tom B. Brown"){.id-not}, [Jack Clark](https://jack-clark.net/about/){.link-annotated-partial .id-not}, [Nicholas Joseph](https://scholar.google.com/citations?user=Z5NeEv4AAAAJ "Nicholas Joseph"){.id-not}, [Ben Mann](https://scholar.google.com/citations?user=McBoXK0AAAAJ "Benjamin Mann"){.id-not}, [Sam McCandlish](https://scholar.google.com/citations?user=gHp0pu4AAAAJ "Sam McCandlish"){.id-not}, Chris Olah, [Jared Kaplan](https://sites.krieger.jhu.edu/jared-kaplan/ "Jared Kaplan"){.link-annotated-partial .id-not}]{.collapse}]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2207.05221%2523anthropic.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2207.05221#anthropic"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 90. `https://arxiv.org/abs/2206.15472`: [`“On-Device Training Under 256KB Memory”`{=html}](https://arxiv.org/abs/2206.15472 "'On-Device Training Under 256KB Memory', Lin et al 2022"){#lin-et-al-2022-smolml .link-annotated}, [Ji Lin, Ligeng Zhu, Wei-Ming Chen[, Wei-Chen Wang, Chuang Gan, [Song Han](https://hanlab.mit.edu/songhan "Song Han—Associate Professor, MIT EECS"){.link-annotated-partial .id-not}]{.collapse}]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2206.15472.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2206.15472"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 91. `https://arxiv.org/abs/2206.04658#nvidia`: [`“BigVGAN: A Universal Neural Vocoder With Large-Scale Training”`{=html}](https://arxiv.org/abs/2206.04658#nvidia "'BigVGAN: A Universal Neural Vocoder with Large-Scale Training', Lee et al 2022"){#lee-et-al-2022-07 .link-annotated}, [Sang-gil Lee, Wei Ping, Boris Ginsburg[, [Bryan Catanzaro](https://scholar.google.com/citations?user=UZ6kI2AAAAAJ "Bryan Catanzaro"){.id-not}, Sungroh Yoon]{.collapse}]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2206.04658%2523nvidia.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2206.04658#nvidia"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 92. `https://arxiv.org/abs/2206.01685`: [`“Toward a Realistic Model of Speech Processing in the Brain With Self-Supervised Learning”`{=html}](https://arxiv.org/abs/2206.01685 "'Toward a realistic model of speech processing in the brain with self-supervised learning', Millet et al 2022"){#millet-et-al-2022 .link-annotated}, [Juliette Millet, Charlotte Caucheteux, Pierre Orhan[, Yves Boubenec, Alexandre Gramfort, Ewan Dunbar, Christophe Pallier, Jean-Remi King]{.collapse}]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2206.01685.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2206.01685"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 93. `https://arxiv.org/abs/2205.14204#google`: [`“M3AE: Multimodal Masked Autoencoders Learn Transferable Representations”`{=html}](https://arxiv.org/abs/2205.14204#google "'M3AE: Multimodal Masked Autoencoders Learn Transferable Representations', Geng et al 2022"){#geng-et-al-2022 .link-annotated}, [Xinyang Geng, Hao Liu, [Lisa Lee](https://en.wikipedia.org/wiki/Lisa_Lee "Lisa Lee"){.id-not}[, Dale Schuurams, [Sergey Levine](https://scholar.google.com/citations?user=8R35rCwAAAAJ "Sergey Levine"){.id-not}, [Pieter Abbeel](https://en.wikipedia.org/wiki/Pieter_Abbeel "Pieter Abbeel"){.id-not}]{.collapse}]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2205.14204%2523google.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2205.14204#google"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 94. `https://arxiv.org/abs/2205.10625#google`: [`“Least-To-Most Prompting Enables Complex Reasoning in Large Language Models”`{=html}](https://arxiv.org/abs/2205.10625#google "'Least-to-Most Prompting Enables Complex Reasoning in Large Language Models', Zhou et al 2022"){#zhou-et-al-2022-1 .link-annotated}, [[Denny Zhou](https://dennyzhou.github.io/ "Denny Zhou’s Home Page"){.link-annotated-partial .id-not}, Nathanael Schärli, Le Hou[, [Jason Wei](https://www.jasonwei.net/ "Jason Wei"){.id-not}, Nathan Scales, Xuezhi Wang, Dale Schuurmans, Olivier Bousquet, [Quoc V. Le](https://en.wikipedia.org/wiki/Quoc_V._Le "Quoc V. Le"){.id-not}, [Ed Chi](https://en.wikipedia.org/wiki/Ed_Chi "Ed Chi"){.id-not}]{.collapse}]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2205.10625%2523google.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2205.10625#google"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 95. `https://arxiv.org/abs/2205.09073#google`: [`“Dialog Inpainting: Turning Documents into Dialogues”`{=html}](https://arxiv.org/abs/2205.09073#google "'Dialog Inpainting: Turning Documents into Dialogues', Dai et al 2022"){#dai-et-al-2022-2 .link-annotated}, [Zhuyun Dai, Arun Tejasvi Chaganty, Vincent Zhao[, Aida Amini, Qazi Mamunur Rashid, Mike Green, Kelvin Guu]{.collapse}]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2205.09073%2523google.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2205.09073#google"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 96. `https://arxiv.org/abs/2205.05131#google`: [`“UL2: Unifying Language Learning Paradigms”`{=html}](https://arxiv.org/abs/2205.05131#google "'UL2: Unifying Language Learning Paradigms', Tay et al 2022"){#tay-et-al-2022-ul2 .link-annotated}, [[Yi Tay](https://www.yitay.net/ "Yi Tay"){.link-annotated-partial .id-not}, [Mostafa Dehghani](https://scholar.google.com/citations?user=MiHOX3QAAAAJ "Mostafa Dehghani"){.id-not}, Vinh Q. Tran[, Xavier Garcia, Dara Bahri, Tal Schuster, Huaixiu Steven Zheng, [Neil Houlsby](https://neilhoulsby.github.io/ "Neil Houlsby"){.link-annotated-partial .id-not}, [Donald Metzler](https://www.don-metzler.net/ "About—Donald Metzler"){.link-annotated-partial .id-not}]{.collapse}]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2205.05131%2523google.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2205.05131#google"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 97. `https://arxiv.org/abs/2205.03983#google`: [`“Building Machine Translation Systems for the Next Thousand Languages”`{=html}](https://arxiv.org/abs/2205.03983#google "'Building Machine Translation Systems for the Next Thousand Languages', Bapna et al 2022"){#bapna-et-al-2022 .link-annotated}, [Ankur Bapna, Isaac Caswell, Julia Kreutzer[, [Orhan Firat](https://orhanfirat.com/ "Orhan Fırat"){.link-annotated-partial .id-not}, Daan van Esch, Aditya Siddhant, Mengmeng Niu, Pallavi Baljekar, Xavier Garcia, Wolfgang Macherey, Theresa Breiner, Vera Axelrod, Jason Riesa, [Yuan Cao](https://en.wikipedia.org/wiki/Yuan_Cao "Yuan Cao"){.id-not}, Mia Xu Chen, Klaus Macherey, Maxim Krikun, Pidong Wang, Alexander Gutkin, Apurva Shah, [Yanping Huang](https://scholar.google.com/citations?user=uEtBQScAAAAJ "Yanping Huang"){.id-not}, [Zhifeng Chen](https://scholar.google.com/citations?user=EemUE4gAAAAJ "Zhifeng Chen"){.id-not}, [Yonghui Wu](https://scholar.google.com/citations?user=bMWMlk4AAAAJ "Yonghui Wu"){.id-not}, Macduff Hughes]{.collapse}]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2205.03983%2523google.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2205.03983#google"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 98. `https://arxiv.org/abs/2205.04596#google`: [`“When Does Dough Become a Bagel? Analyzing the Remaining Mistakes on ImageNet”`{=html}](https://arxiv.org/abs/2205.04596#google "'When does dough become a bagel? Analyzing the remaining mistakes on ImageNet', Vasudevan et al 2022"){#vasudevan-et-al-2022 .link-annotated}, [Vijay Vasudevan, Benjamin Caine, Raphael Gontijo-Lopes[, Sara Fridovich-Keil, Rebecca Roelofs]{.collapse}]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2205.04596%2523google.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2205.04596#google"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 99. `https://arxiv.org/abs/2205.01917#google`: [`“CoCa: Contrastive Captioners Are Image-Text Foundation Models”`{=html}](https://arxiv.org/abs/2205.01917#google "'CoCa: Contrastive Captioners are Image-Text Foundation Models', Yu et al 2022"){#yu-et-al-2022-3 .link-annotated}, [Jiahui Yu, Zirui Wang, Vijay Vasudevan[, Legg Yeung, Mojtaba Seyedhosseini, [Yonghui Wu](https://scholar.google.com/citations?user=bMWMlk4AAAAJ "Yonghui Wu"){.id-not}]{.collapse}]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2205.01917%2523google.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2205.01917#google"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 100. `https://arxiv.org/abs/2205.01397`: [`“Data Determines Distributional Robustness in Contrastive Language Image Pre-Training (CLIP)”`{=html}](https://arxiv.org/abs/2205.01397 "'Data Determines Distributional Robustness in Contrastive Language Image Pre-training (CLIP)', Fang et al 2022"){#fang-et-al-2022-4 .link-annotated}, [Alex Fang, [Gabriel Ilharco](https://gabrielilharco.com/ "Gabriel Ilharco"){.id-not}, [Mitchell Wortsman](https://mitchellnw.github.io/ "Mitchell Wortsman"){.id-not}[, Yuhao Wan, Vaishaal Shankar, Achal Dave, [Ludwig Schmidt](https://en.wikipedia.org/wiki/Ludwig_Schmidt "Ludwig Schmidt"){.id-not}]{.collapse}]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2205.01397.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2205.01397"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 101. `https://arxiv.org/abs/2204.14198#deepmind`: [`“Flamingo: a Visual Language Model for Few-Shot Learning”`{=html}](https://arxiv.org/abs/2204.14198#deepmind "'Flamingo: a Visual Language Model for Few-Shot Learning', Alayrac et al 2022"){#alayrac-et-al-2022 .link-annotated}, [Jean-Baptiste Alayrac, [Jeff Donahue](https://jeffdonahue.com/ "Jeff Donahue"){.id-not}, Pauline Luc[, Antoine Miech, Iain Barr, Yana Hasson, Karel Lenc, Arthur Mensch, Katie Millican, Malcolm Reynolds, Roman Ring, Eliza Rutherford, Serkan Cabi, Tengda Han, Zhitao Gong, Sina Samangooei, Marianne Monteiro, Jacob Menick, Sebastian Borgeaud, [Andrew Brock](https://en.wikipedia.org/wiki/Andrew_Brock "Andrew Brock"){.id-not}, Aida Nematzadeh, Sah, Sharifzadeh, Mikolaj Binkowski, Ricardo Barreira, [Oriol Vinyals](https://en.wikipedia.org/wiki/Oriol_Vinyals "Oriol Vinyals"){.id-not}, [Andrew Zisserman](https://en.wikipedia.org/wiki/Andrew_Zisserman "Andrew Zisserman"){.id-not}, [Karen Simonyan](https://scholar.google.com/citations?user=L7lMQkQAAAAJ "Karen Simonyan"){.id-not}]{.collapse}]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2204.14198%2523deepmind.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2204.14198#deepmind"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 102. `https://arxiv.org/abs/2204.10149`: [`“WebFace260M: A Benchmark for Million-Scale Deep Face Recognition”`{=html}](https://arxiv.org/abs/2204.10149 "'WebFace260M: A Benchmark for Million-Scale Deep Face Recognition', Zhu et al 2022"){#zhu-et-al-2022-5 .link-annotated}, [[Zheng Zhu](https://en.wikipedia.org/wiki/Zheng_Zhu "Zheng Zhu"){.id-not}, Guan Huang, Jiankang Deng[, Yun Ye, Junjie Huang, Xinze Chen, Jiagang Zhu, Tian Yang, Dalong Du, Jiwen Lu, Jie Zhou]{.collapse}]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2204.10149.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2204.10149"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 103. `https://arxiv.org/abs/2204.07705`: [`“Tk-Instruct: Benchmarking Generalization via In-Context Instructions on 1,600+ Language Tasks”`{=html}](https://arxiv.org/abs/2204.07705 "'Tk-Instruct: Benchmarking Generalization via In-Context Instructions on 1,600+ Language Tasks', Wang et al 2022"){#wang-et-al-2022-16 .link-annotated}, [[Yizhong Wang](https://homes.cs.washington.edu/~yizhongw/ "Yizhong Wang—University of Washington"){.link-annotated-partial .id-not}, [Swaroop Mishra](https://swarooprm.github.io/ "Swaroop Mishra"){.link-annotated-partial .id-not}, Pegah Alipoormolabashi[, Yeganeh Kordi, Amirreza Mirzaei, Anjana Arunkumar, Arjun Ashok, Arut Selvan Dhanasekaran, Atharva Naik, David Stap, Eshaan Pathak, Giannis Karamanolakis, Haizhi Gary Lai, Ishan Purohit, Ishani Mondal, Jacob Anderson, Kirby Kuznia, Krima Doshi, Maitreya Patel, Kuntal Kumar Pal, Mehrad Moradshahi, Mihir Parmar, Mirali Purohit, Neeraj Varshney, Phani Rohitha Kaza, Pulkit Verma, Ravsehaj Singh Puri, Rushang Karia, Shailaja Keyur Sampat, Savan Doshi, Siddhartha Mishra, Sujan Reddy, Sumanta Patro, Tanay Dixit, Xudong Shen, Chitta Baral, [Yejin Choi](https://en.wikipedia.org/wiki/Yejin_Choi "Yejin Choi"){.id-not}, [Noah A. Smith](https://nasmith.github.io/ "Noah A. Smith"){.link-annotated-partial .id-not}, [Hannaneh Hajishirzi](https://homes.cs.washington.edu/~hannaneh/ "Hannaneh Hajishirzi—University of Washington"){.link-annotated-partial .id-not}, [Daniel Khashabi](https://danielkhashabi.com/ "Daniel Khashabi"){.link-annotated-partial .id-not}]{.collapse}]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2204.07705.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2204.07705"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 104. `https://www.lesswrong.com/posts/SbAgRYo8tkHwhd9Qx/deepmind-the-podcast-excerpts-on-agi`: [`“DeepMind: The Podcast—Excerpts on AGI”`{=html}](https://www.lesswrong.com/posts/SbAgRYo8tkHwhd9Qx/deepmind-the-podcast-excerpts-on-agi "'DeepMind: The Podcast—Excerpts on AGI', Kiely 2022"){#kiely-2022 .link-annotated}, [William Kiely]{.author .cite-author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Fwww.lesswrong.com%252Fposts%252FSbAgRYo8tkHwhd9Qx%252Fdeepmind-the-podcast-excerpts-on-agi.html "Directory-tag link-bibliography for link https://www.lesswrong.com/posts/SbAgRYo8tkHwhd9Qx/deepmind-the-podcast-excerpts-on-agi"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 105. `https://arxiv.org/abs/2203.15556#deepmind`: [`“Chinchilla: Training Compute-Optimal Large Language Models”`{=html}](https://arxiv.org/abs/2203.15556#deepmind "'Chinchilla: Training Compute-Optimal Large Language Models', Hoffmann et al 2022"){#hoffmann-et-al-2022 .link-annotated}, [Jordan Hoffmann, Sebastian Borgeaud, Arthur Mensch[, Elena Buchatskaya, Trevor Cai, Eliza Rutherford, Diego de Las Casas, Lisa Anne Hendricks, Johannes Welbl, Aidan Clark, Tom Hennigan, Eric Noland, Katie Millican, George van den Driessche, Bogdan Damoc, Aurelia Guy, [Simon Osindero](https://scholar.google.com/citations?user=Jq8ZS5kAAAAJ "Simon Osindero"){.id-not}, [Karen Simonyan](https://scholar.google.com/citations?user=L7lMQkQAAAAJ "Karen Simonyan"){.id-not}, Erich Elsen, Jack W. Rae, [Oriol Vinyals](https://en.wikipedia.org/wiki/Oriol_Vinyals "Oriol Vinyals"){.id-not}, [Laurent Sifre](https://scholar.google.com/citations?user=0kVh58wAAAAJ "Laurent Sifre"){.id-not}]{.collapse}]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2203.15556%2523deepmind.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2203.15556#deepmind"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 106. `https://arxiv.org/abs/2203.11171#google`: [`“Self-Consistency Improves Chain-Of-Thought Reasoning in Language Models”`{=html}](https://arxiv.org/abs/2203.11171#google "'Self-Consistency Improves Chain-of-Thought Reasoning in Language Models', Wang et al 2022"){#wang-et-al-2022-20 .link-annotated}, [Xuezhi Wang, [Jason Wei](https://www.jasonwei.net/ "Jason Wei"){.id-not}, Dale Schuurmans[, [Quoc V. Le](https://en.wikipedia.org/wiki/Quoc_V._Le "Quoc V. Le"){.id-not}, [Ed Chi](https://en.wikipedia.org/wiki/Ed_Chi "Ed Chi"){.id-not}, [Denny Zhou](https://dennyzhou.github.io/ "Denny Zhou’s Home Page"){.link-annotated-partial .id-not}]{.collapse}]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2203.11171%2523google.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2203.11171#google"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 107. `https://arxiv.org/abs/2203.03466#microsoft`: [`“Tensor Programs V: Tuning Large Neural Networks via Zero-Shot Hyperparameter Transfer”`{=html}](https://arxiv.org/abs/2203.03466#microsoft "'Tensor Programs V: Tuning Large Neural Networks via Zero-Shot Hyperparameter Transfer', Yang et al 2022"){#yang-et-al-2022-8 .link-annotated}, [Greg Yang, Edward J. Hu, Igor Babuschkin[, Szymon Sidor, [Xiaodong Liu](https://scholar.google.com/citations?user=NIewcxMAAAAJ "Xiaodong Liu"){.id-not}, David Farhi, [Nick Ryder](https://scholar.google.com/citations?user=rLp0oucAAAAJ "Nick Ryder"){.id-not}, [Jakub Pachocki](https://en.wikipedia.org/wiki/Jakub_Pachocki "Jakub Pachocki"){.id-not}, [Weizhu Chen](https://scholar.google.com/citations?user=LG_E-4EAAAAJ "Weizhu Chen"){.id-not}, [Jianfeng Gao](https://www.microsoft.com/en-us/research/people/jfgao/ "Jianfeng Gao at Microsoft Research"){.link-annotated-partial .id-not}]{.collapse}]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2203.03466%2523microsoft.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2203.03466#microsoft"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 108. `https://arxiv.org/abs/2203.00854`: [`“FastFold: Reducing AlphaFold Training Time from 11 Days to 67 Hours”`{=html}](https://arxiv.org/abs/2203.00854 "'FastFold: Reducing AlphaFold Training Time from 11 Days to 67 Hours', Cheng et al 2022"){#cheng-et-al-2022-2 .link-annotated}, [Shenggan Cheng, Ruidong Wu, Zhongming Yu[, Binrui Li, Xiwen Zhang, Jian Peng, [Yang You](https://en.wikipedia.org/wiki/Yang_You "Yang You"){.id-not}]{.collapse}]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2203.00854.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2203.00854"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 109. `https://arxiv.org/abs/2202.12211#google`: [`“Self-Distilled StyleGAN: Towards Generation from Internet Photos”`{=html}](https://arxiv.org/abs/2202.12211#google "'Self-Distilled StyleGAN: Towards Generation from Internet Photos', Mokady et al 2022"){#mokady-et-al-2022-2 .link-annotated}, [Ron Mokady, Michal Yarom, Omer Tov[, Oran Lang, Daniel Cohen-Or, Tali Dekel, [Michal Irani](https://en.wikipedia.org/wiki/Michal_Irani "Michal Irani"){.id-not}, Inbar Mosseri]{.collapse}]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2202.12211%2523google.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2202.12211#google"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 110. `https://www.nature.com/articles/s42003-022-03036-1`: [`“Brains and Algorithms Partially Converge in Natural Language Processing”`{=html}](https://www.nature.com/articles/s42003-022-03036-1 "'Brains and algorithms partially converge in natural language processing', Caucheteux & King 2022"){#caucheteux-king-2022 .link-annotated}, [Charlotte Caucheteux, Jean-Rémi King]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Fwww.nature.com%252Farticles%252Fs42003-022-03036-1.html "Directory-tag link-bibliography for link https://www.nature.com/articles/s42003-022-03036-1"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 111. `https://arxiv.org/abs/2202.06767#huawei`: [`“Wukong: 100 Million Large-Scale Chinese Cross-Modal Pre-Training Dataset and A Foundation Framework”`{=html}](https://arxiv.org/abs/2202.06767#huawei "'Wukong: 100 Million Large-scale Chinese Cross-modal Pre-training Dataset and A Foundation Framework', Gu et al 2022"){#gu-et-al-2022-2 .link-annotated}, [Jiaxi Gu, Xiaojun Meng, Guansong Lu[, [Lu Hou](https://en.wikipedia.org/wiki/Lu_Hou "Lu Hou"){.id-not}, Minzhe Niu, Hang Xu, Xiaodan Liang, Wei Zhang, [Xin Jiang](https://en.wikipedia.org/wiki/Xin_Jiang "Xin Jiang"){.id-not}, Chunjing Xu]{.collapse}]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2202.06767%2523huawei.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2202.06767#huawei"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 112. `https://arxiv.org/abs/2202.03052#alibaba`: [`“OFA: Unifying Architectures, Tasks, and Modalities Through a Simple Sequence-To-Sequence Learning Framework”`{=html}](https://arxiv.org/abs/2202.03052#alibaba "'OFA: Unifying Architectures, Tasks, and Modalities Through a Simple Sequence-to-Sequence Learning Framework', Wang et al 2022"){#wang-et-al-2022-21 .link-annotated}, [Peng Wang, [An Yang](https://en.wikipedia.org/wiki/An_Yang "An Yang"){.id-not}, Rui Men[, Junyang Lin, Shuai Bai, Zhikang Li, Jianxin Ma, Chang Zhou, Jingren Zhou, Hongxia Yang]{.collapse}]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2202.03052%2523alibaba.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2202.03052#alibaba"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 113. `https://arxiv.org/abs/2202.02317#allen`: [`“Webly Supervised Concept Expansion for General Purpose Vision Models”`{=html}](https://arxiv.org/abs/2202.02317#allen "'Webly Supervised Concept Expansion for General Purpose Vision Models', Kamath et al 2022"){#kamath-et-al-2022 .link-annotated}, [Amita Kamath, [Christopher Clark](https://en.wikipedia.org/wiki/Christopher_Clark "Christopher Clark"){.id-not}, Tanmay Gupta[, Eric Kolve, Derek Hoiem, Aniruddha Kembhavi]{.collapse}]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2202.02317%2523allen.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2202.02317#allen"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 114. `https://arxiv.org/abs/2202.00273`: [`“StyleGAN-XL: Scaling StyleGAN to Large Diverse Datasets”`{=html}](https://arxiv.org/abs/2202.00273 "'StyleGAN-XL: Scaling StyleGAN to Large Diverse Datasets', Sauer et al 2022"){#sauer-et-al-2022 .link-annotated}, [Axel Sauer, Katja Schwarz, Andreas Geiger]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2202.00273.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2202.00273"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 115. `https://arxiv.org/abs/2201.11990#microsoftnvidia`: [`“Using DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, A Large-Scale Generative Language Model”`{=html}](https://arxiv.org/abs/2201.11990#microsoftnvidia "'Using DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, A Large-Scale Generative Language Model', Smith et al 2022"){#smith-et-al-2022-3 .link-annotated}, [Shaden Smith, Mostofa Patwary, Brandon Norick[, Patrick LeGresley, Samyam Rajbhandari, Jared Casper, Zhun Liu, Shrimai Prabhumoye, George Zerveas, Vijay Korthikanti, Elton Zhang, [Rewon Child](https://rewonc.github.io/){.link-annotated-partial .id-not}, Reza Yazdani Aminabadi, Julie Bernauer, Xia Song, [Mohammad Shoeybi](https://scholar.google.com/citations?user=62ElavIAAAAJ "Mohammad Shoeybi"){.id-not}, Yuxiong He, Michael Houston, [Saurabh Tiwary](https://en.wikipedia.org/wiki/Saurabh_Tiwary "Saurabh Tiwary"){.id-not}, [Bryan Catanzaro](https://scholar.google.com/citations?user=UZ6kI2AAAAAJ "Bryan Catanzaro"){.id-not}]{.collapse}]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2201.11990%2523microsoftnvidia.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2201.11990#microsoftnvidia"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 116. `https://arxiv.org/abs/2201.11473#microsoft`: [`“Reasoning Like Program Executors”`{=html}](https://arxiv.org/abs/2201.11473#microsoft "'Reasoning Like Program Executors', Pi et al 2022"){#pi-et-al-2022 .link-annotated}, [Xinyu Pi, [Qian Liu](https://en.wikipedia.org/wiki/Qian_Liu "Qian Liu"){.id-not}, Bei Chen[, Morteza Ziyadi, Zeqi Lin, Yan Gao, Qiang Fu, Jian-Guang Lou, [Weizhu Chen](https://scholar.google.com/citations?user=LG_E-4EAAAAJ "Weizhu Chen"){.id-not}]{.collapse}]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2201.11473%2523microsoft.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2201.11473#microsoft"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 117. `https://arxiv.org/abs/2201.10005#openai`: [`“Text and Code Embeddings by Contrastive Pre-Training”`{=html}](https://arxiv.org/abs/2201.10005#openai "'Text and Code Embeddings by Contrastive Pre-Training', Neelakantan et al 2022"){#neelakantan-et-al-2022 .link-annotated}, [[Arvind Neelakantan](https://scholar.google.com/citations?user=ygTCc6cAAAAJ "Arvind Neelakantan"){.id-not}, Tao Xu, Raul Puri[, [Alec Radford](https://scholar.google.com/citations?user=dOad5HoAAAAJ "Alec Radford"){.id-not}, Jesse Michael Han, Jerry Tworek, Qiming Yuan, Nikolas Tezak, [Jong Wook Kim](https://jongwook.kim/ "Jong Wook Kim"){.link-annotated-partial .id-not}, Chris Hallacy, Johannes Heidecke, [Pranav Shyam](https://scholar.google.com/citations?user=TAomEzQAAAAJ "Pranav Shyam"){.id-not}, Boris Power, Tyna Eloundou Nekoul, [Girish Sastry](https://girishsastry.com/ "Girish Sastry"){.link-annotated-partial .id-not}, [Gretchen Krueger](https://scholar.google.com/citations?user=astFxkwAAAAJ "Gretchen Krueger"){.id-not}, David Schnurr, Felipe Petroski Such, Kenny Hsu, [Madeleine Thompson](https://en.wikipedia.org/wiki/Madeleine_Thompson "Madeleine Thompson"){.id-not}, Tabarak Khan, Toki Sherbakov, Joanne Jang, [Peter Welinder](https://scholar.google.com/citations?user=B76dD7YAAAAJ "Peter Welinder"){.id-not}, [Lilian Weng](https://lilianweng.github.io/ "Lil'Log"){.link-annotated-partial .id-not}]{.collapse}]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2201.10005%2523openai.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2201.10005#openai"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 118. `https://arxiv.org/abs/2201.08371#facebook`: [`“SWAG: Revisiting Weakly Supervised Pre-Training of Visual Perception Models”`{=html}](https://arxiv.org/abs/2201.08371#facebook "'SWAG: Revisiting Weakly Supervised Pre-Training of Visual Perception Models', Singh et al 2022"){#singh-et-al-2022 .link-annotated}, [Mannat Singh, Laura Gustafson, Aaron Adcock[, Vinicius de Freitas Reis, Bugra Gedik, Raj Prateek Kosaraju, Dhruv Mahajan, [Ross Girshick](https://scholar.google.com/citations?user=W8VIEZgAAAAJ "Ross Girshick"){.id-not}, Piotr Dollár, [Laurens van der Maaten](https://lvdmaaten.github.io/ "Laurens van der Maaten"){.link-annotated-partial .id-not}]{.collapse}]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2201.08371%2523facebook.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2201.08371#facebook"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 119. `https://arxiv.org/abs/2201.07520#facebook`: [`“CM3: A Causal Masked Multimodal Model of the Internet”`{=html}](https://arxiv.org/abs/2201.07520#facebook "'CM3: A Causal Masked Multimodal Model of the Internet', Aghajanyan et al 2022"){#aghajanyan-et-al-2022 .link-annotated}, [Armen Aghajanyan, Bernie Huang, Candace Ross[, Vladimir Karpukhin, Hu Xu, [Naman Goyal](https://namangoyal.com/ "Naman Goyal"){.link-annotated-partial .id-not}, Dmytro Okhonko, Mandar Joshi, Gargi Ghosh, [Mike Lewis](https://scholar.google.com/citations?user=SnQnQicAAAAJ "Mike Lewis"){.id-not}, [Luke Zettlemoyer](https://scholar.google.com/citations?user=UjpbO6IAAAAJ "Luke Zettlemoyer"){.id-not}]{.collapse}]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2201.07520%2523facebook.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2201.07520#facebook"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 120. `https://arxiv.org/abs/2201.06910`: [`“ZeroPrompt: Scaling Prompt-Based Pretraining to 1,000 Tasks Improves Zero-Shot Generalization”`{=html}](https://arxiv.org/abs/2201.06910 "'ZeroPrompt: Scaling Prompt-Based Pretraining to 1,000 Tasks Improves Zero-Shot Generalization', Xu et al 2022"){#xu-et-al-2022-7 .link-annotated}, [Hanwei Xu, Yujun Chen, Yulun Du[, Nan Shao, Yanggang Wang, Haiyu Li, [Zhilin Yang](https://scholar.google.com/citations?user=7qXxyJkAAAAJ "Zhilin Yang"){.id-not}]{.collapse}]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2201.06910.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2201.06910"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 121. `https://arxiv.org/abs/2201.03545#facebook`: [`“ConvNeXt: A ConvNet for the 2020s”`{=html}](https://arxiv.org/abs/2201.03545#facebook "'ConvNeXt: A ConvNet for the 2020s', Liu et al 2022"){#liu-et-al-2022-23 .link-annotated}, [[Zhuang Liu](https://liuzhuang13.github.io/ "Zhuang Liu’s Homepage"){.link-annotated-partial .id-not}, Hanzi Mao, Chao-Yuan Wu[, Christoph Feichtenhofer, [Trevor Darrell](https://en.wikipedia.org/wiki/Trevor_Darrell "Trevor Darrell"){.id-not}, Saining Xie]{.collapse}]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2201.03545%2523facebook.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2201.03545#facebook"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 122. `https://royalsocietypublishing.org/doi/10.1098/rstb.2020.0529`: [`“The Evolution of Quantitative Sensitivity”`{=html}](https://royalsocietypublishing.org/doi/10.1098/rstb.2020.0529 "'The evolution of quantitative sensitivity', Bryer et al 2021"){#bryer-et-al-2021 .link-annotated}, [Margaret A. H. Bryer, Sarah E. Koopman, [Jessica F. Cantlon](https://en.wikipedia.org/wiki/Jessica_F._Cantlon "Jessica F. Cantlon"){.id-not}[, [Steven T. Piantadosi](https://colala.berkeley.edu/people/piantadosi/ "Steven T. Piantadosi"){.link-annotated-partial .id-not}, Evan L. MacLean, [Joseph M. Baker](https://en.wikipedia.org/wiki/Joseph_M._Baker "Joseph M. Baker"){.id-not}, Michael J. Beran, Sarah M. Jones, Kerry E. Jordan, Salif Mahamane, Andreas Nieder, Bonnie M. Perdue, Friederike Range, Jeffrey R. Stevens, Masaki Tomonaga, Dorottya J. Ujfalussy, Jennifer Vonk]{.collapse}]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Froyalsocietypublishing.org%252Fdoi%252F10.1098%252Frstb.2020.0529.html "Directory-tag link-bibliography for link https://royalsocietypublishing.org/doi/10.1098/rstb.2020.0529"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 123. `https://arxiv.org/abs/2112.05253`: [`“MAGMA—Multimodal Augmentation of Generative Models through Adapter-Based Finetuning”`{=html}](https://arxiv.org/abs/2112.05253 "'MAGMA—Multimodal Augmentation of Generative Models through Adapter-based Finetuning', Eichenberg et al 2021"){#eichenberg-et-al-2021 .link-annotated}, [Constantin Eichenberg, Sidney Black, Samuel Weinbach[, Letitia Parcalabescu, Anette Frank]{.collapse}]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2112.05253.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2112.05253"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 124. `https://arxiv.org/abs/2112.04426#deepmind`: [`“Improving Language Models by Retrieving from Trillions of Tokens”`{=html}](https://arxiv.org/abs/2112.04426#deepmind "'Improving language models by retrieving from trillions of tokens', Borgeaud et al 2021"){#borgeaud-et-al-2021 .link-annotated}, [Sebastian Borgeaud, Arthur Mensch, Jordan Hoffmann[, Trevor Cai, Eliza Rutherford, Katie Millican, George van den Driessche, Jean-Baptiste Lespiau, Bogdan Damoc, Aidan Clark, Diego de Las Casas, Aurelia Guy, Jacob Menick, Roman Ring, Tom Hennigan, Saffron Huang, Loren Maggiore, Chris Jones, Albin Cassirer, Andy Brock, Michela Paganini, [Geoffrey Irving](https://naml.us/ "Geoffrey Irving"){.link-annotated-partial .id-not}, [Oriol Vinyals](https://en.wikipedia.org/wiki/Oriol_Vinyals "Oriol Vinyals"){.id-not}, [Simon Osindero](https://scholar.google.com/citations?user=Jq8ZS5kAAAAJ "Simon Osindero"){.id-not}, [Karen Simonyan](https://scholar.google.com/citations?user=L7lMQkQAAAAJ "Karen Simonyan"){.id-not}, Jack W. Rae, Erich Elsen, [Laurent Sifre](https://scholar.google.com/citations?user=0kVh58wAAAAJ "Laurent Sifre"){.id-not}]{.collapse}]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2112.04426%2523deepmind.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2112.04426#deepmind"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 125. `https://arxiv.org/abs/2111.12233#microsoft`: [`“LEMON: Scaling Up Vision-Language Pre-Training for Image Captioning”`{=html}](https://arxiv.org/abs/2111.12233#microsoft "'LEMON: Scaling Up Vision-Language Pre-training for Image Captioning', Hu et al 2021"){#hu-et-al-2021-2 .link-annotated}, [Xiaowei Hu, [Zhe Gan](https://zhegan27.github.io/ "Zhe Gan"){.id-not}, Jianfeng Wang[, Zhengyuan Yang, [Zicheng Liu](https://en.wikipedia.org/wiki/Zicheng_Liu "Zicheng Liu"){.id-not}, Yumao Lu, Lijuan Wang]{.collapse}]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2111.12233%2523microsoft.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2111.12233#microsoft"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 126. `https://arxiv.org/abs/2111.12763#google`: [`“Scaling Transformers: Sparse Is Enough in Scaling Transformers”`{=html}](https://arxiv.org/abs/2111.12763#google "'Scaling Transformers: Sparse is Enough in Scaling Transformers', Jaszczur et al 2021"){#jaszczur-et-al-2021 .link-annotated}, [Sebastian Jaszczur, Aakanksha Chowdhery, Afroz Mohiuddin[, [Łukasz Kaiser](https://scholar.google.com/citations?user=JWmiQR0AAAAJ "Łukasz Kaiser"){.id-not}, Wojciech Gajewski, Henryk Michalewski, Jonni Kanerva]{.collapse}]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2111.12763%2523google.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2111.12763#google"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 127. `https://arxiv.org/abs/2111.11904#microsoft`: [`“Can Pre-Trained Language Models Be Used to Resolve Textual and Semantic Merge Conflicts?”`{=html}](https://arxiv.org/abs/2111.11904#microsoft "'Can Pre-trained Language Models be Used to Resolve Textual and Semantic Merge Conflicts?', Zhang et al 2021"){#zhang-et-al-2021-04 .link-annotated}, [Jialu Zhang, Todd Mytkowicz, Mike Kaufman[, Ruzica Piskac, Shuvendu K. Lahiri]{.collapse}]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2111.11904%2523microsoft.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2111.11904#microsoft"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 128. `https://arxiv.org/abs/2111.11133`: [`“L-Verse: Bidirectional Generation Between Image and Text”`{=html}](https://arxiv.org/abs/2111.11133 "'L-Verse: Bidirectional Generation Between Image and Text', Kim et al 2021"){#kim-et-al-2021-3 .link-annotated}, [Taehoon Kim, Gwangmo Song, Sihaeng Lee[, Sangyun Kim, Yewon Seo, Soonyoung Lee, Seung Hwan Kim, Honglak Lee, Kyunghoon Bae]{.collapse}]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2111.11133.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2111.11133"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 129. `https://arxiv.org/abs/2111.11432#microsoft`: [`“Florence: A New Foundation Model for Computer Vision”`{=html}](https://arxiv.org/abs/2111.11432#microsoft "'Florence: A New Foundation Model for Computer Vision', Yuan et al 2021"){#yuan-et-al-2021-1 .link-annotated}, [Lu Yuan, Dongdong Chen, Yi-Ling Chen[, Noel Codella, Xiyang Dai, [Jianfeng Gao](https://www.microsoft.com/en-us/research/people/jfgao/ "Jianfeng Gao at Microsoft Research"){.link-annotated-partial .id-not}, Houdong Hu, [Xuedong Huang](https://en.wikipedia.org/wiki/Xuedong_Huang "Xuedong Huang"){.id-not}, Boxin Li, Chunyuan Li, Ce Liu, Mengchen Liu, [Zicheng Liu](https://en.wikipedia.org/wiki/Zicheng_Liu "Zicheng Liu"){.id-not}, Yumao Lu, [Yu Shi](https://en.wikipedia.org/wiki/Yu_Shi "Yu Shi"){.id-not}, Lijuan Wang, Jianfeng Wang, Bin Xiao, Zhen Xiao, Jianwei Yang, Michael Zeng, Luowei Zhou, Pengchuan Zhang]{.collapse}]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2111.11432%2523microsoft.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2111.11432#microsoft"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 130. `https://arxiv.org/abs/2111.10050#google`: [`“BASIC: Combined Scaling for Open-Vocabulary Image Classification”`{=html}](https://arxiv.org/abs/2111.10050#google "'BASIC: Combined Scaling for Open-Vocabulary Image Classification', Pham et al 2021"){#pham-et-al-2021-1 .link-annotated}, [Hieu Pham, [Zihang Dai](https://scholar.google.com/citations?user=uZqsVXkAAAAJ "Zihang Dai"){.id-not}, Golnaz Ghiasi[, Kenji Kawaguchi, Hanxiao Liu, Adams Wei Yu, Jiahui Yu, Yi-Ting Chen, Minh-Thang Luong, [Yonghui Wu](https://scholar.google.com/citations?user=bMWMlk4AAAAJ "Yonghui Wu"){.id-not}, Mingxing Tan, [Quoc V. Le](https://en.wikipedia.org/wiki/Quoc_V._Le "Quoc V. Le"){.id-not}]{.collapse}]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2111.10050%2523google.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2111.10050#google"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 131. `https://arxiv.org/abs/2111.08267`: [`“Solving Probability and Statistics Problems by Program Synthesis”`{=html}](https://arxiv.org/abs/2111.08267 "'Solving Probability and Statistics Problems by Program Synthesis', Tang et al 2021"){#tang-et-al-2021-1 .link-annotated}, [Leonard Tang, Elizabeth Ke, Nikhil Singh[, [Nakul Verma](https://en.wikipedia.org/wiki/Nakul_Verma "Nakul Verma"){.id-not}, Iddo Drori]{.collapse}]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2111.08267.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2111.08267"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 132. `https://arxiv.org/abs/2111.06377#facebook`: [`“MAE: Masked Autoencoders Are Scalable Vision Learners”`{=html}](https://arxiv.org/abs/2111.06377#facebook "'MAE: Masked Autoencoders Are Scalable Vision Learners', He et al 2021"){#he-et-al-2021-1 .link-annotated}, [[Kaiming He](https://en.wikipedia.org/wiki/Kaiming_He "Kaiming He"){.id-not}, [Xinlei Chen](https://xinleic.xyz/ "Xinlei Chen"){.link-annotated-partial .id-not}, Saining Xie[, Yanghao Li, Piotr Dollár, [Ross Girshick](https://scholar.google.com/citations?user=W8VIEZgAAAAJ "Ross Girshick"){.id-not}]{.collapse}]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2111.06377%2523facebook.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2111.06377#facebook"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 133. `https://arxiv.org/abs/2111.02114#laion`: [`“LAION-400M: Open Dataset of CLIP-Filtered 400 Million Image-Text Pairs”`{=html}](https://arxiv.org/abs/2111.02114#laion "'LAION-400M: Open Dataset of CLIP-Filtered 400 Million Image-Text Pairs', Schuhmann et al 2021"){#schuhmann-et-al-2021 .link-annotated}, [Christoph Schuhmann, Richard Vencu, Romain Beaumont[, Robert Kaczmarczyk, Clayton Mullis, Aarush Katta, Theo Coombes, Jenia Jitsev, Aran Komatsuzaki]{.collapse}]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2111.02114%2523laion.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2111.02114#laion"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 134. `https://arxiv.org/abs/2110.14168#openai`: [`“Training Verifiers to Solve Math Word Problems”`{=html}](https://arxiv.org/abs/2110.14168#openai "'Training Verifiers to Solve Math Word Problems', Cobbe et al 2021"){#cobbe-et-al-2021 .link-annotated}, [[Karl Cobbe](https://scholar.google.com/citations?user=stCljMYAAAAJ "Karl Cobbe"){.id-not}, Vineet Kosaraju, Mohammad Bavarian[, [Jacob Hilton](https://www.jacobh.co.uk/ "Jacob Hilton's Homepage"){.link-annotated-partial .id-not}, Reiichiro Nakano, [Christopher Hesse](https://scholar.google.com/citations?user=SgbbTp4AAAAJ "Christopher Hesse"){.id-not}, [John Schulman](http://joschu.net/ "'John Schulman’s Homepage', Schulman 2026"){.link-annotated-partial .id-not}]{.collapse}]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2110.14168%2523openai.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2110.14168#openai"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 135. `https://arxiv.org/abs/2110.11526#deepmind`: [`“Wide Neural Networks Forget Less Catastrophically”`{=html}](https://arxiv.org/abs/2110.11526#deepmind "'Wide Neural Networks Forget Less Catastrophically', Mirzadeh et al 2021"){#mirzadeh-et-al-2021 .link-annotated}, [Seyed Iman Mirzadeh, Arslan Chaudhry, Dong Yin[, Huiyi Hu, [Razvan Pascanu](https://sites.google.com/view/razp/home){.link-annotated-partial .id-not}, Dilan Gorur, Mehrdad Farajtabar]{.collapse}]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2110.11526%2523deepmind.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2110.11526#deepmind"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 136. `https://arxiv.org/abs/2110.02095#google`: [`“Exploring the Limits of Large Scale Pre-Training”`{=html}](https://arxiv.org/abs/2110.02095#google "'Exploring the Limits of Large Scale Pre-training', Abnar et al 2021"){#abnar-et-al-2021 .link-annotated}, [Samira Abnar, [Mostafa Dehghani](https://scholar.google.com/citations?user=MiHOX3QAAAAJ "Mostafa Dehghani"){.id-not}, [Behnam Neyshabur](https://www.neyshabur.net/ "Behnam Neyshabur"){.id-not}, Hanie Sedghi]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2110.02095%2523google.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2110.02095#google"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 137. `https://arxiv.org/abs/2109.10686#google`: [`“Scale Efficiently: Insights from Pre-Training and Fine-Tuning Transformers”`{=html}](https://arxiv.org/abs/2109.10686#google "'Scale Efficiently: Insights from Pre-training and Fine-tuning Transformers', Tay et al 2021"){#tay-et-al-2021-1 .link-annotated}, [[Yi Tay](https://www.yitay.net/ "Yi Tay"){.link-annotated-partial .id-not}, [Mostafa Dehghani](https://scholar.google.com/citations?user=MiHOX3QAAAAJ "Mostafa Dehghani"){.id-not}, Jinfeng Rao[, [William Fedus](https://scholar.google.com/citations?user=-ZfwQOkAAAAJ "William Fedus"){.id-not}, Samira Abnar, Hyung Won Chung, [Sharan Narang](https://scholar.google.com/citations?user=CWOixywAAAAJ "Sharan Narang"){.link-modified-recently .id-not}, [Dani Yogatama](https://dyogatama.github.io/ "Dani Yogatama"){.link-annotated-partial .id-not}, [Ashish Vaswani](https://en.wikipedia.org/wiki/Ashish_Vaswani "Ashish Vaswani"){.id-not}, [Donald Metzler](https://www.don-metzler.net/ "About—Donald Metzler"){.link-annotated-partial .id-not}]{.collapse}]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2109.10686%2523google.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2109.10686#google"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 138. `https://arxiv.org/abs/2109.07958`: [`“TruthfulQA: Measuring How Models Mimic Human Falsehoods”`{=html}](https://arxiv.org/abs/2109.07958 "'TruthfulQA: Measuring How Models Mimic Human Falsehoods', Lin et al 2021"){#lin-et-al-2021-6 .link-annotated}, [[Stephanie Lin](https://en.wikipedia.org/wiki/Stephanie_Lin "Stephanie Lin"){.id-not}, [Jacob Hilton](https://www.jacobh.co.uk/ "Jacob Hilton's Homepage"){.link-annotated-partial .id-not}, [Owain Evans](https://owainevans.github.io/ "Owain Evans, AI Alignment researcher"){.link-annotated-partial .id-not}]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2109.07958.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2109.07958"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 139. `https://arxiv.org/abs/2109.02593#allen`: [`“General-Purpose Question-Answering With Macaw”`{=html}](https://arxiv.org/abs/2109.02593#allen "'General-Purpose Question-Answering with Macaw', Tafjord & Clark 2021"){#tafjord-clark-2021 .link-annotated}, [Oyvind Tafjord, Peter Clark]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2109.02593%2523allen.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2109.02593#allen"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 140. `https://arxiv.org/abs/2108.13002#microsoft`: [`“A Battle of Network Structures: An Empirical Study of CNN, Transformer, and MLP”`{=html}](https://arxiv.org/abs/2108.13002#microsoft "'A Battle of Network Structures: An Empirical Study of CNN, Transformer, and MLP', Zhao et al 2021"){#zhao-et-al-2021-4 .link-annotated}, [Yucheng Zhao, Guangting Wang, Chuanxin Tang[, Chong Luo, [Wenjun Zeng](https://en.wikipedia.org/wiki/Wenjun_Zeng "Wenjun Zeng"){.id-not}, Zheng-Jun Zha]{.collapse}]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2108.13002%2523microsoft.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2108.13002#microsoft"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 141. `https://arxiv.org/abs/2108.08810#google`: [`“Do Vision Transformers See Like Convolutional Neural Networks?”`{=html}](https://arxiv.org/abs/2108.08810#google "'Do Vision Transformers See Like Convolutional Neural Networks?', Raghu et al 2021"){#raghu-et-al-2021 .link-annotated}, [Maithra Raghu, Thomas Unterthiner, [Simon Kornblith](https://www.zotero.org/simon/cv "'Simon Kornblith CV', Kornblith 2026"){.id-not}[, Chiyuan Zhang, Alexey Dosovitskiy]{.collapse}]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2108.08810%2523google.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2108.08810#google"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 142. `https://arxiv.org/abs/2108.07686`: [`“Scaling Laws for Deep Learning”`{=html}](https://arxiv.org/abs/2108.07686 "'Scaling Laws for Deep Learning', Rosenfeld 2021"){#rosenfeld-2021 .link-annotated}, [[Jonathan S. Rosenfeld](https://scholar.google.com/citations?user=8HYnMeYAAAAJ&view_op=list_works&sortby=pubdate "Jonathan S. Rosenfeld"){.id-not}]{.author .cite-author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2108.07686.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2108.07686"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 143. `https://arxiv.org/abs/2107.02137#baidu`: [`“ERNIE 3.0: Large-Scale Knowledge Enhanced Pre-Training for Language Understanding and Generation”`{=html}](https://arxiv.org/abs/2107.02137#baidu "'ERNIE 3.0: Large-scale Knowledge Enhanced Pre-training for Language Understanding and Generation', Sun et al 2021"){#sun-et-al-2021-4 .link-annotated}, [[Yu Sun](https://yueatsprograms.github.io/ "Yu Sun"){.link-annotated-partial .id-not}, Shuohuan Wang, Shikun Feng[, Siyu Ding, Chao Pang, Junyuan Shang, Jiaxiang Liu, Xuyi Chen, Yanbin Zhao, Yuxiang Lu, Weixin Liu, Zhihua Wu, Weibao Gong, Jianzhong Liang, Zhizhou Shang, [Peng Sun](https://en.wikipedia.org/wiki/Peng_Sun "Peng Sun"){.id-not}, Wei Liu, Xuan Ouyang, Dianhai Yu, Hao Tian, Hua Wu, Haifeng Wang]{.collapse}]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2107.02137%2523baidu.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2107.02137#baidu"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 144. `https://arxiv.org/abs/2107.01294#allen`: [`“Scarecrow: A Framework for Scrutinizing Machine Text”`{=html}](https://arxiv.org/abs/2107.01294#allen "'Scarecrow: A Framework for Scrutinizing Machine Text', Dou et al 2021"){#dou-et-al-2021 .link-annotated}, [Yao Dou, Maxwell Forbes, Rik Koncel-Kedziorski[, [Noah A. Smith](https://nasmith.github.io/ "Noah A. Smith"){.link-annotated-partial .id-not}, [Yejin Choi](https://en.wikipedia.org/wiki/Yejin_Choi "Yejin Choi"){.id-not}]{.collapse}]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2107.01294%2523allen.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2107.01294#allen"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 145. `https://arxiv.org/abs/2106.07411`: [`“Partial Success in Closing the Gap between Human and Machine Vision”`{=html}](https://arxiv.org/abs/2106.07411 "'Partial success in closing the gap between human and machine vision', Geirhos et al 2021"){#geirhos-et-al-2021 .link-annotated}, [[Robert Geirhos](https://scholar.google.com/citations?user=w3kGtMIAAAAJ "Robert Geirhos"){.id-not}, Kantharaju Narayanappa, Benjamin Mitzkus[, Tizian Thieringer, [Matthias Bethge](https://scholar.google.com/citations?user=0z0fNxUAAAAJ "Matthias Bethge"){.id-not}, Felix A. Wichmann, Wiel, Brendel]{.collapse}]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2106.07411.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2106.07411"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 146. `https://arxiv.org/abs/2106.04803#google`: [`“CoAtNet: Marrying Convolution and Attention for All Data Sizes”`{=html}](https://arxiv.org/abs/2106.04803#google "'CoAtNet: Marrying Convolution and Attention for All Data Sizes', Dai et al 2021"){#dai-et-al-2021-2 .link-annotated}, [[Zihang Dai](https://scholar.google.com/citations?user=uZqsVXkAAAAJ "Zihang Dai"){.id-not}, Hanxiao Liu, [Quoc V. Le](https://en.wikipedia.org/wiki/Quoc_V._Le "Quoc V. Le"){.id-not}, Mingxing Tan]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2106.04803%2523google.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2106.04803#google"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 147. `https://arxiv.org/abs/2106.04560#google`: [`“Scaling Vision Transformers”`{=html}](https://arxiv.org/abs/2106.04560#google "'Scaling Vision Transformers', Zhai et al 2021"){#zhai-et-al-2021-3 .link-annotated}, [[Xiaohua Zhai](https://sites.google.com/view/xzhai "Xiaohua Zhai (翟晓华)"){.link-annotated-partial .id-not}, Alexander Kolesnikov, [Neil Houlsby](https://neilhoulsby.github.io/ "Neil Houlsby"){.link-annotated-partial .id-not}, [Lucas Beyer](https://scholar.google.com/citations?user=p2gwhK4AAAAJ "Lucas Beyer"){.id-not}]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2106.04560%2523google.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2106.04560#google"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 148. `https://arxiv.org/abs/2106.03004#google`: [`“Exploring the Limits of Out-Of-Distribution Detection”`{=html}](https://arxiv.org/abs/2106.03004#google "'Exploring the Limits of Out-of-Distribution Detection', Fort et al 2021"){#fort-et-al-2021 .link-annotated}, [Stanislav Fort, Jie Ren, Balaji Lakshminarayanan]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2106.03004%2523google.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2106.03004#google"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 149. `https://arxiv.org/abs/2106.00116`: [`“Effect of Pre-Training Scale on Intra/Inter-Domain Full and Few-Shot Transfer Learning for Natural and Medical X-Ray Chest Images”`{=html}](https://arxiv.org/abs/2106.00116 "'Effect of Pre-Training Scale on Intra/Inter-Domain Full and Few-Shot Transfer Learning for Natural and Medical X-Ray Chest Images', Cherti & Jitsev 2021"){#cherti-jitsev-2021 .link-annotated}, [Mehdi Cherti, Jenia Jitsev]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2106.00116.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2106.00116"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 150. `https://arxiv.org/abs/2105.12806`: [`“A Universal Law of Robustness via Isoperimetry”`{=html}](https://arxiv.org/abs/2105.12806 "'A Universal Law of Robustness via Isoperimetry', Bubeck & Sellke 2021"){#bubeck-sellke-2021 .link-annotated}, [[Sébastien Bubeck](https://en.wikipedia.org/wiki/S%C3%A9bastien_Bubeck "Sébastien Bubeck"){.id-not}, Mark Sellke]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2105.12806.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2105.12806"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 151. `https://m.koreaherald.com/view.php?ud=20210525000824#naver`: [`“Naver Unveils First ‘Hyperscale’ AI Platform”`{=html}](https://m.koreaherald.com/view.php?ud=20210525000824#naver "'Naver unveils first ‘hyperscale’ AI platform', Jae-eun 2021"){#jae-eun-2021 .link-annotated}, [Kang Jae-eun]{.author .cite-author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Fm.koreaherald.com%252Fview.php%253Fud%253D20210525000824%2523naver.html "Directory-tag link-bibliography for link https://m.koreaherald.com/view.php?ud=20210525000824#naver"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 152. `https://arxiv.org/abs/2105.11084#facebook`: [`“Unsupervised Speech Recognition”`{=html}](https://arxiv.org/abs/2105.11084#facebook "'Unsupervised Speech Recognition', Baevski et al 2021"){#baevski-et-al-2021 .link-annotated}, [Alexei Baevski, Wei-Ning Hsu, Alexis Conneau, Michael Auli]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2105.11084%2523facebook.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2105.11084#facebook"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 153. `https://venturebeat.com/ai/google-details-new-ai-accelerator-chips/`: [`“Google Details New AI Accelerator Chips”`{=html}](https://venturebeat.com/ai/google-details-new-ai-accelerator-chips/ "'Google details new AI accelerator chips', Wiggers 2021"){#wiggers-2021 .link-annotated}, [Kyle Wiggers]{.author .cite-author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Fventurebeat.com%252Fai%252Fgoogle-details-new-ai-accelerator-chips%252F.html "Directory-tag link-bibliography for link https://venturebeat.com/ai/google-details-new-ai-accelerator-chips/"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 154. `https://arxiv.org/abs/2105.01601#google`: [`“MLP-Mixer: An All-MLP Architecture for Vision”`{=html}](https://arxiv.org/abs/2105.01601#google "'MLP-Mixer: An all-MLP Architecture for Vision', Tolstikhin et al 2021"){#tolstikhin-et-al-2021 .link-annotated}, [Ilya Tolstikhin, [Neil Houlsby](https://neilhoulsby.github.io/ "Neil Houlsby"){.link-annotated-partial .id-not}, Alexander Kolesnikov[, [Lucas Beyer](https://scholar.google.com/citations?user=p2gwhK4AAAAJ "Lucas Beyer"){.id-not}, [Xiaohua Zhai](https://sites.google.com/view/xzhai "Xiaohua Zhai (翟晓华)"){.link-annotated-partial .id-not}, Thomas Unterthiner, Jessica Yung, Daniel Keysers, [Jakob Uszkoreit](https://scholar.google.com/citations?user=mOG0bwsAAAAJ "Jakob Uszkoreit"){.id-not}, Mario Lucic, Alexey Dosovitskiy]{.collapse}]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2105.01601%2523google.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2105.01601#google"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 155. `https://arxiv.org/abs/2105.00572#facebook`: [`“XLM-R XL: Larger-Scale Transformers for Multilingual Masked Language Modeling”`{=html}](https://arxiv.org/abs/2105.00572#facebook "'XLM-R XL: Larger-Scale Transformers for Multilingual Masked Language Modeling', Goyal et al 2021"){#goyal-et-al-2021-xlmrxl .link-annotated}, [[Naman Goyal](https://namangoyal.com/ "Naman Goyal"){.link-annotated-partial .id-not}, Jingfei Du, Myle Ott[, Giri Anantharaman, Alexis Conneau]{.collapse}]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2105.00572%2523facebook.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2105.00572#facebook"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 156. `https://arxiv.org/abs/2104.14294#facebook`: [`“DINO: Emerging Properties in Self-Supervised Vision Transformers”`{=html}](https://arxiv.org/abs/2104.14294#facebook "'DINO: Emerging Properties in Self-Supervised Vision Transformers', Caron et al 2021"){#caron-et-al-2021 .link-annotated}, [Mathilde Caron, Hugo Touvron, [Ishan Misra](https://imisra.github.io/ "Ishan Misra"){.id-not}[, Hervé Jégou, Julien Mairal, [Piotr Bojanowski](https://scholar.google.com/citations?user=lJ_oh2EAAAAJ "Piotr Bojanowski"){.id-not}, [Armand Joulin](https://scholar.google.com/citations?user=kRJkDakAAAAJ "Armand Joulin"){.id-not}]{.collapse}]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2104.14294%2523facebook.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2104.14294#facebook"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 157. `abstract`: [`“Machine Learning Scaling”`{=html}](/doc/ai/scaling/abstract "'Machine Learning Scaling', Gwern 2021"){#gwern-doc-ai-scaling-abstract-2 .link-annotated-partial}, [[Gwern](/index#abstract "'Gwern Bibliography', Gwern 2010"){.link-annotated-partial .id-not}]{.author .cite-author} [[link-bibliography](/metadata/annotation/link-bibliography/%252Fdoc%252Fai%252Fscaling%252Fabstract.html "Directory-tag link-bibliography for link /doc/ai/scaling/abstract"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 158. `https://arxiv.org/abs/2104.02133#google`: [`“SpeechStew: Simply Mix All Available Speech Recognition Data to Train One Large Neural Network”`{=html}](https://arxiv.org/abs/2104.02133#google "'SpeechStew: Simply Mix All Available Speech Recognition Data to Train One Large Neural Network', Chan et al 2021"){#chan-et-al-2021-2 .link-annotated}, [[William Chan](http://williamchan.ca/ "William Chan"){.link-annotated-partial .id-not}, Daniel Park, Chris Lee[, [Yu Zhang](https://cse.sustech.edu.cn/faculty/~zhangy/ "Yu Zhang’s Homepage"){.link-annotated-partial .id-not}, [Quoc V. Le](https://en.wikipedia.org/wiki/Quoc_V._Le "Quoc V. Le"){.id-not}, [Mohammad Norouzi](https://en.wikipedia.org/wiki/Mohammad_Norouzi "Mohammad Norouzi"){.id-not}]{.collapse}]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2104.02133%2523google.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2104.02133#google"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 159. `https://arxiv.org/abs/2103.14586#google`: [`“Understanding Robustness of Transformers for Image Classification”`{=html}](https://arxiv.org/abs/2103.14586#google "'Understanding Robustness of Transformers for Image Classification', Bhojanapalli et al 2021"){#bhojanapalli-et-al-2021 .link-annotated}, [Srinadh Bhojanapalli, Ayan Chakrabarti, Daniel Glasner[, Daliang Li, Thomas Unterthiner, Andreas Veit]{.collapse}]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2103.14586%2523google.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2103.14586#google"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 160. `https://arxiv.org/abs/2103.13009#allen`: [`“UNICORN on RAINBOW: A Universal Commonsense Reasoning Model on a New Multitask Benchmark”`{=html}](https://arxiv.org/abs/2103.13009#allen "'UNICORN on RAINBOW: A Universal Commonsense Reasoning Model on a New Multitask Benchmark', Lourie et al 2021"){#lourie-et-al-2021 .link-annotated}, [Nicholas Lourie, Ronan Le Bras, Chandra Bhagavatula, [Yejin Choi](https://en.wikipedia.org/wiki/Yejin_Choi "Yejin Choi"){.id-not}]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2103.13009%2523allen.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2103.13009#allen"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 161. `https://arxiv.org/abs/2103.10957#deepmind`: [`“Efficient Visual Pretraining With Contrastive Detection”`{=html}](https://arxiv.org/abs/2103.10957#deepmind "'Efficient Visual Pretraining with Contrastive Detection', Hénaff et al 2021"){#hénaff-et-al-2021 .link-annotated}, [Olivier J. Hénaff, Skanda Koppula, Jean-Baptiste Alayrac[, Aaron van den Oord, [Oriol Vinyals](https://en.wikipedia.org/wiki/Oriol_Vinyals "Oriol Vinyals"){.id-not}, João Carreira]{.collapse}]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2103.10957%2523deepmind.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2103.10957#deepmind"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 162. `https://arxiv.org/abs/2103.07579#google`: [`“Revisiting ResNets: Improved Training and Scaling Strategies”`{=html}](https://arxiv.org/abs/2103.07579#google "'Revisiting ResNets: Improved Training and Scaling Strategies', Bello et al 2021"){#bello-et-al-2021 .link-annotated}, [Irwan Bello, [William Fedus](https://scholar.google.com/citations?user=-ZfwQOkAAAAJ "William Fedus"){.id-not}, Xianzhi Du[, Ekin D. Cubuk, [Aravind Srinivas](https://scholar.google.com/citations?user=GhrKC1gAAAAJ "Aravind Srinivas"){.id-not}, Tsung-Yi Lin, Jonathon Shlens, [Barret Zoph](https://barretzoph.github.io/ "Barret Zoph"){.link-annotated-partial .id-not}]{.collapse}]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2103.07579%2523google.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2103.07579#google"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 163. `https://ai.meta.com/blog/learning-from-videos-to-understand-the-world/`: [`“Learning from Videos to Understand the World”`{=html}](https://ai.meta.com/blog/learning-from-videos-to-understand-the-world/ "'Learning from videos to understand the world', Zweig et al 2021"){#zweig-et-al-2021 .link-annotated}, [Geoffrey Zweig, [Polina Kuznetsova](https://scholar.google.com/citations?user=JdEJEicAAAAJ "Polina Kuznetsova"){.id-not}, Michael Auli, Francois Fagan]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Fai.meta.com%252Fblog%252Flearning-from-videos-to-understand-the-world%252F.html "Directory-tag link-bibliography for link https://ai.meta.com/blog/learning-from-videos-to-understand-the-world/"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 164. `https://arxiv.org/abs/2103.06561`: [`“WenLan: Bridging Vision and Language by Large-Scale Multi-Modal Pre-Training”`{=html}](https://arxiv.org/abs/2103.06561 "'WenLan: Bridging Vision and Language by Large-Scale Multi-Modal Pre-Training', Huo et al 2021"){#huo-et-al-2021 .link-annotated}, [Yuqi Huo, Manli Zhang, Guangzhen Liu[, Haoyu Lu, Yizhao Gao, Guoxing Yang, Jingyuan Wen, Heng Zhang, Baogui Xu, Weihao Zheng, Zongzheng Xi, Yueqian Yang, Anwen Hu, Jinming Zhao, Ruichen Li, Yida Zhao, Liang Zhang, Yuqing Song, Xin Hong, Wanqing Cui, Danyang Hou, Yingyan Li, [Junyi Li](https://en.wikipedia.org/wiki/Junyi_Li "Junyi Li"){.id-not}, Peiyu Liu, Zheng Gong, Chuhao Jin, Yuchong Sun, Shizhe Chen, [Zhiwu Lu](https://en.wikipedia.org/wiki/Zhiwu_Lu "Zhiwu Lu"){.id-not}, Zhicheng Dou, Qin Jin, Yanyan Lan, Wayne Xin Zhao, Ruihua Song, Ji-Rong Wen]{.collapse}]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2103.06561.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2103.06561"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 165. `https://arxiv.org/abs/2103.01988#facebook`: [`“SEER: Self-Supervised Pretraining of Visual Features in the Wild”`{=html}](https://arxiv.org/abs/2103.01988#facebook "'SEER: Self-supervised Pretraining of Visual Features in the Wild', Goyal et al 2021"){#goyal-et-al-2021-seer .link-annotated}, [Priya Goyal, Mathilde Caron, Benjamin Lefaudeux[, [Min Xu](https://en.wikipedia.org/wiki/Min_Xu "Min Xu"){.id-not}, Pengchao Wang, Vivek Pai, Mannat Singh, Vitaliy Liptchinsky, [Ishan Misra](https://imisra.github.io/ "Ishan Misra"){.id-not}, [Armand Joulin](https://scholar.google.com/citations?user=kRJkDakAAAAJ "Armand Joulin"){.id-not}, [Piotr Bojanowski](https://scholar.google.com/citations?user=lJ_oh2EAAAAJ "Piotr Bojanowski"){.id-not}]{.collapse}]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2103.01988%2523facebook.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2103.01988#facebook"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 166. `https://arxiv.org/abs/2102.09672#openai`: [`“Improved Denoising Diffusion Probabilistic Models”`{=html}](https://arxiv.org/abs/2102.09672#openai "'Improved Denoising Diffusion Probabilistic Models', Nichol & Dhariwal 2021"){#nichol-dhariwal-2021 .link-annotated}, [Alex Nichol, [Prafulla Dhariwal](https://prafulladhariwal.com/ "Prafulla Dhariwal"){.link-annotated-partial .id-not}]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2102.09672%2523openai.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2102.09672#openai"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 167. `https://arxiv.org/abs/2102.05918#google`: [`“ALIGN: Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervision”`{=html}](https://arxiv.org/abs/2102.05918#google "'ALIGN: Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervision', Jia et al 2021"){#jia-et-al-2021 .link-annotated}, [Chao Jia, Yinfei Yang, [Ye Xia](https://en.wikipedia.org/wiki/Ye_Xia "Ye Xia"){.id-not}[, Yi-Ting Chen, Zarana Parekh, Hieu Pham, [Quoc V. Le](https://en.wikipedia.org/wiki/Quoc_V._Le "Quoc V. Le"){.id-not}, Yunhsuan Sung, Zhen Li, Tom Duerig]{.collapse}]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2102.05918%2523google.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2102.05918#google"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 168. `https://arxiv.org/abs/2102.06171#deepmind`: [`“NFNet: High-Performance Large-Scale Image Recognition Without Normalization”`{=html}](https://arxiv.org/abs/2102.06171#deepmind "'NFNet: High-Performance Large-Scale Image Recognition Without Normalization', Brock et al 2021"){#brock-et-al-2021 .link-annotated}, [[Andrew Brock](https://en.wikipedia.org/wiki/Andrew_Brock "Andrew Brock"){.id-not}, Soham De, [Samuel L. Smith](https://en.wikipedia.org/wiki/Samuel_L._Smith "Samuel L. Smith"){.id-not}, [Karen Simonyan](https://scholar.google.com/citations?user=L7lMQkQAAAAJ "Karen Simonyan"){.id-not}]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2102.06171%2523deepmind.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2102.06171#deepmind"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 169. `https://arxiv.org/abs/2102.02888#microsoft`: [`“1-Bit Adam: Communication Efficient Large-Scale Training With Adam’s Convergence Speed”`{=html}](https://arxiv.org/abs/2102.02888#microsoft "'1-bit Adam: Communication Efficient Large-Scale Training with Adam’s Convergence Speed', Tang et al 2021"){#tang-et-al-2021-1bitadam .link-annotated}, [Hanlin Tang, Shaoduo Gan, Ammar Ahmad Awan[, Samyam Rajbhandari, Conglong Li, Xiangru Lian, Ji Liu, Ce Zhang, Yuxiong He]{.collapse}]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2102.02888%2523microsoft.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2102.02888#microsoft"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 170. `https://arxiv.org/abs/2102.01951#scaling&org=deepmind`: [`“Mind the Gap: Assessing Temporal Generalization in Neural Language Models § Scaling”`{=html}](https://arxiv.org/abs/2102.01951#scaling&org=deepmind "'Mind the Gap: Assessing Temporal Generalization in Neural Language Models § Scaling', Lazaridou et al 2021"){#lazaridou-et-al-2021 .link-annotated}, [Angeliki Lazaridou, Adhiguna Kuncoro, Elena Gribovskaya[, Devang Agrawal, Adam Liska, Tayfun Terzi, Mai Gimenez, Cyprien de Masson d'Autume, Tomas Kocisky, Sebastian Ruder, [Dani Yogatama](https://dyogatama.github.io/ "Dani Yogatama"){.link-annotated-partial .id-not}, Kris Cao, Susannah Young, Phil Blunsom]{.collapse}]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2102.01951%2523scaling%2526org%253Ddeepmind.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2102.01951#scaling&org=deepmind"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 171. `https://arxiv.org/pdf/2102.01951#page=7&org=deepmind`: [`“Mind the Gap: Assessing Temporal Generalization in Neural Language Models § Dynamic Evaluation”`{=html}](https://arxiv.org/pdf/2102.01951#page=7&org=deepmind "'Mind the Gap: Assessing Temporal Generalization in Neural Language Models § Dynamic Evaluation', Lazaridou et al 2021 (page 7 org deepmind)"){#lazaridou-et-al-2021-page-7-org-deepmind .link-annotated}, [Angeliki Lazaridou, Adhiguna Kuncoro, Elena Gribovskaya[, Devang Agrawal, Adam Liska, Tayfun Terzi, Mai Gimenez, Cyprien de Masson d'Autume, Tomas Kocisky, Sebastian Ruder, [Dani Yogatama](https://dyogatama.github.io/ "Dani Yogatama"){.link-annotated-partial .id-not}, Kris Cao, Susannah Young, Phil Blunsom]{.collapse}]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fpdf%252F2102.01951%2523page%253D7%2526org%253Ddeepmind.html "Directory-tag link-bibliography for link https://arxiv.org/pdf/2102.01951#page=7&org=deepmind"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 172. `https://arxiv.org/abs/2003.10580#google`: [`“Meta Pseudo Labels”`{=html}](https://arxiv.org/abs/2003.10580#google "'Meta Pseudo Labels', Pham et al 2021"){#pham-et-al-2021-2 .link-annotated}, [Hieu Pham, [Zihang Dai](https://scholar.google.com/citations?user=uZqsVXkAAAAJ "Zihang Dai"){.id-not}, Qizhe Xie[, Minh-Thang Luong, [Quoc V. Le](https://en.wikipedia.org/wiki/Quoc_V._Le "Quoc V. Le"){.id-not}]{.collapse}]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2003.10580%2523google.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2003.10580#google"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 173. `https://cdn.openai.com/papers/Learning_Transferable_Visual_Models_From_Natural_Language_Supervision.pdf`: [`“CLIP: Learning Transferable Visual Models From Natural Language Supervision”`{=html}](https://cdn.openai.com/papers/Learning_Transferable_Visual_Models_From_Natural_Language_Supervision.pdf "'CLIP: Learning Transferable Visual Models From Natural Language Supervision', Radford et al 2021"){#radford-et-al-2021 .link-annotated}, [[Alec Radford](https://scholar.google.com/citations?user=dOad5HoAAAAJ "Alec Radford"){.id-not}, [Jong Wook Kim](https://jongwook.kim/ "Jong Wook Kim"){.link-annotated-partial .id-not}, Chris Hallacy[, [Aditya A. Ramesh](https://scholar.google.com/citations?user=60K82BkAAAAJ "Aditya A. Ramesh"){.id-not}, [Gabriel Goh](https://scholar.google.com/citations?user=2O3IZlkAAAAJ "Gabriel Goh"){.id-not}, [Sandhini Agarwal](https://scholar.google.com/citations?user=8UZIqcoAAAAJ "Sandhini Agarwal"){.id-not}, [Girish Sastry](https://girishsastry.com/ "Girish Sastry"){.link-annotated-partial .id-not}, [Amanda Askell](https://askell.io/ "About Me"){.link-annotated-partial .id-not}, [Pamela Mishkin](https://scholar.google.com/citations?user=fzLxuAIAAAAJ "Pamela Mishkin"){.id-not}, [Jack Clark](https://jack-clark.net/about/){.link-annotated-partial .id-not}, [Gretchen Krueger](https://scholar.google.com/citations?user=astFxkwAAAAJ "Gretchen Krueger"){.id-not}, [Ilya Sutskever](https://en.wikipedia.org/wiki/Ilya_Sutskever "Ilya Sutskever"){.link-annotated-partial .id-not}]{.collapse}]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Fcdn.openai.com%252Fpapers%252FLearning_Transferable_Visual_Models_From_Natural_Language_Supervision.pdf.html "Directory-tag link-bibliography for link https://cdn.openai.com/papers/Learning_Transferable_Visual_Models_From_Natural_Language_Supervision.pdf"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 174. `https://www.alignmentforum.org/posts/k2SNji3jXaLGhBeYP/extrapolating-gpt-n-performance`: [`“Extrapolating GPT-N Performance”`{=html}](https://www.alignmentforum.org/posts/k2SNji3jXaLGhBeYP/extrapolating-gpt-n-performance "'Extrapolating GPT-N performance', Finnveden 2020"){#finnveden-2020 .link-annotated}, [Lukas Finnveden]{.author .cite-author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Fwww.alignmentforum.org%252Fposts%252Fk2SNji3jXaLGhBeYP%252Fextrapolating-gpt-n-performance.html "Directory-tag link-bibliography for link https://www.alignmentforum.org/posts/k2SNji3jXaLGhBeYP/extrapolating-gpt-n-performance"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 175. `https://arxiv.org/abs/2012.00413`: [`“CPM: A Large-Scale Generative Chinese Pre-Trained Language Model”`{=html}](https://arxiv.org/abs/2012.00413 "'CPM: A Large-scale Generative Chinese Pre-trained Language Model', Zhang et al 2020"){#zhang-et-al-2020-04 .link-annotated}, [Zhengyan Zhang, [Xu Han](https://arxiv.org/search/cs?searchtype=author&query=Han,+X "Search"){.id-not}, [Hao Zhou](https://en.wikipedia.org/wiki/Hao_Zhou "Hao Zhou"){.id-not}[, Pei Ke, Yuxian Gu, Deming Ye, Yujia Qin, Yusheng Su, Haozhe Ji, Jian Guan, Fanchao Qi, Xiaozhi Wang, Yanan Zheng, Guoyang Zeng, Huanqi Cao, Shengqi Chen, Daixuan Li, Zhenbo Sun, [Zhiyuan Liu](https://nlp.csai.tsinghua.edu.cn/~lzy/ "Zhiyuan Liu"){.link-annotated-partial .id-not}, Minlie Huang, Wentao Han, [Jie Tang](https://en.wikipedia.org/wiki/Jie_Tang "Jie Tang"){.id-not}, Juanzi Li, Xiaoyan Zhu, [Maosong Sun](https://www.cs.tsinghua.edu.cn/csen/info/1312/4394.htm "Maosong SUN-Department of Computer Science and Technology, Tsinghua University"){.link-annotated-partial .id-not}]{.collapse}]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2012.00413.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2012.00413"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 176. `https://arxiv.org/abs/2011.10650#openai`: [`“Very Deep VAEs Generalize Autoregressive Models and Can Outperform Them on Images”`{=html}](https://arxiv.org/abs/2011.10650#openai "'Very Deep VAEs Generalize Autoregressive Models and Can Outperform Them on Images', Child 2020"){#child-2020 .link-annotated}, [[Rewon Child](https://rewonc.github.io/){.link-annotated-partial .id-not}]{.author .cite-author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2011.10650%2523openai.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2011.10650#openai"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 177. `https://arxiv.org/abs/2010.14701#openai`: [`“Scaling Laws for Autoregressive Generative Modeling”`{=html}](https://arxiv.org/abs/2010.14701#openai "'Scaling Laws for Autoregressive Generative Modeling', Henighan et al 2020"){#henighan-et-al-2020 .link-annotated}, [[Tom Henighan](https://www.tomhenighan.com/ "Tom Henighan"){.link-annotated-partial .id-not}, [Jared Kaplan](https://sites.krieger.jhu.edu/jared-kaplan/ "Jared Kaplan"){.link-annotated-partial .id-not}, Mor Katz[, [Mark Chen](https://event.technologyreview.com/emtech-mit-2023/speaker/901826/mark-chen "Speaker Details: EmTech MIT 2023"){.link-annotated-partial .id-not}, [Christopher Hesse](https://scholar.google.com/citations?user=SgbbTp4AAAAJ "Christopher Hesse"){.id-not}, Jacob Jackson, Heewoo Jun, [Tom B. Brown](https://scholar.google.com/citations?user=RLvsC94AAAAJ "Tom B. Brown"){.id-not}, [Prafulla Dhariwal](https://prafulladhariwal.com/ "Prafulla Dhariwal"){.link-annotated-partial .id-not}, [Scott Gray](https://scholar.google.com/citations?user=sRId4vsAAAAJ "Scott Gray"){.id-not}, Chris Hallacy, [Ben Mann](https://scholar.google.com/citations?user=McBoXK0AAAAJ "Benjamin Mann"){.id-not}, [Alec Radford](https://scholar.google.com/citations?user=dOad5HoAAAAJ "Alec Radford"){.id-not}, [Aditya A. Ramesh](https://scholar.google.com/citations?user=60K82BkAAAAJ "Aditya A. Ramesh"){.id-not}, [Nick Ryder](https://scholar.google.com/citations?user=rLp0oucAAAAJ "Nick Ryder"){.id-not}, [Daniel M. Ziegler](https://scholar.google.com/citations?user=YzfbfDgAAAAJ "Daniel M. Ziegler"){.id-not}, [John Schulman](http://joschu.net/ "'John Schulman’s Homepage', Schulman 2026"){.link-annotated-partial .id-not}, [Dario Amodei](https://en.wikipedia.org/wiki/Dario_Amodei "Dario Amodei"){.link-annotated-partial .id-not}, [Sam McCandlish](https://scholar.google.com/citations?user=gHp0pu4AAAAJ "Sam McCandlish"){.id-not}]{.collapse}]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2010.14701%2523openai.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2010.14701#openai"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 178. `https://arxiv.org/abs/2010.14571#google`: [`“Language ID in the Wild: Unexpected Challenges on the Path to a Thousand-Language Web Text Corpus”`{=html}](https://arxiv.org/abs/2010.14571#google "'Language ID in the Wild: Unexpected Challenges on the Path to a Thousand-Language Web Text Corpus', Caswell et al 2020"){#caswell-et-al-2020 .link-annotated}, [Isaac Caswell, Theresa Breiner, Daan van Esch, Ankur Bapna]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2010.14571%2523google.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2010.14571#google"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 179. `https://arxiv.org/abs/2010.10504#google`: [`“Pushing the Limits of Semi-Supervised Learning for Automatic Speech Recognition”`{=html}](https://arxiv.org/abs/2010.10504#google "'Pushing the Limits of Semi-Supervised Learning for Automatic Speech Recognition', Zhang et al 2020"){#zhang-et-al-2020-06 .link-annotated}, [[Yu Zhang](https://cse.sustech.edu.cn/faculty/~zhangy/ "Yu Zhang’s Homepage"){.link-annotated-partial .id-not}, James Qin, Daniel S. Park[, [Wei Han](https://en.wikipedia.org/wiki/Wei_Han "Wei Han"){.id-not}, Chung-Cheng Chiu, Ruoming Pang, [Quoc V. Le](https://en.wikipedia.org/wiki/Quoc_V._Le "Quoc V. Le"){.id-not}, [Yonghui Wu](https://scholar.google.com/citations?user=bMWMlk4AAAAJ "Yonghui Wu"){.id-not}]{.collapse}]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2010.10504%2523google.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2010.10504#google"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 180. `https://ai.meta.com/blog/introducing-many-to-many-multilingual-machine-translation/`: [`“The First AI Model That Translates 100 Languages without Relying on English Data”`{=html}](https://ai.meta.com/blog/introducing-many-to-many-multilingual-machine-translation/ "'The first AI model that translates 100 languages without relying on English data', Fan 2020"){#fan-2020 .link-annotated}, [Angela Fan]{.author .cite-author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Fai.meta.com%252Fblog%252Fintroducing-many-to-many-multilingual-machine-translation%252F.html "Directory-tag link-bibliography for link https://ai.meta.com/blog/introducing-many-to-many-multilingual-machine-translation/"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 181. `https://arxiv.org/abs/2010.11929#google`: [`“Vision Transformer: An Image Is Worth 16×16 Words: Transformers for Image Recognition at Scale”`{=html}](https://arxiv.org/abs/2010.11929#google "'Vision Transformer: An Image is Worth 16×16 Words: Transformers for Image Recognition at Scale', Dosovitskiy et al 2020"){#dosovitskiy-et-al-2020 .link-annotated}, [Alexey Dosovitskiy, [Lucas Beyer](https://scholar.google.com/citations?user=p2gwhK4AAAAJ "Lucas Beyer"){.id-not}, Alexander Kolesnikov[, Dirk Weissenborn, [Xiaohua Zhai](https://sites.google.com/view/xzhai "Xiaohua Zhai (翟晓华)"){.link-annotated-partial .id-not}, Thomas Unterthiner, [Mostafa Dehghani](https://scholar.google.com/citations?user=MiHOX3QAAAAJ "Mostafa Dehghani"){.id-not}, Matthias Minderer, Georg Heigold, Sylvain Gelly, [Jakob Uszkoreit](https://scholar.google.com/citations?user=mOG0bwsAAAAJ "Jakob Uszkoreit"){.id-not}, [Neil Houlsby](https://neilhoulsby.github.io/ "Neil Houlsby"){.link-annotated-partial .id-not}]{.collapse}]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2010.11929%2523google.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2010.11929#google"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 182. `https://www.openphilanthropy.org/research/new-report-on-how-much-computational-power-it-takes-to-match-the-human-brain/`: [`“New Report on How Much Computational Power It Takes to Match the Human Brain”`{=html}](https://www.openphilanthropy.org/research/new-report-on-how-much-computational-power-it-takes-to-match-the-human-brain/ "'New Report on How Much Computational Power It Takes to Match the Human Brain', Carlsmith 2020"){#carlsmith-2020 .link-annotated}, [Joseph Carlsmith]{.author .cite-author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Fwww.openphilanthropy.org%252Fresearch%252Fnew-report-on-how-much-computational-power-it-takes-to-match-the-human-brain%252F.html "Directory-tag link-bibliography for link https://www.openphilanthropy.org/research/new-report-on-how-much-computational-power-it-takes-to-match-the-human-brain/"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 183. `https://arxiv.org/abs/2009.03393#openai`: [`“Generative Language Modeling for Automated Theorem Proving”`{=html}](https://arxiv.org/abs/2009.03393#openai "'Generative Language Modeling for Automated Theorem Proving', Polu & Sutskever 2020"){#polu-sutskever-2020 .link-annotated}, [Stanislas Polu, [Ilya Sutskever](https://en.wikipedia.org/wiki/Ilya_Sutskever "Ilya Sutskever"){.link-annotated-partial .id-not}]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2009.03393%2523openai.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2009.03393#openai"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 184. `https://arxiv.org/abs/2008.09037`: [`“Accuracy and Performance Comparison of Video Action Recognition Approaches”`{=html}](https://arxiv.org/abs/2008.09037 "'Accuracy and Performance Comparison of Video Action Recognition Approaches', Hutchinson et al 2020"){#hutchinson-et-al-2020 .link-annotated}, [[Matthew Hutchinson](https://en.wikipedia.org/wiki/Matthew_Hutchinson "Matthew Hutchinson"){.id-not}, Siddharth Samsi, William Arcand[, David Bestor, Bill Bergeron, Chansup Byun, Michael Houle, Matthew Hubbell, Michael Jones, Jeremy Kepner, Andrew Kirby, Peter Michaleas, Lauren Milechin, Julie Mullen, [Andrew Prout](https://en.wikipedia.org/wiki/Andrew_Prout "Andrew Prout"){.id-not}, Antonio Rosa, Albert Reuther, Charles Yee, Vijay Gadepally]{.collapse}]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2008.09037.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2008.09037"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 185. `https://www.lesswrong.com/posts/Wnqua6eQkewL3bqsF/matt-botvinick-on-the-spontaneous-emergence-of-learning`: [`“Matt Botvinick on the Spontaneous Emergence of Learning Algorithms”`{=html}](https://www.lesswrong.com/posts/Wnqua6eQkewL3bqsF/matt-botvinick-on-the-spontaneous-emergence-of-learning "'Matt Botvinick on the spontaneous emergence of learning algorithms', Scholl 2020"){#scholl-2020 .link-annotated}, [Adam Scholl]{.author .cite-author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Fwww.lesswrong.com%252Fposts%252FWnqua6eQkewL3bqsF%252Fmatt-botvinick-on-the-spontaneous-emergence-of-learning.html "Directory-tag link-bibliography for link https://www.lesswrong.com/posts/Wnqua6eQkewL3bqsF/matt-botvinick-on-the-spontaneous-emergence-of-learning"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 186. `https://arxiv.org/abs/2008.02217`: [`“Hopfield Networks Is All You Need”`{=html}](https://arxiv.org/abs/2008.02217 "'Hopfield Networks is All You Need', Ramsauer et al 2020"){#ramsauer-et-al-2020 .link-annotated}, [Hubert Ramsauer, Bernhard Schäfl, Johannes Lehner[, Philipp Seidl, Michael Widrich, Lukas Gruber, Markus Holzleitner, Milena Pavlović, Geir Kjetil Sandve, Victor Greiff, David Kreil, Michael Kopp, Günter Klambauer, Johannes Brandstetter, [Sepp Hochreiter](https://en.wikipedia.org/wiki/Sepp_Hochreiter "Sepp Hochreiter"){.id-not}]{.collapse}]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2008.02217.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2008.02217"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 187. `https://arxiv.org/abs/2007.06225`: [`“ProtTrans: Towards Cracking the Language of Life’s Code Through Self-Supervised Deep Learning and High Performance Computing”`{=html}](https://arxiv.org/abs/2007.06225 "'ProtTrans: Towards Cracking the Language of Life’s Code Through Self-Supervised Deep Learning and High Performance Computing', Elnaggar et al 2020"){#elnaggar-et-al-2020 .link-annotated}, [[Ahmed Elnaggar](https://en.wikipedia.org/wiki/Ahmed_Elnaggar "Ahmed Elnaggar"){.id-not}, Michael Heinzinger, Christian Dallago[, Ghalia Rihawi, Yu Wang, [Llion Jones](https://scholar.google.com/citations?user=_3_P5VwAAAAJ "Llion Jones"){.id-not}, Tom Gibbs, Tamas Feher, Christoph Angerer, [Martin Steinegger](https://en.wikipedia.org/wiki/Martin_Steinegger "Martin Steinegger"){.id-not}, Debsindhu Bhowmik, [Burkhard Rost](https://en.wikipedia.org/wiki/Burkhard_Rost "Burkhard Rost"){.id-not}]{.collapse}]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2007.06225.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2007.06225"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 188. `https://arxiv.org/abs/2007.03898#nvidia`: [`“NVAE: A Deep Hierarchical Variational Autoencoder”`{=html}](https://arxiv.org/abs/2007.03898#nvidia "'NVAE: A Deep Hierarchical Variational Autoencoder', Vahdat & Kautz 2020"){#vahdat-kautz-2020 .link-annotated}, [Arash Vahdat, Jan Kautz]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2007.03898%2523nvidia.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2007.03898#nvidia"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 189. `https://arxiv.org/abs/2006.11477#facebook`: [`“Wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations”`{=html}](https://arxiv.org/abs/2006.11477#facebook "'wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations', Baevski et al 2020"){#baevski-et-al-2020 .link-annotated}, [Alexei Baevski, Henry Zhou, Abdelrahman Mohamed, Michael Auli]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2006.11477%2523facebook.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2006.11477#facebook"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 190. `https://arxiv.org/abs/2006.10621`: [`“On the Predictability of Pruning Across Scales”`{=html}](https://arxiv.org/abs/2006.10621 "'On the Predictability of Pruning Across Scales', Rosenfeld et al 2020"){#rosenfeld-et-al-2020 .link-annotated}, [[Jonathan S. Rosenfeld](https://scholar.google.com/citations?user=8HYnMeYAAAAJ&view_op=list_works&sortby=pubdate "Jonathan S. Rosenfeld"){.id-not}, [Jonathan Frankle](http://www.jfrankle.com/ "Jonathan Frankle—Chief Neural network Scientist at Databricks"){.link-annotated-partial .id-not}, [Michael Carbin](https://people.csail.mit.edu/mcarbin/ "Michael Carbin"){.link-annotated-partial .id-not}, [Nir Shavit](https://en.wikipedia.org/wiki/Nir_Shavit "Nir Shavit"){.id-not}]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2006.10621.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2006.10621"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 191. `2020-chen-2.pdf#openai`: [`“IGPT: Generative Pretraining from Pixels”`{=html}](/doc/ai/nn/transformer/gpt/dall-e/1/2020-chen-2.pdf#openai "'iGPT: Generative Pretraining from Pixels', Chen et al 2020"){#chen-igpt-paper .link-annotated}, [[Mark Chen](https://event.technologyreview.com/emtech-mit-2023/speaker/901826/mark-chen "Speaker Details: EmTech MIT 2023"){.link-annotated-partial .id-not}, [Alec Radford](https://scholar.google.com/citations?user=dOad5HoAAAAJ "Alec Radford"){.id-not}, [Rewon Child](https://rewonc.github.io/){.link-annotated-partial .id-not}[, Jeff Wu, Heewoo Jun, [Prafulla Dhariwal](https://prafulladhariwal.com/ "Prafulla Dhariwal"){.link-annotated-partial .id-not}, David Luan, [Ilya Sutskever](https://en.wikipedia.org/wiki/Ilya_Sutskever "Ilya Sutskever"){.link-annotated-partial .id-not}]{.collapse}]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/%252Fdoc%252Fai%252Fnn%252Ftransformer%252Fgpt%252Fdall-e%252F1%252F2020-chen-2.pdf%2523openai.html "Directory-tag link-bibliography for link /doc/ai/nn/transformer/gpt/dall-e/1/2020-chen-2.pdf#openai"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 192. `https://arxiv.org/abs/2006.09882#facebook`: [`“SwAV: Unsupervised Learning of Visual Features by Contrasting Cluster Assignments”`{=html}](https://arxiv.org/abs/2006.09882#facebook "'SwAV: Unsupervised Learning of Visual Features by Contrasting Cluster Assignments', Caron et al 2020"){#caron-et-al-2020 .link-annotated}, [Mathilde Caron, [Ishan Misra](https://imisra.github.io/ "Ishan Misra"){.id-not}, Julien Mairal[, Priya Goyal, [Piotr Bojanowski](https://scholar.google.com/citations?user=lJ_oh2EAAAAJ "Piotr Bojanowski"){.id-not}, [Armand Joulin](https://scholar.google.com/citations?user=kRJkDakAAAAJ "Armand Joulin"){.id-not}]{.collapse}]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2006.09882%2523facebook.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2006.09882#facebook"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 193. `https://openai.com/index/image-gpt/`: [`“Image GPT (IGPT): We Find That, Just As a Large Transformer Model Trained on Language Can Generate Coherent Text, the Same Exact Model Trained on Pixel Sequences Can Generate Coherent Image Completions and Samples”`{=html}](https://openai.com/index/image-gpt/ "'Image GPT (iGPT): We find that, just as a large transformer model trained on language can generate coherent text, the same exact model trained on pixel sequences can generate coherent image completions and samples', Chen et al 2020"){#chen-et-al-2020-blog .link-annotated}, [[Mark Chen](https://event.technologyreview.com/emtech-mit-2023/speaker/901826/mark-chen "Speaker Details: EmTech MIT 2023"){.link-annotated-partial .id-not}, [Alec Radford](https://scholar.google.com/citations?user=dOad5HoAAAAJ "Alec Radford"){.id-not}, [Ilya Sutskever](https://en.wikipedia.org/wiki/Ilya_Sutskever "Ilya Sutskever"){.link-annotated-partial .id-not}]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Fopenai.com%252Findex%252Fimage-gpt%252F.html "Directory-tag link-bibliography for link https://openai.com/index/image-gpt/"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 194. `https://www.microsoft.com/en-us/research/blog/zero-2-deepspeed-shattering-barriers-of-deep-learning-speed-scale/`: [`“ZeRO-2 & DeepSpeed: Shattering Barriers of Deep Learning Speed & Scale”`{=html}](https://www.microsoft.com/en-us/research/blog/zero-2-deepspeed-shattering-barriers-of-deep-learning-speed-scale/ "'ZeRO-2 & DeepSpeed: Shattering barriers of deep learning speed & scale', Team 2020"){#team-2020-1 .link-annotated}, [DeepSpeed Team]{.author .cite-author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Fwww.microsoft.com%252Fen-us%252Fresearch%252Fblog%252Fzero-2-deepspeed-shattering-barriers-of-deep-learning-speed-scale%252F.html "Directory-tag link-bibliography for link https://www.microsoft.com/en-us/research/blog/zero-2-deepspeed-shattering-barriers-of-deep-learning-speed-scale/"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 195. `https://openai.com/research/jukebox`: [`“Jukebox: We’re Introducing Jukebox, a Neural Net That Generates Music, including Rudimentary Singing, As Raw Audio in a Variety of Genres and Artist Styles. We’re Releasing the Model Weights and Code, along With a Tool to Explore the Generated Samples.”`{=html}](https://openai.com/research/jukebox "'Jukebox: We’re introducing Jukebox, a neural net that generates music, including rudimentary singing, as raw audio in a variety of genres and artist styles. We’re releasing the model weights and code, along with a tool to explore the generated samples.', Dhariwal et al 2020"){#jukebox-blog .link-annotated}, [[Prafulla Dhariwal](https://prafulladhariwal.com/ "Prafulla Dhariwal"){.link-annotated-partial .id-not}, Heewoo Jun, [Christine Payne](https://en.wikipedia.org/wiki/Christine_Payne "Christine Payne"){.id-not}[, [Jong Wook Kim](https://jongwook.kim/ "Jong Wook Kim"){.link-annotated-partial .id-not}, [Alec Radford](https://scholar.google.com/citations?user=dOad5HoAAAAJ "Alec Radford"){.id-not}, [Ilya Sutskever](https://en.wikipedia.org/wiki/Ilya_Sutskever "Ilya Sutskever"){.link-annotated-partial .id-not}]{.collapse}]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Fopenai.com%252Fresearch%252Fjukebox.html "Directory-tag link-bibliography for link https://openai.com/research/jukebox"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 196. `https://ai.meta.com/blog/state-of-the-art-open-source-chatbot/`: [`“Blender: A State-Of-The-Art Open Source Chatbot”`{=html}](https://ai.meta.com/blog/state-of-the-art-open-source-chatbot/ "'Blender: A state-of-the-art open source chatbot', Roller et al 2020"){#blender-blog .link-annotated}, [Stephen Roller, [Jason Weston](https://en.wikipedia.org/wiki/Jason_Weston "Jason Weston"){.id-not}, Emily Dinan]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Fai.meta.com%252Fblog%252Fstate-of-the-art-open-source-chatbot%252F.html "Directory-tag link-bibliography for link https://ai.meta.com/blog/state-of-the-art-open-source-chatbot/"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 197. `https://arxiv.org/abs/2004.08366#google`: [`“DynamicEmbedding: Extending TensorFlow for Colossal-Scale Applications”`{=html}](https://arxiv.org/abs/2004.08366#google "'DynamicEmbedding: Extending TensorFlow for Colossal-Scale Applications', Zeng et al 2020"){#zeng-et-al-2020 .link-annotated}, [Yun Zeng, Siqi Zuo, Dongcai Shen]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2004.08366%2523google.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2004.08366#google"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 198. `https://arxiv.org/abs/2004.07159#alibaba`: [`“PALM: Pre-Training an Autoencoding & Autoregressive Language Model for Context-Conditioned Generation”`{=html}](https://arxiv.org/abs/2004.07159#alibaba "'PALM: Pre-training an Autoencoding & Autoregressive Language Model for Context-conditioned Generation', Bi et al 2020"){#bi-et-al-2020 .link-annotated}, [Bin Bi, Chenliang Li, Chen Wu[, [Ming Yan](https://en.wikipedia.org/wiki/Ming_Yan "Ming Yan"){.id-not}, [Wei Wang](https://web.cs.ucla.edu/~weiwang/ "Wei Wang’s Home Page"){.link-annotated-partial .id-not}, Songfang Huang, Fei Huang, Luo Si]{.collapse}]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2004.07159%2523alibaba.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2004.07159#alibaba"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 199. `https://www.technologyreview.com/2020/02/17/844721/ai-openai-moonshot-elon-musk-sam-altman-greg-brockman-messy-secretive-reality/`: [`“The Messy, Secretive Reality behind OpenAI’s Bid to save the World: The AI Moonshot Was Founded in the Spirit of Transparency. This Is the inside Story of How Competitive Pressure Eroded That Idealism”`{=html}](https://www.technologyreview.com/2020/02/17/844721/ai-openai-moonshot-elon-musk-sam-altman-greg-brockman-messy-secretive-reality/ "'The messy, secretive reality behind OpenAI’s bid to save the world: The AI moonshot was founded in the spirit of transparency. This is the inside story of how competitive pressure eroded that idealism', Hao 2020"){#hao-2020 .link-annotated}, [[Karen Hao](https://en.wikipedia.org/wiki/Karen_Hao "Karen Hao"){.id-not}]{.author .cite-author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Fwww.technologyreview.com%252F2020%252F02%252F17%252F844721%252Fai-openai-moonshot-elon-musk-sam-altman-greg-brockman-messy-secretive-reality%252F.html "Directory-tag link-bibliography for link https://www.technologyreview.com/2020/02/17/844721/ai-openai-moonshot-elon-musk-sam-altman-greg-brockman-messy-secretive-reality/"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 200. `https://arxiv.org/abs/2002.05709#google`: [`“A Simple Framework for Contrastive Learning of Visual Representations”`{=html}](https://arxiv.org/abs/2002.05709#google "'A Simple Framework for Contrastive Learning of Visual Representations', Chen et al 2020"){#chen-et-al-2020-simclrv1 .link-annotated}, [Ting Chen, [Simon Kornblith](https://www.zotero.org/simon/cv "'Simon Kornblith CV', Kornblith 2026"){.id-not}, [Mohammad Norouzi](https://en.wikipedia.org/wiki/Mohammad_Norouzi "Mohammad Norouzi"){.id-not}, [Geoffrey Hinton](https://en.wikipedia.org/wiki/Geoffrey_Hinton "Geoffrey Hinton"){.link-annotated-partial .id-not}]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2002.05709%2523google.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2002.05709#google"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 201. `https://www.microsoft.com/en-us/research/blog/turing-nlg-a-17-billion-parameter-language-model-by-microsoft/`: [`“Turing-NLG: A 17-Billion-Parameter Language Model by Microsoft”`{=html}](https://www.microsoft.com/en-us/research/blog/turing-nlg-a-17-billion-parameter-language-model-by-microsoft/ "'Turing-NLG: A 17-billion-parameter language model by Microsoft', Rosset 2020"){#rosset-2020 .link-annotated}, [Corby Rosset]{.author .cite-author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Fwww.microsoft.com%252Fen-us%252Fresearch%252Fblog%252Fturing-nlg-a-17-billion-parameter-language-model-by-microsoft%252F.html "Directory-tag link-bibliography for link https://www.microsoft.com/en-us/research/blog/turing-nlg-a-17-billion-parameter-language-model-by-microsoft/"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 202. `https://research.google/blog/towards-a-conversational-agent-that-can-chat-aboutanything/`: [`“Towards a Conversational Agent That Can Chat About…Anything”`{=html}](https://research.google/blog/towards-a-conversational-agent-that-can-chat-aboutanything/ "'Towards a Conversational Agent that Can Chat About…Anything', Adiwardana & Luong 2020"){#adiwardana-luong-2020 .link-annotated}, [Daniel Adiwardana, Thang Luong]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Fresearch.google%252Fblog%252Ftowards-a-conversational-agent-that-can-chat-aboutanything%252F.html "Directory-tag link-bibliography for link https://research.google/blog/towards-a-conversational-agent-that-can-chat-aboutanything/"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 203. `https://arxiv.org/abs/2001.08361#openai`: [`“Scaling Laws for Neural Language Models”`{=html}](https://arxiv.org/abs/2001.08361#openai "'Scaling Laws for Neural Language Models', Kaplan et al 2020"){#kaplan-et-al-2020 .link-annotated}, [[Jared Kaplan](https://sites.krieger.jhu.edu/jared-kaplan/ "Jared Kaplan"){.link-annotated-partial .id-not}, [Sam McCandlish](https://scholar.google.com/citations?user=gHp0pu4AAAAJ "Sam McCandlish"){.id-not}, [Tom Henighan](https://www.tomhenighan.com/ "Tom Henighan"){.link-annotated-partial .id-not}[, [Tom B. Brown](https://scholar.google.com/citations?user=RLvsC94AAAAJ "Tom B. Brown"){.id-not}, [Ben Chess](https://www.linkedin.com/in/ben-chess-69bba21 "Crusoe"){.link-annotated-partial .id-not}, [Rewon Child](https://rewonc.github.io/){.link-annotated-partial .id-not}, [Scott Gray](https://scholar.google.com/citations?user=sRId4vsAAAAJ "Scott Gray"){.id-not}, [Alec Radford](https://scholar.google.com/citations?user=dOad5HoAAAAJ "Alec Radford"){.id-not}, [Jeffrey Wu](https://scholar.google.com/citations?user=x4JAvwMAAAAJ "Jeffrey Wu"){.id-not}, [Dario Amodei](https://en.wikipedia.org/wiki/Dario_Amodei "Dario Amodei"){.link-annotated-partial .id-not}]{.collapse}]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2001.08361%2523openai.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2001.08361#openai"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 204. `https://www.youtube.com/watch?v=kY2NHSKBi10`: [`“The Importance of Deconstruction”`{=html}](https://www.youtube.com/watch?v=kY2NHSKBi10 "'The Importance of Deconstruction', Weinberger 2020"){#weinberger-2020 .link-annotated}, [[Kilian Q. Weinberger](https://www.cs.cornell.edu/~kilian/ "Welcome"){.link-annotated-partial .id-not}]{.author .cite-author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Fwww.youtube.com%252Fwatch%253Fv%253DkY2NHSKBi10.html "Directory-tag link-bibliography for link https://www.youtube.com/watch?v=kY2NHSKBi10"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 205. `https://openai.com/research/deep-double-descent`: [`“Deep Double Descent: We Show That the Double Descent Phenomenon Occurs in CNNs, ResNets, and Transformers: Performance First Improves, Then Gets Worse, and Then Improves Again With Increasing Model Size, Data Size, or Training Time”`{=html}](https://openai.com/research/deep-double-descent "'Deep Double Descent: We show that the double descent phenomenon occurs in CNNs, ResNets, and transformers: performance first improves, then gets worse, and then improves again with increasing model size, data size, or training time', Nakkiran et al 2019"){#nakkiran-et-al-2019-1 .link-annotated}, [[Preetum Nakkiran](https://preetum.nakkiran.org/ "Preetum Nakkiran"){.id-not}, [Gal Kaplun](https://www.galkaplun.com/ "Gal Kaplun"){.link-annotated-partial .id-not}, [Yamini Bansal](https://scholar.google.com/citations?user=uj1OljkAAAAJ "Yamini Bansal"){.id-not}[, [Tristan Yang](https://scholar.google.com/scholar?q=Tristan%20Yang){.id-not}, [Boaz Barak](https://en.wikipedia.org/wiki/Boaz_Barak "Boaz Barak"){.id-not}, [Ilya Sutskever](https://en.wikipedia.org/wiki/Ilya_Sutskever "Ilya Sutskever"){.link-annotated-partial .id-not}]{.collapse}]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Fopenai.com%252Fresearch%252Fdeep-double-descent.html "Directory-tag link-bibliography for link https://openai.com/research/deep-double-descent"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 206. `https://arxiv.org/abs/1911.13299`: [`“What’s Hidden in a Randomly Weighted Neural Network?”`{=html}](https://arxiv.org/abs/1911.13299 "'What’s Hidden in a Randomly Weighted Neural Network?', Ramanujan et al 2019"){#ramanujan-et-al-2019 .link-annotated}, [Vivek Ramanujan, [Mitchell Wortsman](https://mitchellnw.github.io/ "Mitchell Wortsman"){.id-not}, Aniruddha Kembhavi[, [Ali Farhadi](https://en.wikipedia.org/wiki/Ali_Farhadi "Ali Farhadi"){.id-not}, Mohammad Rastegari]{.collapse}]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F1911.13299.html "Directory-tag link-bibliography for link https://arxiv.org/abs/1911.13299"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 207. `https://arxiv.org/abs/1911.05722#facebook`: [`“Momentum Contrast for Unsupervised Visual Representation Learning”`{=html}](https://arxiv.org/abs/1911.05722#facebook "'Momentum Contrast for Unsupervised Visual Representation Learning', He et al 2019"){#he-et-al-2019-1 .link-annotated}, [[Kaiming He](https://en.wikipedia.org/wiki/Kaiming_He "Kaiming He"){.id-not}, Haoqi Fan, Yuxin Wu[, Saining Xie, [Ross Girshick](https://scholar.google.com/citations?user=W8VIEZgAAAAJ "Ross Girshick"){.id-not}]{.collapse}]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F1911.05722%2523facebook.html "Directory-tag link-bibliography for link https://arxiv.org/abs/1911.05722#facebook"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 208. `https://arxiv.org/abs/1911.04252#google`: [`“Self-Training With Noisy Student Improves ImageNet Classification”`{=html}](https://arxiv.org/abs/1911.04252#google "'Self-training with Noisy Student improves ImageNet classification', Xie et al 2019"){#xie-et-al-2019-2 .link-annotated}, [Qizhe Xie, Minh-Thang Luong, [Eduard Hovy](https://en.wikipedia.org/wiki/Eduard_Hovy "Eduard Hovy"){.id-not}, [Quoc V. Le](https://en.wikipedia.org/wiki/Quoc_V._Le "Quoc V. Le"){.id-not}]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F1911.04252%2523google.html "Directory-tag link-bibliography for link https://arxiv.org/abs/1911.04252#google"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 209. `https://arxiv.org/abs/1911.02116#facebook`: [`“Unsupervised Cross-Lingual Representation Learning at Scale”`{=html}](https://arxiv.org/abs/1911.02116#facebook "'Unsupervised Cross-lingual Representation Learning at Scale', Conneau et al 2019"){#conneau-et-al-2019 .link-annotated}, [Alexis Conneau, Kartikay Khandelwal, [Naman Goyal](https://namangoyal.com/ "Naman Goyal"){.link-annotated-partial .id-not}[, Vishrav Chaudhary, Guillaume Wenzek, Francisco Guzmán, Edouard Grave, Myle Ott, [Luke Zettlemoyer](https://scholar.google.com/citations?user=UjpbO6IAAAAJ "Luke Zettlemoyer"){.id-not}, [Veselin Stoyanov](https://en.wikipedia.org/wiki/Veselin_Stoyanov "Veselin Stoyanov"){.id-not}]{.collapse}]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F1911.02116%2523facebook.html "Directory-tag link-bibliography for link https://arxiv.org/abs/1911.02116#facebook"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 210. `https://arxiv.org/abs/1910.02054#microsoft`: [`“ZeRO: Memory Optimizations Toward Training Trillion Parameter Models”`{=html}](https://arxiv.org/abs/1910.02054#microsoft "'ZeRO: Memory Optimizations Toward Training Trillion Parameter Models', Rajbhandari et al 2019"){#rajbhandari-et-al-2019 .link-annotated}, [Samyam Rajbhandari, Jeff Rasley, Olatunji Ruwase, Yuxiong He]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F1910.02054%2523microsoft.html "Directory-tag link-bibliography for link https://arxiv.org/abs/1910.02054#microsoft"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 211. `https://arxiv.org/abs/1909.11740`: [`“UNITER: UNiversal Image-TExt Representation Learning”`{=html}](https://arxiv.org/abs/1909.11740 "'UNITER: UNiversal Image-TExt Representation Learning', Chen et al 2019"){#chen-et-al-2019-3 .link-annotated}, [Yen-Chun Chen, Linjie Li, Licheng Yu[, Ahmed El Kholy, Faisal Ahmed, [Zhe Gan](https://zhegan27.github.io/ "Zhe Gan"){.id-not}, Yu Cheng, Jingjing Liu]{.collapse}]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F1909.11740.html "Directory-tag link-bibliography for link https://arxiv.org/abs/1909.11740"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 212. `https://arxiv.org/abs/1909.05858#salesforce`: [`“CTRL: A Conditional Transformer Language Model For Controllable Generation”`{=html}](https://arxiv.org/abs/1909.05858#salesforce "'CTRL: A Conditional Transformer Language Model For Controllable Generation', Keskar et al 2019"){#keskar-et-al-2019 .link-annotated}, [[Nitish Shirish Keskar](https://keskarnitish.github.io/ "Nitish Shirish Keskar"){.id-not}, Bryan McCann, Lav R. Varshney[, [Caiming Xiong](http://cmxiong.com/ "Caiming Xiong—Home Page"){.link-annotated-partial .id-not}, [Richard Socher](https://www.socher.org/ "Richard Socher"){.id-not}]{.collapse}]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F1909.05858%2523salesforce.html "Directory-tag link-bibliography for link https://arxiv.org/abs/1909.05858#salesforce"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 213. `https://nv-adlr.github.io/MegatronLM`: [`“MegatronLM: Training Billion+ Parameter Language Models Using GPU Model Parallelism”`{=html}](https://nv-adlr.github.io/MegatronLM "'MegatronLM: Training Billion+ Parameter Language Models Using GPU Model Parallelism', ADLR 2019"){#adlr-2019 .link-annotated}, [NVID I. A. ADLR]{.author .cite-author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Fnv-adlr.github.io%252FMegatronLM.html "Directory-tag link-bibliography for link https://nv-adlr.github.io/MegatronLM"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 214. `https://arxiv.org/abs/1907.11692#facebook`: [`“RoBERTa: A Robustly Optimized BERT Pretraining Approach”`{=html}](https://arxiv.org/abs/1907.11692#facebook "'RoBERTa: A Robustly Optimized BERT Pretraining Approach', Liu et al 2019"){#liu-et-al-2019-roberta .link-annotated}, [Yinhan Liu, Myle Ott, [Naman Goyal](https://namangoyal.com/ "Naman Goyal"){.link-annotated-partial .id-not}[, Jingfei Du, Mandar Joshi, [Danqi Chen](https://en.wikipedia.org/wiki/Danqi_Chen "Danqi Chen"){.id-not}, [Omer Levy](https://scholar.google.com/citations?user=PZVd2h8AAAAJ "Omer Levy"){.id-not}, [Mike Lewis](https://scholar.google.com/citations?user=SnQnQicAAAAJ "Mike Lewis"){.id-not}, [Luke Zettlemoyer](https://scholar.google.com/citations?user=UjpbO6IAAAAJ "Luke Zettlemoyer"){.id-not}, [Veselin Stoyanov](https://en.wikipedia.org/wiki/Veselin_Stoyanov "Veselin Stoyanov"){.id-not}]{.collapse}]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F1907.11692%2523facebook.html "Directory-tag link-bibliography for link https://arxiv.org/abs/1907.11692#facebook"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 215. `https://arxiv.org/abs/1907.07640`: [`“Robustness Properties of Facebook’s ResNeXt WSL Models”`{=html}](https://arxiv.org/abs/1907.07640 "'Robustness properties of Facebook’s ResNeXt WSL models', Orhan 2019"){#orhan-2019 .link-annotated}, [A. Emin Orhan]{.author .cite-author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F1907.07640.html "Directory-tag link-bibliography for link https://arxiv.org/abs/1907.07640"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 216. `https://arxiv.org/abs/1907.02544`: [`“Large Scale Adversarial Representation Learning”`{=html}](https://arxiv.org/abs/1907.02544 "'Large Scale Adversarial Representation Learning', Donahue & Simonyan 2019"){#donahue-simonyan-2019 .link-annotated}, [[Jeff Donahue](https://jeffdonahue.com/ "Jeff Donahue"){.id-not}, [Karen Simonyan](https://scholar.google.com/citations?user=L7lMQkQAAAAJ "Karen Simonyan"){.id-not}]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F1907.02544.html "Directory-tag link-bibliography for link https://arxiv.org/abs/1907.02544"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 217. `https://arxiv.org/abs/1906.06669`: [`“One Epoch Is All You Need”`{=html}](https://arxiv.org/abs/1906.06669 "'One Epoch Is All You Need', Komatsuzaki 2019"){#komatsuzaki-2019 .link-annotated}, [Aran Komatsuzaki]{.author .cite-author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F1906.06669.html "Directory-tag link-bibliography for link https://arxiv.org/abs/1906.06669"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 218. `https://david-abel.github.io/notes/icml_2019.pdf`: [`“ICML 2019 Notes”`{=html}](https://david-abel.github.io/notes/icml_2019.pdf "'ICML 2019 Notes', Abel 2019"){#abel-2019 .link-annotated}, [David Abel]{.author .cite-author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Fdavid-abel.github.io%252Fnotes%252Ficml_2019.pdf.html "Directory-tag link-bibliography for link https://david-abel.github.io/notes/icml_2019.pdf"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 219. `https://arxiv.org/abs/1905.11946#google`: [`“EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks”`{=html}](https://arxiv.org/abs/1905.11946#google "'EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks', Tan & Le 2019"){#tan-le-2019 .link-annotated}, [Mingxing Tan, [Quoc V. Le](https://en.wikipedia.org/wiki/Quoc_V._Le "Quoc V. Le"){.id-not}]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F1905.11946%2523google.html "Directory-tag link-bibliography for link https://arxiv.org/abs/1905.11946#google"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 220. `https://arxiv.org/abs/1905.10843`: [`“Asymptotic Learning Curves of Kernel Methods: Empirical Data versus Teacher-Student Paradigm”`{=html}](https://arxiv.org/abs/1905.10843 "'Asymptotic learning curves of kernel methods: empirical data versus Teacher-Student paradigm', Spigler et al 2019"){#spigler-et-al-2019 .link-annotated}, [Stefano Spigler, Mario Geiger, Matthieu Wyart]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F1905.10843.html "Directory-tag link-bibliography for link https://arxiv.org/abs/1905.10843"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 221. `https://arxiv.org/abs/1905.03197`: [`“UniLM: Unified Language Model Pre-Training for Natural Language Understanding and Generation”`{=html}](https://arxiv.org/abs/1905.03197 "'UniLM: Unified Language Model Pre-training for Natural Language Understanding and Generation', Dong et al 2019"){#dong-et-al-2019 .link-annotated}, [[Li Dong](https://scholar.google.com/citations?user=wEfQgPgAAAAJ "Li Dong"){.id-not}, Nan Yang, Wenhui Wang[, [Furu Wei](https://scholar.google.com/citations?user=G-V1VpwAAAAJ "Furu Wei"){.id-not}, [Xiaodong Liu](https://scholar.google.com/citations?user=NIewcxMAAAAJ "Xiaodong Liu"){.id-not}, Yu Wang, [Jianfeng Gao](https://www.microsoft.com/en-us/research/people/jfgao/ "Jianfeng Gao at Microsoft Research"){.link-annotated-partial .id-not}, Ming Zhou, [Hsiao-Wuen Hon](https://en.wikipedia.org/wiki/Hsiao-Wuen_Hon "Hsiao-Wuen Hon"){.id-not}]{.collapse}]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F1905.03197.html "Directory-tag link-bibliography for link https://arxiv.org/abs/1905.03197"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 222. `https://arxiv.org/abs/1905.00546#facebook`: [`“Billion-Scale Semi-Supervised Learning for Image Classification”`{=html}](https://arxiv.org/abs/1905.00546#facebook "'Billion-scale semi-supervised learning for image classification', Yalniz et al 2019"){#yalniz-et-al-2019 .link-annotated}, [I. Zeki Yalniz, Hervé Jégou, Kan Chen[, Manohar Paluri, Dhruv Mahajan]{.collapse}]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F1905.00546%2523facebook.html "Directory-tag link-bibliography for link https://arxiv.org/abs/1905.00546#facebook"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 223. `http://www.incompleteideas.net/IncIdeas/BitterLesson.html`: [`“The Bitter Lesson”`{=html}](http://www.incompleteideas.net/IncIdeas/BitterLesson.html "'The Bitter Lesson', Sutton 2019"){#sutton-2019-2 .link-annotated}, [[Rich Sutton](https://en.wikipedia.org/wiki/Rich_Sutton "Rich Sutton"){.id-not}]{.author .cite-author} [[link-bibliography](/metadata/annotation/link-bibliography/http%253A%252F%252Fwww.incompleteideas.net%252FIncIdeas%252FBitterLesson.html.html "Directory-tag link-bibliography for link http://www.incompleteideas.net/IncIdeas/BitterLesson.html"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 224. `https://openai.com/index/better-language-models/`: [`“Better Language Models and Their Implications”`{=html}](https://openai.com/index/better-language-models/ "'Better Language Models and Their Implications', Radford et al 2019"){#gpt-2-blog .link-annotated}, [[Alec Radford](https://scholar.google.com/citations?user=dOad5HoAAAAJ "Alec Radford"){.id-not}, [Jeffrey Wu](https://scholar.google.com/citations?user=x4JAvwMAAAAJ "Jeffrey Wu"){.id-not}, [Dario Amodei](https://en.wikipedia.org/wiki/Dario_Amodei "Dario Amodei"){.link-annotated-partial .id-not}[, [Daniela Amodei](https://en.wikipedia.org/wiki/Daniela_Amodei "Daniela Amodei"){.id-not}, [Jack Clark](https://jack-clark.net/about/){.link-annotated-partial .id-not}, [Miles Brundage](https://www.milesbrundage.com/ "Miles Brundage—About Me"){.link-annotated-partial .id-not}, [Ilya Sutskever](https://en.wikipedia.org/wiki/Ilya_Sutskever "Ilya Sutskever"){.link-annotated-partial .id-not}]{.collapse}]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Fopenai.com%252Findex%252Fbetter-language-models%252F.html "Directory-tag link-bibliography for link https://openai.com/index/better-language-models/"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 225. `https://melaniemitchell.me/aibook/`: [`“Artificial Intelligence: A Guide for Thinking Humans § Prologue: Terrified”`{=html}](https://melaniemitchell.me/aibook/ "'Artificial Intelligence: A Guide for Thinking Humans § Prologue: Terrified', Mitchell 2019"){#mitchell-2019 .link-annotated}, [[Melanie Mitchell](https://en.wikipedia.org/wiki/Melanie_Mitchell "Melanie Mitchell"){.id-not}]{.author .cite-author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Fmelaniemitchell.me%252Faibook%252F.html "Directory-tag link-bibliography for link https://melaniemitchell.me/aibook/"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 226. `https://openai.com/research/how-ai-training-scales`: [`“How AI Training Scales”`{=html}](https://openai.com/research/how-ai-training-scales "'How AI Training Scales', McCandlish et al 2018"){#mccandlish-et-al-2018 .link-annotated}, [[Sam McCandlish](https://scholar.google.com/citations?user=gHp0pu4AAAAJ "Sam McCandlish"){.id-not}, [Jared Kaplan](https://sites.krieger.jhu.edu/jared-kaplan/ "Jared Kaplan"){.link-annotated-partial .id-not}, [Dario Amodei](https://en.wikipedia.org/wiki/Dario_Amodei "Dario Amodei"){.link-annotated-partial .id-not}]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Fopenai.com%252Fresearch%252Fhow-ai-training-scales.html "Directory-tag link-bibliography for link https://openai.com/research/how-ai-training-scales"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 227. `https://slatestarcodex.com/2018/11/26/is-science-slowing-down-2/`: [`“Is Science Slowing Down?”`{=html}](https://slatestarcodex.com/2018/11/26/is-science-slowing-down-2/ "'Is Science Slowing Down?', Alexander 2018"){#alexander-2018-2 .link-annotated}, [[Scott Alexander](https://www.astralcodexten.com/ "Scott Alexander"){.link-annotated-partial .id-not}]{.author .cite-author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Fslatestarcodex.com%252F2018%252F11%252F26%252Fis-science-slowing-down-2%252F.html "Directory-tag link-bibliography for link https://slatestarcodex.com/2018/11/26/is-science-slowing-down-2/"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 228. `https://arxiv.org/pdf/1809.11096#page=8&org=deepmind`: [`“BigGAN: Large Scale GAN Training For High Fidelity Natural Image Synthesis § 5.2 Additional Evaluation On JFT-300M”`{=html}](https://arxiv.org/pdf/1809.11096#page=8&org=deepmind "'BigGAN: Large Scale GAN Training For High Fidelity Natural Image Synthesis § 5.2 Additional Evaluation On JFT-300M', Brock et al 2018 (page 8 org deepmind)"){#brock-et-al-2018-page-8-org-deepmind .link-annotated}, [[Andrew Brock](https://en.wikipedia.org/wiki/Andrew_Brock "Andrew Brock"){.id-not}, [Jeff Donahue](https://jeffdonahue.com/ "Jeff Donahue"){.id-not}, [Karen Simonyan](https://scholar.google.com/citations?user=L7lMQkQAAAAJ "Karen Simonyan"){.id-not}]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fpdf%252F1809.11096%2523page%253D8%2526org%253Ddeepmind.html "Directory-tag link-bibliography for link https://arxiv.org/pdf/1809.11096#page=8&org=deepmind"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 229. `https://arxiv.org/abs/1808.01097`: [`“CurriculumNet: Weakly Supervised Learning from Large-Scale Web Images”`{=html}](https://arxiv.org/abs/1808.01097 "'CurriculumNet: Weakly Supervised Learning from Large-Scale Web Images', Guo et al 2018"){#guo-et-al-2018-2 .link-annotated}, [Sheng Guo, Weilin Huang, Haozhi Zhang[, Chenfan Zhuang, Dengke Dong, Matthew R. Scott, Dinglong Huang]{.collapse}]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F1808.01097.html "Directory-tag link-bibliography for link https://arxiv.org/abs/1808.01097"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 230. `https://s3-us-west-2.amazonaws.com/openai-assets/research-covers/language-unsupervised/language_understanding_paper.pdf#page=5`: [`“GPT-1: Improving Language Understanding by Generative Pre-Training § Model Specifications”`{=html}](https://s3-us-west-2.amazonaws.com/openai-assets/research-covers/language-unsupervised/language_understanding_paper.pdf#page=5 "'GPT-1: Improving Language Understanding by Generative Pre-Training § Model specifications', Radford et al 2018 (page 5)"){#radford-et-al-2018-page-5 .link-annotated}, [[Alec Radford](https://scholar.google.com/citations?user=dOad5HoAAAAJ "Alec Radford"){.id-not}, [Karthik Rajagopal Narasimhan](https://karthikncode.github.io/ "'Karthik R. Narasimhan homepage', Narasimhan 2026"){.id-not}, [Tim Salimans](https://research.google/people/106222/ "Tim Salimans"){.link-annotated-partial .id-not}, [Ilya Sutskever](https://en.wikipedia.org/wiki/Ilya_Sutskever "Ilya Sutskever"){.link-annotated-partial .id-not}]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Fs3-us-west-2.amazonaws.com%252Fopenai-assets%252Fresearch-covers%252Flanguage-unsupervised%252Flanguage_understanding_paper.pdf%2523page%253D5.html "Directory-tag link-bibliography for link https://s3-us-west-2.amazonaws.com/openai-assets/research-covers/language-unsupervised/language_understanding_paper.pdf#page=5"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 231. `https://arxiv.org/abs/1805.00932#facebook`: [`“Exploring the Limits of Weakly Supervised Pretraining”`{=html}](https://arxiv.org/abs/1805.00932#facebook "'Exploring the Limits of Weakly Supervised Pretraining', Mahajan et al 2018"){#mahajan-et-al-2018-2 .link-annotated}, [Dhruv Mahajan, [Ross Girshick](https://scholar.google.com/citations?user=W8VIEZgAAAAJ "Ross Girshick"){.id-not}, Vignesh Ramanathan[, [Kaiming He](https://en.wikipedia.org/wiki/Kaiming_He "Kaiming He"){.id-not}, Manohar Paluri, [Yixuan Li](https://en.wikipedia.org/wiki/Yixuan_Li "Yixuan Li"){.id-not}, Ashwin Bharambe, [Laurens van der Maaten](https://lvdmaaten.github.io/ "Laurens van der Maaten"){.link-annotated-partial .id-not}]{.collapse}]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F1805.00932%2523facebook.html "Directory-tag link-bibliography for link https://arxiv.org/abs/1805.00932#facebook"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 232. `https://arxiv.org/abs/1801.06146`: [`“ULMFiT: Universal Language Model Fine-Tuning for Text Classification”`{=html}](https://arxiv.org/abs/1801.06146 "'ULMFiT: Universal Language Model Fine-tuning for Text Classification', Howard & Ruder 2018"){#howard-ruder-2018 .link-annotated}, [Jeremy Howard, Sebastian Ruder]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F1801.06146.html "Directory-tag link-bibliography for link https://arxiv.org/abs/1801.06146"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 233. `https://smerity.com/articles/2018/limited_compute.html`: [`“The Compute and Data Moats Are Dead”`{=html}](https://smerity.com/articles/2018/limited_compute.html "'The compute and data moats are dead', Merity 2018"){#merity-2018 .link-annotated}, [[Stephen Merity](https://state.smerity.com/ "State of the Smerity"){.link-annotated-partial .id-not}]{.author .cite-author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Fsmerity.com%252Farticles%252F2018%252Flimited_compute.html.html "Directory-tag link-bibliography for link https://smerity.com/articles/2018/limited_compute.html"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 234. `https://arxiv.org/abs/1706.06083`: [`“Towards Deep Learning Models Resistant to Adversarial Attacks”`{=html}](https://arxiv.org/abs/1706.06083 "'Towards Deep Learning Models Resistant to Adversarial Attacks', Madry et al 2017"){#madry-et-al-2017 .link-annotated}, [[Aleksander Madry](http://madry.mit.edu/ "Homepage: Aleksander Mądry"){.link-annotated-partial .id-not}, Aleksandar Makelov, [Ludwig Schmidt](https://en.wikipedia.org/wiki/Ludwig_Schmidt "Ludwig Schmidt"){.id-not}[, Dimitris Tsipras, Adrian Vladu]{.collapse}]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F1706.06083.html "Directory-tag link-bibliography for link https://arxiv.org/abs/1706.06083"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 235. `https://arxiv.org/abs/1706.01427#deepmind`: [`“A Simple Neural Network Module for Relational Reasoning”`{=html}](https://arxiv.org/abs/1706.01427#deepmind "'A simple neural network module for relational reasoning', Santoro et al 2017"){#santoro-et-al-2017 .link-annotated}, [[Adam Santoro](https://scholar.google.com/citations?user=evIkDWoAAAAJ "Adam Santoro"){.id-not}, David Raposo, David G. T. Barrett[, [Mateusz Malinowski](https://en.wikipedia.org/wiki/Mateusz_Malinowski "Mateusz Malinowski"){.id-not}, [Razvan Pascanu](https://sites.google.com/view/razp/home){.link-annotated-partial .id-not}, Peter Battaglia, [Timothy Lillicrap](https://en.wikipedia.org/wiki/Timothy_Lillicrap "Timothy Lillicrap"){.id-not}]{.collapse}]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F1706.01427%2523deepmind.html "Directory-tag link-bibliography for link https://arxiv.org/abs/1706.01427#deepmind"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 236. `https://arxiv.org/abs/1705.07750#deepmind`: [`“Quo Vadis, Action Recognition? A New Model I3D and the Kinetics Dataset”`{=html}](https://arxiv.org/abs/1705.07750#deepmind "'Quo Vadis, Action Recognition? A New Model I3D and the Kinetics Dataset', Carreira & Zisserman 2017"){#carreira-zisserman-2017 .link-annotated}, [Joao Carreira, [Andrew Zisserman](https://en.wikipedia.org/wiki/Andrew_Zisserman "Andrew Zisserman"){.id-not}]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F1705.07750%2523deepmind.html "Directory-tag link-bibliography for link https://arxiv.org/abs/1705.07750#deepmind"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 237. `https://arxiv.org/abs/1705.05640`: [`“WebVision Challenge: Visual Learning and Understanding With Web Data”`{=html}](https://arxiv.org/abs/1705.05640 "'WebVision Challenge: Visual Learning and Understanding With Web Data', Li et al 2017"){#li-et-al-2017-1 .link-annotated}, [Wen Li, Limin Wang, [Wei Li](https://scholar.google.com/citations?user=7IUCbE4AAAAJ "Wei Li"){.id-not}[, Eirikur Agustsson, Jesse Berent, Abhinav Gupta, Rahul Sukthankar, Luc Van Gool]{.collapse}]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F1705.05640.html "Directory-tag link-bibliography for link https://arxiv.org/abs/1705.05640"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 238. `https://blogs.microsoft.com/ai/microsoft-researchers-win-imagenet-computer-vision-challenge/`: [`“Microsoft Researchers Win ImageNet Computer Vision Challenge”`{=html}](https://blogs.microsoft.com/ai/microsoft-researchers-win-imagenet-computer-vision-challenge/ "'Microsoft researchers win ImageNet computer vision challenge', Linn 2015"){#linn-2015 .link-annotated}, [Allison Linn]{.author .cite-author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Fblogs.microsoft.com%252Fai%252Fmicrosoft-researchers-win-imagenet-computer-vision-challenge%252F.html "Directory-tag link-bibliography for link https://blogs.microsoft.com/ai/microsoft-researchers-win-imagenet-computer-vision-challenge/"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 239. `https://arxiv.org/abs/1512.02595#baidu`: [`“Deep Speech 2: End-To-End Speech Recognition in English and Mandarin”`{=html}](https://arxiv.org/abs/1512.02595#baidu "'Deep Speech 2: End-to-End Speech Recognition in English and Mandarin', Amodei et al 2015"){#amodei-et-al-2015 .link-annotated}, [[Dario Amodei](https://en.wikipedia.org/wiki/Dario_Amodei "Dario Amodei"){.link-annotated-partial .id-not}, Rishita Anubhai, Eric Battenberg[, Carl Case, Jared Casper, [Bryan Catanzaro](https://scholar.google.com/citations?user=UZ6kI2AAAAAJ "Bryan Catanzaro"){.id-not}, Jingdong Chen, Mike Chrzanowski, Adam Coates, Greg Diamos, Erich Elsen, Jesse Engel, Linxi Fan, Christopher Fougner, Tony Han, Awni Hannun, Billy Jun, Patrick LeGresley, Libby Lin, [Sharan Narang](https://scholar.google.com/citations?user=CWOixywAAAAJ "Sharan Narang"){.link-modified-recently .id-not}, [Andrew Ng](https://en.wikipedia.org/wiki/Andrew_Ng "Andrew Ng"){.id-not}, [Sherjil Ozair](https://sherjil.ozair.ai/ "Sherjil Ozair"){.id-not}, Ryan Prenger, Jonathan Raiman, [Sanjeev Satheesh](https://scholar.google.com/citations?user=VUi7eM8AAAAJ "Sanjeev Satheesh"){.id-not}, [David Seetapun](https://en.wikipedia.org/wiki/David_Seetapun "David Seetapun"){.id-not}, Shubho Sengupta, [Yi Wang](https://en.wikipedia.org/wiki/Yi_Wang "Yi Wang"){.id-not}, Zhiqian Wang, Chong Wang, Bo Xiao, [Dani Yogatama](https://dyogatama.github.io/ "Dani Yogatama"){.link-annotated-partial .id-not}, Jun Zhan, Zhenyao Zhu]{.collapse}]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F1512.02595%2523baidu.html "Directory-tag link-bibliography for link https://arxiv.org/abs/1512.02595#baidu"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 240. `https://arxiv.org/abs/1511.06789#google`: [`“The Unreasonable Effectiveness of Noisy Data for Fine-Grained Recognition”`{=html}](https://arxiv.org/abs/1511.06789#google "'The Unreasonable Effectiveness of Noisy Data for Fine-Grained Recognition', Krause et al 2015"){#krause-et-al-2015 .link-annotated}, [[Jonathan Krause](https://scholar.google.com/citations?user=7DwDYzkAAAAJ "Jonathan Krause"){.id-not}, Benjamin Sapp, Andrew Howard[, Howard Zhou, Alexander Toshev, Tom Duerig, James Philbin, [Li Fei-Fei](https://en.wikipedia.org/wiki/Li_Fei-Fei "Li Fei-Fei"){.id-not}]{.collapse}]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F1511.06789%2523google.html "Directory-tag link-bibliography for link https://arxiv.org/abs/1511.06789#google"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 241. `https://arxiv.org/abs/1511.02251#facebook`: [`“Learning Visual Features from Large Weakly Supervised Data”`{=html}](https://arxiv.org/abs/1511.02251#facebook "'Learning Visual Features from Large Weakly Supervised Data', Joulin et al 2015"){#joulin-et-al-2015 .link-annotated}, [[Armand Joulin](https://scholar.google.com/citations?user=kRJkDakAAAAJ "Armand Joulin"){.id-not}, [Laurens van der Maaten](https://lvdmaaten.github.io/ "Laurens van der Maaten"){.link-annotated-partial .id-not}, Allan Jabri, Nicolas Vasilache]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F1511.02251%2523facebook.html "Directory-tag link-bibliography for link https://arxiv.org/abs/1511.02251#facebook"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 242. `https://openaccess.thecvf.com/content_cvpr_2015/papers/Xiao_Learning_From_Massive_2015_CVPR_paper.pdf#baidu`: [`“Clothing-1M: Learning from Massive Noisy Labeled Data for Image Classification”`{=html}](https://openaccess.thecvf.com/content_cvpr_2015/papers/Xiao_Learning_From_Massive_2015_CVPR_paper.pdf#baidu "'Clothing-1M: Learning from Massive Noisy Labeled Data for Image Classification', Xiao et al 2015"){#xiao-et-al-2015 .link-annotated}, [Tong Xiao, [Tian Xia](https://en.wikipedia.org/wiki/Tian_Xia "Tian Xia"){.id-not}, Yi Yang[, Chang Huang, [Xiaogang Wang](https://scholar.google.com/citations?user=-B5JgjsAAAAJ "Xiaogang Wang"){.id-not}]{.collapse}]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Fopenaccess.thecvf.com%252Fcontent_cvpr_2015%252Fpapers%252FXiao_Learning_From_Massive_2015_CVPR_paper.pdf%2523baidu.html "Directory-tag link-bibliography for link https://openaccess.thecvf.com/content_cvpr_2015/papers/Xiao_Learning_From_Massive_2015_CVPR_paper.pdf#baidu"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 243. `https://arxiv.org/abs/1402.1869`: [`“On the Number of Linear Regions of Deep Neural Networks”`{=html}](https://arxiv.org/abs/1402.1869 "'On the Number of Linear Regions of Deep Neural Networks', Montúfar et al 2014"){#montúfar-et-al-2014 .link-annotated}, [Guido Montúfar, [Razvan Pascanu](https://sites.google.com/view/razp/home){.link-annotated-partial .id-not}, [Kyunghyun Cho](https://www.kyunghyuncho.me/ "Kyunghyun Cho"){.link-annotated-partial .id-not}, [Yoshua Bengio](https://en.wikipedia.org/wiki/Yoshua_Bengio "Yoshua Bengio"){.id-not}]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F1402.1869.html "Directory-tag link-bibliography for link https://arxiv.org/abs/1402.1869"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 244. `http://www.lrec-conf.org/proceedings/lrec2014/pdf/1097_Paper.pdf`: [`“N-Gram Counts and Language Models from the Common Crawl”`{=html}](http://www.lrec-conf.org/proceedings/lrec2014/pdf/1097_Paper.pdf "'N-gram Counts and Language Models from the Common Crawl', Buck et al 2014"){#buck-et-al-2014 .link-annotated}, [Christian Buck, Kenneth Heafield, Bas van Ooyen]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/http%253A%252F%252Fwww.lrec-conf.org%252Fproceedings%252Flrec2014%252Fpdf%252F1097_Paper.pdf.html "Directory-tag link-bibliography for link http://www.lrec-conf.org/proceedings/lrec2014/pdf/1097_Paper.pdf"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 245. `https://aclanthology.org/P13-2121.pdf`: [`“Scalable Modified Kneser-Ney Language Model Estimation”`{=html}](https://aclanthology.org/P13-2121.pdf "'Scalable Modified Kneser-Ney Language Model Estimation', Heafield et al 2013"){#heafield-et-al-2013 .link-annotated}, [Kenneth Heafield, Ivan Pouzyrevsky, Jonathan H. Clark, [Philipp Koehn](https://en.wikipedia.org/wiki/Philipp_Koehn "Philipp Koehn"){.id-not}]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Faclanthology.org%252FP13-2121.pdf.html "Directory-tag link-bibliography for link https://aclanthology.org/P13-2121.pdf"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 246. `2011-collobert.pdf`: [`“Natural Language Processing (Almost) from Scratch”`{=html}](/doc/psychology/linguistics/2011-collobert.pdf "'Natural Language Processing (Almost) from Scratch', Collobert et al 2011"){#collobert-et-al-2011 .link-annotated}, [Ronan Collobert, [Jason Weston](https://en.wikipedia.org/wiki/Jason_Weston "Jason Weston"){.id-not}, [Leon Bottou](https://en.wikipedia.org/wiki/Leon_Bottou "Leon Bottou"){.id-not}[, Michael Karlen, [Koray Kavukcuoglu](https://scholar.google.com/citations?user=sGFyDIUAAAAJ "Koray Kavukcuoglu"){.id-not}, Pavel Kuksa]{.collapse}]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/%252Fdoc%252Fpsychology%252Flinguistics%252F2011-collobert.pdf.html "Directory-tag link-bibliography for link /doc/psychology/linguistics/2011-collobert.pdf"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 247. `2010-mikolov.pdf`: [`“Recurrent Neural Network Based Language Model”`{=html}](/doc/ai/nn/rnn/2010-mikolov.pdf "'Recurrent Neural Network Based Language Model', Mikolov et al 2010"){#mikolov-et-al-2010 .link-annotated}, [[Tomas Mikolov](https://en.wikipedia.org/wiki/Tomas_Mikolov "Tomas Mikolov"){.id-not}, [Martin Karafiat](https://en.wikipedia.org/wiki/Martin_Karafiat "Martin Karafiat"){.id-not}, Lukas Burget[, Jan Cernocky, Sanjeev Khudanpur]{.collapse}]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/%252Fdoc%252Fai%252Fnn%252Frnn%252F2010-mikolov.pdf.html "Directory-tag link-bibliography for link /doc/ai/nn/rnn/2010-mikolov.pdf"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 248. `2010-hameed.pdf`: [`“Understanding Sources of Inefficiency in General-Purpose Chips”`{=html}](/doc/cs/hardware/2010-hameed.pdf "'Understanding sources of inefficiency in general-purpose chips', Hameed et al 2010"){#hameed-et-al-2010 .link-annotated}, [Rehan Hameed, Wajahat Qadeer, Megan Wachs[, Omid Azizi, Alex Solomatnikov, Benjamin C. Lee, Stephen Richardson, [Christos Kozyrakis](https://en.wikipedia.org/wiki/Christos_Kozyrakis "Christos Kozyrakis"){.id-not}, [Mark Alan Horowitz](https://en.wikipedia.org/wiki/Mark_Alan_Horowitz "Mark Alan Horowitz"){.id-not}]{.collapse}]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/%252Fdoc%252Fcs%252Fhardware%252F2010-hameed.pdf.html "Directory-tag link-bibliography for link /doc/cs/hardware/2010-hameed.pdf"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 249. `https://dw2blog.com/2009/11/02/halloween-nightmare-scenario-early-2020s/`: [`“Halloween Nightmare Scenario, Early 2020’s”`{=html}](https://dw2blog.com/2009/11/02/halloween-nightmare-scenario-early-2020s/ "'Halloween nightmare scenario, early 2020’s', Wood 2009"){#wood-2009 .link-annotated}, [David Wood]{.author .cite-author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Fdw2blog.com%252F2009%252F11%252F02%252Fhalloween-nightmare-scenario-early-2020s%252F.html "Directory-tag link-bibliography for link https://dw2blog.com/2009/11/02/halloween-nightmare-scenario-early-2020s/"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 250. `2009-koren.pdf`: [`“Matrix Factorization Techniques for Recommender Systems”`{=html}](/doc/ai/tabular/2009-koren.pdf "'Matrix factorization techniques for recommender systems', Koren et al 2009"){#koren-et-al-2009 .link-annotated}, [Yehuda Koren, Robert Bell, Chris Volinsky]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/%252Fdoc%252Fai%252Ftabular%252F2009-koren.pdf.html "Directory-tag link-bibliography for link /doc/ai/tabular/2009-koren.pdf"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 251. `https://web.archive.org/web/20230718144747/https://frc.ri.cmu.edu/~hpm/project.archive/robot.papers/2004/Predictions.html`: [`“Robot Predictions Evolution”`{=html}](https://web.archive.org/web/20230718144747/https://frc.ri.cmu.edu/~hpm/project.archive/robot.papers/2004/Predictions.html "'Robot Predictions Evolution', Moravec 2004"){#moravec-2004 .link-annotated}, [[Hans Moravec](https://en.wikipedia.org/wiki/Hans_Moravec "Hans Moravec"){.link-annotated-partial .id-not}]{.author .cite-author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Fweb.archive.org%252Fweb%252F20230718144747%252Fhttps%253A%252F%252Ffrc.ri.cmu.edu%252F~hpm%252Fproject.archive%252Frobot.papers%252F2004%252FPredictions.html.html "Directory-tag link-bibliography for link https://web.archive.org/web/20230718144747/https://frc.ri.cmu.edu/~hpm/project.archive/robot.papers/2004/Predictions.html"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 252. `2003-perlich.pdf`: [`“Tree Induction versus Logistic Regression: A Learning-Curve Analysis”`{=html}](/doc/ai/scaling/2003-perlich.pdf "'Tree Induction versus Logistic Regression: A Learning-Curve Analysis', Perlich et al 2003"){#perlich-et-al-2003 .link-annotated}, [Claudia Perlich, [Foster Provost](https://en.wikipedia.org/wiki/Foster_Provost "Foster Provost"){.id-not}, Jeffrey S. Simonoff]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/%252Fdoc%252Fai%252Fscaling%252F2003-perlich.pdf.html "Directory-tag link-bibliography for link /doc/ai/scaling/2003-perlich.pdf"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 253. `http://infolab.stanford.edu/~backrub/google.html`: [`“The Anatomy of a Large-Scale Hypertextual Web Search Engine”`{=html}](http://infolab.stanford.edu/~backrub/google.html "'The Anatomy of a Large-Scale Hypertextual Web Search Engine', Brin & Page 1998"){#brin-page-1998 .link-annotated}, [[Sergey Brin](https://en.wikipedia.org/wiki/Sergey_Brin "Sergey Brin"){.link-annotated-partial .id-not}, [Lawrence Page](https://en.wikipedia.org/wiki/Lawrence_Page "Lawrence Page"){.id-not}]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/http%253A%252F%252Finfolab.stanford.edu%252F~backrub%252Fgoogle.html.html "Directory-tag link-bibliography for link http://infolab.stanford.edu/~backrub/google.html"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 254. `https://paulfchristiano.com/`: [`“Homepage of Paul F. Christiano”`{=html}](https://paulfchristiano.com/ "'Homepage of Paul F. Christiano', Christiano 2026"){#_Rx5I2W4d .link-annotated}, [Paul F. Christiano]{.author .cite-author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Fpaulfchristiano.com%252F.html "Directory-tag link-bibliography for link https://paulfchristiano.com/"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block}