“‘AI Emergence’ Tag”,2021-04-17 (backlinks)⁠:

Bibliography for tag ai/scaling/emergence, most recent first: 4 related tags, 35 annotations, & 13 links (parent).

• See Also
• Links
  • “Foundational Challenges in Assuring Alignment and Safety of Large Language Models”, Anwar et al 2024
  • “A Phase Transition between Positional and Semantic Learning in a Solvable Model of Dot-Product Attention”, Cui et al 2024
  • “Compositional Capabilities of Autoregressive Transformers: A Study on Synthetic, Interpretable Tasks”, Ramesh et al 2023
  • “Training Dynamics of Contextual N-Grams in Language Models”, Quirke et al 2023
  • “Compositional Abilities Emerge Multiplicatively: Exploring Diffusion Models on a Synthetic Task”, Okawa et al 2023
  • “A Theory for Emergence of Complex Skills in Language Models”, Arora & Goyal2023
  • “Teaching Arithmetic to Small Transformers”, Lee et al 2023
  • “Schema-Learning and Rebinding As Mechanisms of In-Context Learning and Emergence”, Swaminathan et al 2023
  • “8 Things to Know about Large Language Models”, Bowman2023
  • “The Quantization Model of Neural Scaling”, Michaud et al 2023
  • “Toolformer: Language Models Can Teach Themselves to Use Tools”, Schick et al 2023
  • “Interactive-Chain-Prompting (INTERCPT): Ambiguity Resolution for Crosslingual Conditional Generation With Interaction”, Pilault et al 2023
  • “Broken Neural Scaling Laws”, Caballero et al 2022
  • “U-PaLM: Transcending Scaling Laws With 0.1% Extra Compute”, Tay et al 2022
  • “Challenging BIG-Bench Tasks (BBH) and Whether Chain-Of-Thought Can Solve Them”, Suzgun et al 2022
  • “Language Models Are Multilingual Chain-Of-Thought Reasoners”, Shi et al 2022
  • “Hidden Progress in Deep Learning: SGD Learns Parities Near the Computational Limit”, Barak et al 2022
  • “Emergent Abilities of Large Language Models”, Wei et al 2022
  • “Beyond the Imitation Game: Quantifying and Extrapolating the Capabilities of Language Models”, Srivastava et al 2022
  • “Data Distributional Properties Drive Emergent Few-Shot Learning in Transformers”, Chan et al 2022
  • “PaLM: Scaling Language Modeling With Pathways”, Chowdhery et al 2022
  • “In-Context Learning and Induction Heads”, Olsson et al 2022
  • “Predictability and Surprise in Large Generative Models”, Ganguli et al 2022
  • “The Effects of Reward Misspecification: Mapping and Mitigating Misaligned Models”, Pan et al 2022
  • “A Mathematical Framework for Transformer Circuits”, Elhage et al 2021
  • “Scaling Language Models: Methods, Analysis & Insights from Training Gopher”, Rae et al 2021
  • “A General Language Assistant As a Laboratory for Alignment”, Askell et al 2021
  • “Mapping Language Models to Grounded Conceptual Spaces”, Patel & Pavlick2021
  • “Program Synthesis With Large Language Models”, Austin et al 2021
  • “MMLU: Measuring Massive Multitask Language Understanding”, Hendrycks et al 2020
  • “GPT-3: Language Models Are Few-Shot Learners”, Brown et al 2020
  • “Emergence in Cognitive Science”, McClelland2010
  • “Observed Universality of Phase Transitions in High-Dimensional Geometry, With Implications for Modern Data Analysis and Signal Processing”, Donoho & Tanner2009
  • “The Phase Transition In Human Cognition § Phase Transitions in Language Processing”, Spivey et al 2009 (page 13)
  • “A Dynamic Systems Model of Cognitive and Language Growth”, Geert1991
  • Sort By Magic
    • transformer-circuits
    • emergent-capabilities
    • few-shot-learning
  • Wikipedia
• Miscellaneous
• Bibliography

Backlinks: