Bibliography:

‘Transformer’ tag
‘GPT-2 fiction’ tag
‘GPT-2’ tag
‘GPT-2 nonfiction’ tag
/doc/ai/nn/transformer/gpt/2/poetry
‘GPT-3 fiction’ tag
‘GPT-3 humor’ tag
‘GPT-3’ tag
‘GPT-3 nonfiction’ tag
‘GPT-3 poetry’ tag
‘GPT-4 fiction’ tag
‘GPT-4’ tag
‘GPT-4 nonfiction’ tag
‘GPT-4 poetry’ tag
‘Sydney (AI)’ tag
‘GPT-5’ tag
‘GPT calibration’ tag
‘Claude AI’ tag
‘Codex’ tag
‘DALL·E 1’ tag
‘DALL·E 2’ tag
‘DALL·E 3’ tag
‘DALL·E’ tag
‘GPT fiction’ tag
‘inner monologue (AI)’ tag
‘instruct-tuning LLMs’ tag
‘Jukebox’ tag
‘LaMDA’ tag
‘GPT non-fiction’ tag
‘PaLM 2’ tag
‘PaLM’ tag
‘GPT poetry’ tag
‘Whisper NN’ tag
‘LM tokenization’ tag
‘video generation’ tag
GPT-3 Semantic Derealization
Research Ideas
You should write more online—it’s still a good time
Machine Learning Scaling
Continuous Autoregressive Models with Noise Augmentation Avoid Error Accumulation
Do Large Language Models Perform Latent Multi-Hop Reasoning without Exploiting Shortcuts?
2:4 Sparse Llama: Smaller Models for Efficient GPU Inference
Medical Adaptation of Large Language and Vision-Language Models: Are We Making Progress?
Model Equality Testing: Which Model Is This API Serving?
Centaur: a foundation model of human cognition
Do LLMs estimate uncertainty well in instruction-following?
Interpretable Contrastive Monte Carlo Tree Search Reasoning
nGPT: Normalized Transformer with Representation Learning on the Hypersphere
LLM Applications I Want To See
994c2f94d62a984842ed3fa41412926dccca6241.html
Ensemble everything everywhere: Multi-scale aggregation for adversarial robustness
Token Erasure as a Footprint of Implicit Vocabulary Items in LLMs
Resolving Discrepancies in Compute-Optimal Scaling of Language Models
When Parts are Greater Than Sums: Individual LLM Components Can Outperform Full Models
Nemotron-4 340B Technical Report
DataComp-LM: In search of the next generation of training sets for language models
How Do Large Language Models Acquire Factual Knowledge During Pretraining?
Be like a Goldfish, Don’t Memorize! Mitigating Memorization in Generative LLMs
Discovering Preference Optimization Algorithms with and for Large Language Models
MCTSr: Accessing GPT-4 level Mathematical Olympiad Solutions via Monte Carlo Tree Self-refine with LLaMA-3-8B
For Chinese Students, the New Tactic Against AI Checks: More AI
MAP-Neo: Highly Capable and Transparent Bilingual Large Language Model Series
Superposed Decoding: Multiple Generations from a Single Autoregressive Inference Pass
SpaceByte: Towards Deleting Tokenization from Large Language Modeling
Towards smaller, faster decoder-only transformers: Architectural variants and their implications
Design of highly functional genome editors by modeling the universe of CRISPR-Cas sequences
From r to Q^✱: Your Language Model is Secretly a Q-Function
CATS: Contextually-Aware Thresholding for Sparsity in Large Language Models
CulturalTeaming: AI-Assisted Interactive Red-Teaming for Challenging LLMs’ (Lack of) Multicultural Knowledge
Training LLMs over Neurally Compressed Text
Reverse Training to Nurse the Reversal Curse
Evolutionary Optimization of Model Merging Recipes
Yi: Open Foundation Models by 01.AI
Actions Speak Louder than Words: Trillion-Parameter Sequential Transducers for Generative Recommendations (HSTU)
Fast Adversarial Attacks on Language Models In One GPU Minute
Autonomous Data Selection with Language Models for Mathematical Texts
Grandmaster-Level Chess Without Search
Neural Networks Learn Statistics of Increasing Complexity
Arrows of Time for Large Language Models
SliceGPT: Compress Large Language Models by Deleting Rows and Columns
Excuse me, sir? Your language model is leaking (information)
TinyLlama: An Open-Source Small Language Model
LLaMA Pro: Progressive LLaMA with Block Expansion
Generative AI is already widespread in the public sector
Beyond Chinchilla-Optimal: Accounting for Inference in Language Model Scaling Laws
TinyGPT-V: Efficient Multimodal Large Language Model via Small Backbones
Reasons to Reject? Aligning Language Models with Judgments
Generative Multimodal Models are In-Context Learners
Frugal LMs Trained to Invoke Symbolic Solvers Achieve Parameter-Efficient Arithmetic Reasoning
Object Recognition as Next Token Prediction
MEDITRON-70B: Scaling Medical Pretraining for Large Language Models
Localizing Lying in Llama: Understanding Instructed Dishonesty on True-False Questions Through Prompting, Probing, and Patching
OpenAI researchers warned board of AI breakthrough ahead of CEO ouster, sources say
Positional Description Matters for Transformers Arithmetic
Watermarks in the Sand: Impossibility of Strong Watermarking for Generative Models
Tensor Trust: Interpretable Prompt Injection Attacks from an Online Game
Learn Your Tokens: Word-Pooled Tokenization for Language Modeling
Llemma: An Open Language Model For Mathematics
In-Context Pretraining (ICP): Language Modeling Beyond Document Boundaries
OSD: Online Speculative Decoding
Let Models Speak Ciphers: Multiagent Debate through Embeddings
OpenWebMath: An Open Dataset of High-Quality Mathematical Web Text
xVal: A Continuous Number Encoding for Large Language Models
MetaMath: Bootstrap Your Own Mathematical Questions for Large Language Models
Language Modeling Is Compression
Sparse Autoencoders Find Highly Interpretable Features in Language Models
Anchor Points: Benchmarking Models with Much Fewer Examples
When Less is More: Investigating Data Pruning for Pretraining LLMs at Scale
Language Reward Modulation for Pretraining Reinforcement Learning
ReST: Reinforced Self-Training (ReST) for Language Modeling
Studying Large Language Model Generalization with Influence Functions
Multimodal Neurons in Pretrained Text-Only Transformers
Skill-it! A Data-Driven Skills Framework for Understanding and Training Language Models
Length Generalization in Arithmetic Transformers
Are aligned neural networks adversarially aligned?
Improving Long-Horizon Imitation Through Instruction Prediction
Large Language Models Sometimes Generate Purely Negatively-Reinforced Text
SpQR: A Sparse-Quantized Representation for Near-Lossless LLM Weight Compression
Undetectable Watermarks for Language Models
Improving Language Models with Advantage-based Offline Policy Gradients
Accelerating Transformer Inference for Translation via Parallel Decoding
DoReMi: Optimizing Data Mixtures Speeds Up Language Model Pretraining
Memorization for Good: Encryption with Autoregressive Language Models
MEGABYTE: Predicting Million-byte Sequences with Multiscale Transformers
Finding Neurons in a Haystack: Case Studies with Sparse Probing
Inflection AI, Startup From Ex-DeepMind Leaders, Launches Pi—A Chattier Chatbot
Emergent and Predictable Memorization in Large Language Models
A Comparative Study between Full-Parameter and LoRA-based Fine-Tuning on Chinese Instruction Data for Instruction Following Large Language Model
Shall We Pretrain Autoregressive Language Models with Retrieval? A Comprehensive Study
How Large-Language Models Can Revolutionize Military Planning
Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling
8 Things to Know about Large Language Models
BloombergGPT: A Large Language Model for Finance
The Quantization Model of Neural Scaling
Int-4 LLaMa is not enough—Int-3 and beyond: More compression, easier to build apps on LLMs that run locally
Consistency Analysis of ChatGPT
Rewarding Chatbots for Real-World Engagement with Millions of Users
Beyond the Pass Mark: the Accuracy of ChatGPT and Bing in the National Medical Licensure Examination in Japan
SpikeGPT: Generative Pre-trained Language Model with Spiking Neural Networks
A Prompt Pattern Catalog to Enhance Prompt Engineering with ChatGPT
BiLD: Big Little Transformer Decoder
Data Selection for Language Models via Importance Resampling
In-Context Retrieval-Augmented Language Models
Crawling the Internal Knowledge-Base of Language Models
Big Tech was moving cautiously on AI. Then came ChatGPT. Google, Facebook and Microsoft helped build the scaffolding of AI. Smaller companies are taking it to the masses, forcing Big Tech to react
Rock Guitar Tablature Generation via Natural Language Processing
InPars-Light: Cost-Effective Unsupervised Training of Efficient Rankers
A New Chat Bot Is a ‘Code Red’ for Google’s Search Business: A new wave of chat bots like ChatGPT use artificial intelligence that could reinvent or even replace the traditional internet search engine
Why Can GPT Learn In-Context? Language Models Secretly Perform Gradient Descent as Meta-Optimizers
Rethinking the Role of Scale for In-Context Learning: An Interpretability-based Case Study at 66 Billion Scale
Interpreting Neural Networks through the Polytope Lens
SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Models
InstructPix2Pix: Learning to Follow Image Editing Instructions
Galactica: A Large Language Model for Science
Large Language Models Struggle to Learn Long-Tail Knowledge
The CRINGE Loss: Learning what language not to model
Mysteries of mode collapse § Inescapable wedding parties
GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers
What is my math transformer doing? – 3 results on interpretability and generalization
When Life Gives You Lemons, Make Cherryade: Converting Feedback from Bad Responses into Good Labels
Can language models handle recursively nested grammatical structures? A case study on comparing models and humans
Evaluating Parameter Efficient Learning for Generation
BioGPT: Generative Pre-trained Transformer for Biomedical Text Generation and Mining
Arithmetic Sampling: Parallel Diverse Decoding for Large Language Models
MTEB: Massive Text Embedding Benchmark
Foundation Transformers
Ask Me Anything (AMA): A simple strategy for prompting language models
Is Reinforcement Learning (Not) for Natural Language Processing: Benchmarks, Baselines, and Building Blocks for Natural Language Policy Optimization
Sparrow: Improving alignment of dialogue agents via targeted human judgements
Generate rather than Retrieve (GenRead): Large Language Models are Strong Context Generators
FP8 Formats for Deep Learning
Petals: Collaborative Inference and Fine-tuning of Large Models
LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scale
Meaning without reference in large language models
Effidit: Your AI Writing Assistant
Language models show human-like content effects on reasoning
LM-Nav: Robotic Navigation with Large Pre-Trained Models of Language, Vision, and Action
Can Foundation Models Talk Causality?
NOAH: Neural Prompt Search
ZeroQuant: Efficient and Affordable Post-Training Quantization for Large-Scale Transformers
Quark: Controllable Text Generation with Reinforced Unlearning
RankGen: Improving Text Generation with Large Ranking Models
Opal: Multimodal Image Generation for News Illustration
What Language Model to Train if You Have One Million GPU Hours?
WAVPROMPT: Towards Few-Shot Spoken Language Understanding with Frozen Language Models
Shared computational principles for language processing in humans and deep language models
Vector-quantized Image Modeling with Improved VQGAN
Brains and algorithms partially converge in natural language processing
Quantifying Memorization Across Neural Language Models
A Contrastive Framework for Neural Text Generation
AdaPrompt: Adaptive Model Training for Prompt-based NLP
InPars: Data Augmentation for Information Retrieval using Large Language Models
ROME: Locating and Editing Factual Associations in GPT
Cedille: A large autoregressive French language model
Data Scaling Laws in NMT: The Effect of Noise and Architecture
PromptSource: An Integrated Development Environment and Repository for Natural Language Prompts
Using DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, A Large-Scale Generative Language Model
Language Models as Zero-Shot Planners: Extracting Actionable Knowledge for Embodied Agents
WANLI: Worker and AI Collaboration for Natural Language Inference Dataset Creation
A Survey of Controllable Text Generation using Transformer-based Pre-trained Language Models
The Defeat of the Winograd Schema Challenge
Learning To Retrieve Prompts for In-Context Learning
Learning to Prompt for Continual Learning
Amortized Noisy Channel Neural Machine Translation
Few-shot Instruction Prompts for Pretrained Language Models to Detect Social Biases
PROMPT WAYWARDNESS: The Curious Case of Discretized Interpretation of Continuous Prompts
LMTurk: Few-Shot Learners as Crowdsourcing Workers
Improving language models by retrieving from trillions of tokens
Linear algebra with transformers
Zero-Shot Image-to-Text Generation for Visual-Semantic Arithmetic
Long-range and hierarchical language predictions in brains and algorithms
True Few-Shot Learning with Prompts—A Real-World Perspective
Few-shot Named Entity Recognition with Cloze Questions
Evaluating Distributional Distortion in Neural Language Modeling
On Transferability of Prompt Tuning for Natural Language Understanding
CLUES: Few-Shot Learning Evaluation in Natural Language Understanding
Recent Advances in Natural Language Processing via Large Pre-Trained Language Models: A Survey
Fast Model Editing at Scale
Yuan 1.0: Large-Scale Pre-trained Language Model in Zero-Shot and Few-Shot Learning
Towards a Unified View of Parameter-Efficient Transfer Learning
A Few More Examples May Be Worth Billions of Parameters
Scaling Laws for Neural Machine Translation
Can Language Models Encode Perceptual Structure Without Grounding? A Case Study in Color
What Changes Can Large-scale Language Models Bring? Intensive Study on HyperCLOVA: Billions-scale Korean Generative Pretrained Transformers
Medically Aware GPT-3 as a Data Generator for Medical Dialogue Summarization
General-Purpose Question-Answering with Macaw
An Empirical Exploration in Quality Filtering of Text Data
Want To Reduce Labeling Cost? GPT-3 Can Help
Multimodal Few-Shot Learning with Frozen Language Models
Cutting Down on Prompts and Parameters: Simple Few-Shot Learning with Language Models
RASP: Thinking Like Transformers
ByT5: Towards a token-free future with pre-trained byte-to-byte models
Anthropic raises $124 million to build more reliable, general AI systems
Naver unveils first ‘hyperscale’ AI platform
Scaling Laws for Language Transfer Learning
GPT Understands, Too
How Many Data Points is a Prompt Worth?
Pretrained Transformers as Universal Computation Engines
Language Models have a Moral Dimension
Learning Chess Blindfolded: Evaluating Language Models on State Tracking
Investigating the Limitations of the Transformers with Simple Arithmetic Tasks
Proof Artifact Co-training for Theorem Proving with Language Models
Clinical Outcome Prediction from Admission Notes using Self-Supervised Knowledge Integration
Scaling Laws for Transfer
MAUVE: Measuring the Gap Between Neural Text and Human Text using Divergence Frontiers
Apparently ‘what ho’ is a corruption of…
Making Pre-trained Language Models Better Few-shot Learners
Thinking ahead: prediction in context as a keystone of language in humans and machines
CPM: A Large-scale Generative Chinese Pre-trained Language Model
L2L: Training Large Neural Networks with Constant Memory using a New Execution Algorithm
Summarize, Outline, and Elaborate: Long-Text Generation via Hierarchical Supervision from Extractive Summaries
The neural architecture of language: Integrative reverse-engineering converges on a model for predictive processing
RoFT: A Tool for Evaluating Human Detection of Machine-Generated Text
A Systematic Characterization of Sampling Algorithms for Open-ended Language Generation
Generative Language Modeling for Automated Theorem Proving
Learning to summarize from human feedback
ETHICS: Aligning AI With Shared Human Values
Mirostat: A Neural Text Decoding Algorithm that Directly Controls Perplexity
Climbing towards NLU: On Meaning, Form, and Understanding in the Age of Data
Transformers are RNNs: Fast Autoregressive Transformers with Linear Attention
OpenAI API Beta homepage
Trading Off Diversity and Quality in Natural Language Generation
Scaling Laws from the Data Manifold Dimension
Unigram LM: Byte Pair Encoding is Suboptimal for Language Model Pretraining
Direct Fit to Nature: An Evolutionary Perspective on Biological and Artificial Neural Networks
Pop Music Transformer: Beat-based Modeling and Generation of Expressive Pop Piano Compositions
Scaling Laws for Neural Language Models
Reformer: The Efficient Transformer
What does BERT dream of? A visual investigation of nightmares in Sesame Street
Generative Language Modeling for Automated Theorem Proving § Experiments
Plug and Play Language Models: A Simple Approach to Controlled Text Generation
How Can We Know What Language Models Know?
CommonGen: A Constrained Text Generation Challenge for Generative Commonsense Reasoning
Generalization through Memorization: Nearest Neighbor Language Models
DialoGPT: Large-Scale Generative Pre-training for Conversational Response Generation
CTRL: A Conditional Transformer Language Model For Controllable Generation
Smaller, faster, cheaper, lighter: Introducing DistilGPT, a distilled version of GPT
Language Modeling State-of-the-art leaderboards
Neural Text Generation with Unlikelihood Training
GROVER: Defending Against Neural Fake News
Generative Modeling with Sparse Transformers: We’ve developed the Sparse Transformer, a deep neural network which sets new records at predicting what comes next in a sequence—whether text, images, or sound. It uses an algorithmic improvement of the attention mechanism to extract patterns from sequences 30× longer than possible previously
The Curious Case of Neural Text Degeneration
Smart Vet: Autocompleting Sentences in Veterinary Medical Records
Transformer-XL: Attentive Language Models Beyond a Fixed-Length Context
Music Transformer: Generating Music with Long-Term Structure
Universal Transformers
Adversarial Reprogramming of Neural Networks
GPT-1: Improving Language Understanding with Unsupervised Learning
GPT-1: Improving Language Understanding by Generative Pre-Training
GPT-1: Improving Language Understanding by Generative Pre-Training § Model specifications
Deep reinforcement learning from human preferences § Appendix A.2: Atari
Learning to Generate Reviews and Discovering Sentiment
Design a Role-Playing Game Using 200 Words or Less.
68dc29784a54b4b94a1215b358244b267755a4e1.html
How Does In-Context Learning Work? A Framework for Understanding the Differences from Traditional Supervised Learning
bdf17c80e1ed5dc516811f03acef03415b220143.html
AI Dungeon: Dragon Model Upgrade—You Can Now Play AI Dungeon With One of the Most Powerful AI Models in the World.
52d2fe5633e74d1f355221dba088b17ff34db79d.html
Introducing AI Dungeon Translate: AI Dungeon Players Can Now Translate Their Stories into Emojis by Just Clicking a Button. [ 🤔 💯 🤷‍♂️ 🤔 🤔 🤔 💯]
OpenAI API Alchemy: Emoji Storytelling 🤖
Llama-3.1-405B Now Runs at 969 Tokens/s on Cerebras Inference
I Blew $720 on 100 Notebooks from Alibaba and Started a Paper Website Business
b295888386dd5c1f2e2a679fd7b84432811d3917.html
AlphaStar: Mastering the Real-Time Strategy Game StarCraft II
Transformers As Variational Autoencoders
BlinkDL/RWKV-LM: RWKV Is an RNN With Transformer-Level LLM Performance. It Can Be Directly Trained like a GPT (parallelizable). So It’s Combining the Best of RNN and Transformer—Great Performance, Fast Inference, Saves VRAM, Fast Training, "Infinite" Ctx_len, and Free Sentence Embedding.
Efficient, Reusable RNNs and LSTMs for Torch
Updated Training?
Karpathy/minGPT: A Minimal PyTorch Re-Implementation of the OpenAI GPT (Generative Pretrained Transformer) Training
Minimaxir/textgenrnn: Easily Train Your Own Text-Generating Neural Network of Any Size and Complexity on Any Text Dataset With a Few Lines of Code.
Loom: Multiversal Tree Writing Interface for Human-AI Collaboration
ab1b1b61962d42831ad82c1ecaab2a7d3aef8423.html
Zphang/minimal-Opt
Math: OpenAI API Can Do Some Math out of the Gate, but Most Math It Seems It Has to Learn. Many Times, the Numbers That It Spits out Are Just Random. However, including Different Priming Prompts Can Result in Decent Results.
Deep Learning for Assisting the Process of Music Composition (part 3)
Google DeepMind’s Grandmaster-Level Chess Without Search
The Technology Behind BLOOM Training
Psych-101 Dataset [For Centaur]
The Gostak
Imprompter
Your Next New Best Friend Might Be a Robot
I Made a Custom Gpt That Incorporates Advertisement/product Placement With Its...
The Annotated Transformer
Homepage of Paul F. Christiano
Data Exfiltration from Slack AI via Indirect Prompt Injection
Introductory Antimemetics (abandoned First Draft)
Jared Kaplan
Meditations on Moloch
Stream Seaandsailor
Humans Who Are Not Concentrating Are Not General Intelligences
Monitor: An AI-Driven Observability Interface
This Is the OpenAI API. It Makes Spookily Good Twitter Bots. 13⁄10 Would Retweet
22435625719a0806d8474097cd0740ce131d6684.html
AMA Conjecture, A New Alignment Startup
WikiCrow
1754102bd82fb703b3c17ea47232c04c76f7452e.html
ChatGPT As Muse, Not Oracle
Interpreting GPT: the Logit Lens
Assessing AlephAlpha’s Multimodal Model
Is GPT-3 a Good Rationalist?
We Are Conjecture, A New Alignment Research Startup
Investigating Causal Understanding in LLMs
A One-Question Turing Test for GPT-3
This Mystical Book Was Co-Authored by a Disturbingly Realistic AI
The Guy Behind the Fake AI Halloween Parade Listing Says You’ve Got It All Wrong
Season 1 Ep. 22 OpenAI's Ilya Sutskever: The Man Who Made AI Work
WELM
I'Ve Been Testing the Largest of @OpenAI's Models With AI Dungeon and Been Constantly Impressed at How Interesting and Dynamic the Characters Are, like This Queen, Long Thought to Be Dead, Hiding from Enemies and Not Happy about Me Prying into Her Personal Life.
OpenAI now generates about 100 billion words per day
design#future-tag-features

[Transclude the forward-link's context]
2024-zhao-figure2-roughllmdecisionboundariesonsimplebinaryclassificationtaskdespite128examples.png
2023-03-20-gpt4-scottalexander-halfanhourbeforedawninsanfranciscosample.png
2023-bommarito-figure1-gpt3cpaaccountingexamperformancebyexamsection.jpg
2023-bommarito-figure2-progressofgpt3overtimeoncpaaccountingexam.jpg
2023-jakesch-figure10-participantsdidnotnoticemodelslanttowardsapositionaffectedtheirownargumentwriting.jpg
2023-jakesch-figure3-participantseditorialwritingaboutsocialmediabenefitswereaffectedbygpt3promptslants.jpg
2023-jakesch-figure5-hastyparticipantseditorialwritingweremoreaffectedbyslantedgpt3prompts.jpg
2023-jakesch-figure6-slantedmodelpromptschangedpeoplesopinionafterwritinganeditorial.jpg
2023-jakesch-figure9-participantsdidnotnoticemodelslanttowardsaposition.jpg
2023-qin-figure1-chatgptvsgpt35on20nlpdatasets.png
2022-08-19-gwern-meme-deathknockingatdoor-deeplearningscalingsuccesses.png
2022-08-06-gwern-meme-netflixliegirl-studyingdeeplearningscaling.jpg
2022-05-22-gwern-meme-tintinwhataweekhuh-2ndanniversaryofgpt3paper.png
2022-bommarito-figure1-gpt3performanceonbarexambycategory.jpg
2022-bommarito-figure2-increaseofgpt3modelaccuracyonbarexambysize.jpg
2022-ganguli-figure2-visualizationofsuccessfulredteamattacksonlanguagemodels.png
2021-05-25-naver-hyperclova-computescaling0137bto82b.jpg
2021-01-11-gwern-meme-dogbarkcanthurtyou-aiscaling.jpg
2021-almeida-figure2-lhoptgpt3hyperparametertuningscalinglaw.jpg
2021-almeida-figure3-lhoptlearnedhyperparameteroptimizationongpt2largewikitext103speedupdouble.jpg
2021-askell-figure2-prmptingimprovesalignmentwithmodelscalingwithdecreasingalignmenttax.jpg
2021-askell-figure5-anthropcgptlearnshumanpreferencesatn500withgreaterscale.jpg
2021-dou-figure2-errorsbymodel.png
2021-dou-figure3-errorsbytype.png
2021-dou-figure4-errorsbydecodingsamplingstrategyhyperparameters.png
2021-hernandez-transferlearning-figure2-transferscaling.png
2021-kim-figure4-datatransferfromenglishtochinese.jpg
2021-kim-figure5-transferfromenglishtochinesespanishgerman.jpg
2021-nogueira-figure1-additionperformanceofnumberorthographies.png
2021-solaiman-figure3-largergpt3modelsfinetunebetteronpalmstoxicitydataset.jpg
2020-06-21-openai-beta-gpt3-playgroundui.png
2020-06-18-karpathy-expandingbrainmeme-gpt3metalearning.jpg
2020-04-01-gwern-gpt2-5k-midi-training.png
2020-02-03-gpt21.5b-archiveofourownao3-model-510427-samples-topp090.txt
2020-02-03-gpt21.5b-videogamewalkthrough-model-174925-samples-topp090.txt
2020-01-20-gwern-gpt2-25k-midi-training.png
2020-01-15-gwern-gpt2-preferencelearning-abc-combinedmodel-klregularized-finalrun.png
2020-bostrom-unigramlm-figure1-unigramlmvsbpe.png
2020-brown-figure31-gpt3scaling.png
2020-brown-figure313-humanabilitytodetectmodelgeneratednewsstories.jpg
2020-brown-gpt3-figure13-meanperformancescalingcurve.png
2020-hendrycks-figure1b-gpt3-qascaling.png
2020-henighan-figure1-scalingacrossdomains.jpg
2020-henighan-figure11-pretrainingimageclassificationscaling.png
2020-henighan-figure2-universalmodelsizescaling.jpg
2020-henighan-figure3-domainmodelsizescaling.png
2020-henighan-figure31-qandamodelscaling.jpg
2020-henighan-table1-autoregressivemodelsscalingpowerlaws.png
2020-kaplan-appendix1-summaryofneurallanguagemodelscalingpowerlaws.png
2020-kaplan-figure1-dlscaling.jpg
2020-kaplan-figure15-projectingscaling.png
2020-kaplan-figure7-scalingrnnsvstransformersshowsrnnplateau.png
2020-zhang-figure1-thelikelihoodtrap.png
2019-12-26-gwern-gpt2-preferencelearning-abc-combinedmodel-klregularized-collapse.png
2019-12-17-gwern-gpt2-preferencelearning-abc-terminal.png
2019-12-16-gwern-gpt2-15b-poetry-tensorboard-100tputraining.png
2019-12-13-gwern-gpt2-15b-poetry-tensorboard-97tputraining.png
2019-12-13-gwern-gpt2-preferencelearning-abc-combinedmodel-halfbounce.png
2019-12-12-gwern-gpt2-abc-score-polkaebbbab.png
2019-11-19-gwern-gpt2-15b-poetry-tensorboard-1tputraining.jpg
2019-keskar-table1-ctrlsamplesdemonstratingmetadatainfluenceontextcompletions.png
2019-keskar-table2-ctrltextsamplesusingonlymetadatawithoutaprompt.png
2019-keskar-table3-ctrltextsamplesshowinginfluenceofurllinksasprefixmetadata.png
2019-keskar-table4-ctrltextsamplesusingtemplatizedcontrolcodesforspecifictaskslikeqaortranslation.png
2019-keskar-table5-ctrltextsamplesmixingzeroshotgeneralizationofmetadata.png
2019-keskar-table7-datasetsandcontrolcodesmetadata.png
2019-openai-gpt2-demo-recyclingtextsample.jpg
2019-radford-figure4-gpt2validationloss.jpg
2019-ziegler-preferencelearning-figure1-architecture.png
2018-huang-magenta-musictransformer-attentionvisualization.jpg
https://adamkarvonen.github.io/machine_learning/2024/01/03/chess-world-models.html
https://adversa.ai/blog/universal-llm-jailbreak-chatgpt-gpt-4-bard-bing-anthropic-and-beyond/
06933b9b9a363a8ba3702bd147712068db1eb095.html
https://ai.meta.com/blog/meta-llama-3/
https://amaarora.github.io/2020/02/18/annotatedGPT2.html
cb2faf56d50cf24fedb37741275b7c7d7b5ab2ba.html
https://analyticsindiamag.com/when-chatgpt-attempted-upsc-exam/
8981287b811c3f5fa5cc43daa446d65b4dbff19d.html
https://blog.eleuther.ai/trlx-exploratory-analysis/
https://blog.helix.ml/p/how-we-got-fine-tuning-mistral-7b
https://carper.ai/instruct-gpt-announcement/
766cb8b990cab4b23efdc653265df90bc0acb688.html
https://chat.lmsys.org/
https://colab.research.google.com/drive/1c6VccMPsOMAUQCKU4BVDRd5Y32qkozmK
https://culture.org/ghosts/
c1635afce38b785e481517daa051f35e788f2d67.html
https://davidrozado.substack.com/p/the-political-preferences-of-llms
https://embracethered.com/blog/posts/2023/chatgpt-webpilot-data-exfil-via-markdown-injection/#responsible-disclosure
938a66908d685ba0973f77a6f0d816e0c639a763.html#responsible-disclosure
https://eprint.iacr.org/2021/686
https://gandalf.lakera.ai/
https://github.com/NolanoOrg/llama-int4-quant/
https://github.com/antimatter15/alpaca.cpp
https://github.com/brexhq/prompt-engineering
https://github.com/castorini/transformers-arithmetic
https://github.com/epfLLM/meditron
https://github.com/gh18l/CrawlGPT
https://github.com/greshake/llm-security
dee78cf7ed56d108fe08e2dc89c1ffa9152bef7f.html
https://github.com/jujumilk3/leaked-system-prompts/tree/main
https://github.com/langchain-ai/langchain
https://github.com/qwopqwop200/GPTQ-for-LLaMa
https://github.com/ray-project/llm-numbers
https://github.com/sanjeevanahilan/nanoChatGPT
https://github.com/sberbank-ai/ru-gpts
https://github.com/sgrvinod/chess-transformers
https://hedgehogreview.com/issues/markets-and-the-good/articles/language-machinery
https://homepages.inf.ed.ac.uk/abmayne/publications/sennrich2016NAACL.pdf
https://huggingface.co/Gustavosta/MagicPrompt-Stable-Diffusion
https://huggingface.co/blog/rlhf
https://jaykmody.com/blog/gpt-from-scratch/
117a3f3397e403dbf919c3b186127f6858466e66.html
https://justine.lol/oneliners/
28689672dc2eb3f54bf94739d33bc4e016693ba5.html
https://meteorfrom.space/
e90b927ed9f69f6be08dc4013bc3c118d999de4b.html
https://mi.eng.cam.ac.uk/projects/cued-rnnlm/papers/Interspeech15.pdf
https://minimaxir.com/2023/03/new-chatgpt-overlord/
be35928db14bc14ec4e887559e50a2562a010f80.html
https://openai.com/blog/customizing-gpt-3
https://openai.com/blog/our-approach-to-alignment-research/
https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4627587
https://platform.openai.com/docs/gptbot
616202c012ede04b03ea65cc1d5466b95e632256.html
https://platform.openai.com/docs/guides/gpt-best-practices
9033b0629fd31a0385ba66b2116ac2ac35de7943.html
https://platform.openai.com/docs/guides/prompt-engineering
https://promptarmor.substack.com/p/data-exfiltration-from-writercom
https://qwenlm.github.io/blog/qwen1.5-32b/
08bde6ef08ee40f2148261db1032a29c3eb72c80.html
https://research.google/blog/transformer-a-novel-neural-network-architecture-for-language-understanding/
https://simonwillison.net/2023/Apr/14/worst-that-can-happen/
https://simonwillison.net/2023/Aug/3/weird-world-of-llms/
https://techtualist.substack.com/p/i-wrote-a-script-for-gpt-3-to-take
https://thezvi.substack.com/p/jailbreaking-the-chatgpt-on-release
https://til.simonwillison.net/llms/llama-7b-m2
28a21d92edf1a849d82e5c7825697e68cb30916b.html
https://web.archive.org/web/20240102075620/https://www.jailbreakchat.com/
https://www.alignmentforum.org/posts/YEioD8YLgxih3ydxP/why-simulator-ais-want-to-be-active-inference-ais
https://www.askviable.com/blog/why-we-chose-gpt-3-embeddings-for-the-clustering-behind-our-feedback-reports
31ddb5cc20ec19860de9803a3df7abce079680cb.html
https://www.brown.edu/news/2023-04-25/open-web-text
c0bcbe0d3fa93bef8abde06c137af0115131a2a8.html
https://www.buildt.ai/blog/viral-ripout
578673ced29982f87eb8e930f5e6d692a44fed4e.html
https://www.forbes.com/sites/thomasbrewster/2023/11/16/chatgpt-becomes-a-social-media-spy-assistant/
https://www.forefront.ai/blog-posts/how-to-fine-tune-gpt-neox
6e78db460ac938fc3f6a6d56837b2adf8ef99428.html
https://www.freaktakes.com/p/the-past-and-present-of-computer
https://www.lawfaremedia.org/article/chatgpt-unbound
https://www.lesswrong.com/posts/8viQEp8KBg2QSW4Yc/solidgoldmagikarp-iii-glitch-token-archaeology
https://www.lesswrong.com/posts/GyaDCzsyQgc48j8t3/linear-encoding-of-character-level-information-in-gpt-j
https://www.lesswrong.com/posts/PDLfpRwSynu73mxGw/basic-facts-about-language-model-internals-1
https://www.lesswrong.com/posts/SCqDipWAhZ49JNdmL/paper-llms-trained-on-a-is-b-fail-to-learn-b-is-a#eKhSncieBquLsFTXZ
https://www.lesswrong.com/posts/YKfNZAmiLdepDngwi/gpt-175bee
https://www.lesswrong.com/posts/axxnpQi8FyBPE4rbq/hutter-prize-for-prompts?commentId=WKNXFtQWzfSs9mGih
https://www.lesswrong.com/posts/dFbfCLZA4pejckeKc/a-mechanistic-explanation-for-solidgoldmagikarp-like-tokens
https://www.lesswrong.com/posts/etoMr4vcnP7joQHWa/notes-from-a-prompt-factory
https://www.lesswrong.com/posts/wqRqb7h6ZC48iDgfK/tentatively-found-600-monosemantic-features-in-a-small-lm
https://www.microsoft.com/en-us/research/blog/phi-2-the-surprising-power-of-small-language-models/
https://www.nature.com/articles/d41586-021-00530-0
https://www.nature.com/articles/s41586-023-06647-8
https://www.nytimes.com/interactive/2023/04/26/upshot/gpt-from-scratch.html
https://www.oneusefulthing.org/p/working-with-ai-two-paths-to-prompting
https://www.politico.eu/article/italian-privacy-regulator-bans-chatgpt/
https://www.reddit.com/r/ChatGPT/comments/12xai7j/spamming_the_word_stop_2300_times_or_probably_any/
https://www.reddit.com/r/ChatGPT/comments/15y4mqx/i_asked_chatgpt_to_maximize_its_censorship/
https://www.reddit.com/r/GPT3/comments/ra6nk4/had_gpt3_generate_the_onion_headlines/
e4720e5d01ba14d0e9b13bb2c42a6d37923c65df.html
https://www.reddit.com/r/GPT3/comments/tgud2t/my_new_favorite_thing_is_making_gpt3_create/
d13049475d40f32148a1a313cb8b0bc45edb885e.html
https://www.reddit.com/r/MachineLearning/comments/12xwzt9/d_be_careful_with_user_facing_apps_using_llms/
https://www.reddit.com/r/MachineLearning/comments/v42pej/p_this_is_the_worst_ai_ever_gpt4chan_model/
9d2c91d560fbecb82b5e11fc87b57cf391b9a95a.html
https://www.roft.io/
https://www.sfchronicle.com/projects/2021/jessica-simulation-artificial-intelligence/
70bbc07b9a27d32fd8b69bba025aceee6628bd66.html
https://www.youtube.com/watch?v=dO4TPJkeaaU
https://www.youtube.com/watch?v=kLC8AHZX4N8&t=394s
https://x.com/Altimor/status/1825659507617460439
https://x.com/ChrisJBakke/status/1736533308849443121
https://x.com/D_Rod_Tweets/status/1628030272745746432
https://x.com/DrJimFan/status/1631709224387624962
https://x.com/ESYudkowsky/status/1635577836525469697
https://x.com/NolanoOrg/status/1634027966651834370
https://x.com/OfficialLoganK/status/1664476604658069511
https://x.com/OpenAI/status/1676072388436594688
https://x.com/RiversHaveWings/status/1459646450275553285
https://x.com/arvind_io/status/1488257004783112192
https://x.com/bentossell/status/1598673037976543240
https://x.com/chillzaza_/status/1710795541087469647
https://x.com/colin_fraser/status/1630763222671454208
https://x.com/colin_fraser/status/1635350490484719618
https://x.com/colin_fraser/status/1635360285187018752
https://x.com/colin_fraser/status/1635450606013014016
https://x.com/cyrilzakka/status/1646532570597982208
https://x.com/dust4ai/status/1587104029712203778
https://x.com/emollick/status/1759633391098732967
https://x.com/fluffykittnmeow/status/1737639861350269213
https://x.com/francoisfleuret/status/1714531085512544760
https://x.com/goodside/status/1558622567635865600
https://x.com/jakezward/status/1728032639402037610
https://x.com/jamesjyu/status/1467568693806649346
https://x.com/jheitzeb/status/1612130278293803009
https://x.com/jjvincent/status/1648594881198039040
https://x.com/labenz/status/1611750398712332292
https://x.com/mathemagic1an/status/1595410144522813440
https://x.com/mayfer/status/1581388723635523584
https://x.com/mranti/status/1822959677216551095
https://x.com/parisba/status/1719523035450167535
https://x.com/repligate/status/1630593115407937536
https://x.com/sdtoyer/status/1729933591541670287
https://x.com/thesephist/status/1617747154231259137
https://x.com/thisisdaleb/status/1628891229562847233
https://x.com/thiteanish/status/1635188333705043969
https://x.com/voooooogel/status/1865189744776507809
https://x.com/wenquai/status/1748016021808595242
https://x.com/woj_zaremba/status/1191773448999034880
https://x.com/yacineMTB/status/1737523618832425273
Interpretable Contrastive Monte Carlo Tree Search Reasoning
https%253A%252F%252Farxiv.org%252Fabs%252F2410.01707.html
Ensemble everything everywhere: Multi-scale aggregation for adversarial robustness
https%253A%252F%252Farxiv.org%252Fabs%252F2408.05446.html
Token Erasure as a Footprint of Implicit Vocabulary Items in LLMs
https%253A%252F%252Farxiv.org%252Fabs%252F2406.20086.html
Resolving Discrepancies in Compute-Optimal Scaling of Language Models
https%253A%252F%252Farxiv.org%252Fabs%252F2406.19146.html
When Parts are Greater Than Sums: Individual LLM Components Can Outperform Full Models
https%253A%252F%252Farxiv.org%252Fabs%252F2406.13131.html
DataComp-LM: In search of the next generation of training sets for language models
Luke Zettlemoyer
https%253A%252F%252Farxiv.org%252Fabs%252F2406.11794.html
MCTSr: Accessing GPT-4 level Mathematical Olympiad Solutions via Monte Carlo Tree Self-refine with LLaMA-3-8B
https%253A%252F%252Farxiv.org%252Fabs%252F2406.07394.html
Superposed Decoding: Multiple Generations from a Single Autoregressive Inference Pass
https%253A%252F%252Farxiv.org%252Fabs%252F2405.18400.html
From r to Q^✱: Your Language Model is Secretly a Q-Function
https%253A%252F%252Farxiv.org%252Fabs%252F2404.12358.html
CulturalTeaming: AI-Assisted Interactive Red-Teaming for Challenging LLMs’ (Lack of) Multicultural Knowledge
https%253A%252F%252Farxiv.org%252Fabs%252F2404.06664.html
Actions Speak Louder than Words: Trillion-Parameter Sequential Transducers for Generative Recommendations (HSTU)
https%253A%252F%252Farxiv.org%252Fabs%252F2402.17152%2523facebook.html
Fast Adversarial Attacks on Language Models In One GPU Minute
https%253A%252F%252Farxiv.org%252Fabs%252F2402.15570.html
Autonomous Data Selection with Language Models for Mathematical Texts
https%253A%252F%252Farxiv.org%252Fabs%252F2402.07625.html
Grandmaster-Level Chess Without Search
https%253A%252F%252Farxiv.org%252Fabs%252F2402.04494%2523deepmind.html
SliceGPT: Compress Large Language Models by Deleting Rows and Columns
https%253A%252F%252Farxiv.org%252Fabs%252F2401.15024%2523microsoft.html
TinyLlama: An Open-Source Small Language Model
https%253A%252F%252Farxiv.org%252Fabs%252F2401.02385.html
TinyGPT-V: Efficient Multimodal Large Language Model via Small Backbones
https%253A%252F%252Farxiv.org%252Fabs%252F2312.16862.html
MEDITRON-70B: Scaling Medical Pretraining for Large Language Models
https%253A%252F%252Farxiv.org%252Fabs%252F2311.16079.html
OpenAI researchers warned board of AI breakthrough ahead of CEO ouster, sources say
https%253A%252F%252Fwww.reuters.com%252Ftechnology%252Fsam-altmans-ouster-openai-was-precipitated-by-letter-board-about-ai-breakthrough-2023-11-22%252F.html
OpenWebMath: An Open Dataset of High-Quality Mathematical Web Text
https%253A%252F%252Farxiv.org%252Fabs%252F2310.06786.html
MetaMath: Bootstrap Your Own Mathematical Questions for Large Language Models
https%253A%252F%252Farxiv.org%252Fabs%252F2309.12284.html
Language Modeling Is Compression
https%253A%252F%252Farxiv.org%252Fabs%252F2309.10668%2523deepmind.html
Large Language Models Sometimes Generate Purely Negatively-Reinforced Text
https%253A%252F%252Farxiv.org%252Fabs%252F2306.07567.html
DoReMi: Optimizing Data Mixtures Speeds Up Language Model Pretraining
Percy Liang
https%253A%252F%252Farxiv.org%252Fabs%252F2305.10429%2523google.html
Inflection AI, Startup From Ex-DeepMind Leaders, Launches Pi—A Chattier Chatbot
https%253A%252F%252Fwww.forbes.com%252Fsites%252Falexkonrad%252F2023%252F05%252F02%252Finflection-ai-ex-deepmind-launches-pi-chatbot%252F.html
Shall We Pretrain Autoregressive Language Models with Retrieval? A Comprehensive Study
https%253A%252F%252Farxiv.org%252Fabs%252F2304.06762%2523nvidia.html
How Large-Language Models Can Revolutionize Military Planning
https%253A%252F%252Fwarontherocks.com%252F2023%252F04%252Fhow-large-language-models-can-revolutionize-military-planning%252F.html
The Quantization Model of Neural Scaling
https%253A%252F%252Farxiv.org%252Fabs%252F2303.13506.html
Int-4 LLaMa is not enough—Int-3 and beyond: More compression, easier to build apps on LLMs that run locally
https%253A%252F%252Fnolanoorg.substack.com%252Fp%252Fint-4-llama-is-not-enough-int-3-and.html
Beyond the Pass Mark: the Accuracy of ChatGPT and Bing in the National Medical Licensure Examination in Japan
https%253A%252F%252Fosf.io%252F5uxra%252F.html
SpikeGPT: Generative Pre-trained Language Model with Spiking Neural Networks
https%253A%252F%252Farxiv.org%252Fabs%252F2302.13939.html
Data Selection for Language Models via Importance Resampling
Percy Liang
https%253A%252F%252Farxiv.org%252Fabs%252F2302.03169.html
A New Chat Bot Is a ‘Code Red’ for Google’s Search Business: A new wave of chat bots like ChatGPT use artificial intelligence that could reinvent or even replace the traditional internet search engine
https://x.com/cademetz
https%253A%252F%252Fwww.nytimes.com%252F2022%252F12%252F21%252Ftechnology%252Fai-chatgpt-google-search.html.html
SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Models
https%253A%252F%252Farxiv.org%252Fabs%252F2211.10438.html
InstructPix2Pix: Learning to Follow Image Editing Instructions
https%253A%252F%252Farxiv.org%252Fabs%252F2211.09800.html
Galactica: A Large Language Model for Science
https%253A%252F%252Farxiv.org%252Fabs%252F2211.09085%2523facebook.html
Large Language Models Struggle to Learn Long-Tail Knowledge
Colin Raffel
https%253A%252F%252Farxiv.org%252Fabs%252F2211.08411.html
GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers
https%253A%252F%252Farxiv.org%252Fabs%252F2210.17323.html
Evaluating Parameter Efficient Learning for Generation
https%253A%252F%252Farxiv.org%252Fabs%252F2210.13673%2523nvidia.html
BioGPT: Generative Pre-trained Transformer for Biomedical Text Generation and Mining
https%253A%252F%252Farxiv.org%252Fabs%252F2210.10341%2523microsoft.html
Arithmetic Sampling: Parallel Diverse Decoding for Large Language Models
https%253A%252F%252Farxiv.org%252Fabs%252F2210.15458%2523google.html
Foundation Transformers
Furu Wei
https%253A%252F%252Farxiv.org%252Fabs%252F2210.06423%2523microsoft.html
Ask Me Anything (AMA): A simple strategy for prompting language models
https%253A%252F%252Farxiv.org%252Fabs%252F2210.02441.html
Is Reinforcement Learning (Not) for Natural Language Processing: Benchmarks, Baselines, and Building Blocks for Natural Language Policy Optimization
Hannaneh Hajishirzi—University of Washington
https%253A%252F%252Farxiv.org%252Fabs%252F2210.01241.html
LM-Nav: Robotic Navigation with Large Pre-Trained Models of Language, Vision, and Action
Sergey Levine
https%253A%252F%252Farxiv.org%252Fabs%252F2207.04429.html
ZeroQuant: Efficient and Affordable Post-Training Quantization for Large-Scale Transformers
https%253A%252F%252Farxiv.org%252Fabs%252F2206.01861%2523microsoft.html
Shared computational principles for language processing in humans and deep language models
Omer Levy
https%253A%252F%252Fwww.nature.com%252Farticles%252Fs41593-022-01026-4.html
Vector-quantized Image Modeling with Improved VQGAN
https%253A%252F%252Farxiv.org%252Fabs%252F2110.04627%2523google.html
Brains and algorithms partially converge in natural language processing
https%253A%252F%252Fwww.nature.com%252Farticles%252Fs42003-022-03036-1.html
Using DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, A Large-Scale Generative Language Model
https%253A%252F%252Farxiv.org%252Fabs%252F2201.11990%2523microsoftnvidia.html
WANLI: Worker and AI Collaboration for Natural Language Inference Dataset Creation
Noah A. Smith
https%253A%252F%252Fswabhs.com%252Fassets%252Fpdf%252Fwanli.pdf%2523allen.html
Improving language models by retrieving from trillions of tokens
Karen Simonyan
https%253A%252F%252Farxiv.org%252Fabs%252F2112.04426%2523deepmind.html
True Few-Shot Learning with Prompts—A Real-World Perspective
https%253A%252F%252Farxiv.org%252Fabs%252F2111.13440.html
CLUES: Few-Shot Learning Evaluation in Natural Language Understanding
Jianfeng Gao at Microsoft Research
https%253A%252F%252Farxiv.org%252Fabs%252F2111.02570%2523microsoft.html
Fast Model Editing at Scale
https%253A%252F%252Farxiv.org%252Fabs%252F2110.11309.html
General-Purpose Question-Answering with Macaw
https%253A%252F%252Farxiv.org%252Fabs%252F2109.02593%2523allen.html
RASP: Thinking Like Transformers
https%253A%252F%252Farxiv.org%252Fabs%252F2106.06981.html
ByT5: Towards a token-free future with pre-trained byte-to-byte models
Colin Raffel
https%253A%252F%252Farxiv.org%252Fabs%252F2105.13626%2523google.html
Naver unveils first ‘hyperscale’ AI platform
https%253A%252F%252Fm.koreaherald.com%252Fview.php%253Fud%253D20210525000824%2523naver.html
Generative Language Modeling for Automated Theorem Proving
https%253A%252F%252Farxiv.org%252Fabs%252F2009.03393%2523openai.html
Climbing towards NLU: On Meaning, Form, and Understanding in the Age of Data
https%253A%252F%252Faclanthology.org%252F2020.acl-main.463.pdf.html
Scaling Laws from the Data Manifold Dimension
Jared Kaplan
https%253A%252F%252Farxiv.org%252Fabs%252F2004.10802.html
Scaling Laws for Neural Language Models
Jared Kaplan
Sam McCandlish
Alec Radford
https%253A%252F%252Farxiv.org%252Fabs%252F2001.08361%2523openai.html
Reformer: The Efficient Transformer
Łukasz Kaiser
https%253A%252F%252Farxiv.org%252Fabs%252F2001.04451%2523google.html
CTRL: A Conditional Transformer Language Model For Controllable Generation
Caiming Xiong—Home Page
Richard Socher
https%253A%252F%252Farxiv.org%252Fabs%252F1909.05858%2523salesforce.html
Language Modeling State-of-the-art leaderboards
https%253A%252F%252Fpaperswithcode.com%252Ftask%252Flanguage-modelling.html
Transformer-XL: Attentive Language Models Beyond a Fixed-Length Context
Zihang Dai
Zhilin Yang
https://www.cs.cmu.edu/~./yiming/
https%253A%252F%252Farxiv.org%252Fabs%252F1901.02860.html
Music Transformer: Generating Music with Long-Term Structure
https%253A%252F%252Fmagenta.tensorflow.org%252Fmusic-transformer.html
GPT-1: Improving Language Understanding by Generative Pre-Training § Model specifications
Alec Radford
Tim Salimans
https%253A%252F%252Fs3-us-west-2.amazonaws.com%252Fopenai-assets%252Fresearch-covers%252Flanguage-unsupervised%252Flanguage_understanding_paper.pdf%2523page%253D5.html
Homepage of Paul F. Christiano
https%253A%252F%252Fpaulfchristiano.com%252F.html
Wikipedia Bibliography: