- See Also
-
Links
- “Tractable Control for Autoregressive Language Generation”, Et Al 2023
- “Characterizing Attribution and Fluency Tradeoffs for Retrieval-Augmented Large Language Models”, Et Al 2023
- “Witscript 3: A Hybrid AI System for Improvising Jokes in a Conversation”, 2023
- “Fast Inference from Transformers via Speculative Decoding”, Et Al 2022
- “The CRINGE Loss: Learning What Language Not to Model”, Et Al 2022
- “Contrastive Search Is What You Need For Neural Text Generation”, 2022
- “Help Me Write a Poem: Instruction Tuning As a Vehicle for Collaborative Poetry Writing (CoPoet)”, Et Al 2022
- “Arithmetic Sampling: Parallel Diverse Decoding for Large Language Models”, Et Al 2022
- “Most Language Models Can Be Poets Too: An AI Writing Assistant and Constrained Text Generation Studio”, Et Al 2022
- “Ask Me Anything (AMA): A Simple Strategy for Prompting Language Models”, Et Al 2022
- “Out of One, Many: Using Language Models to Simulate Human Samples”, Et Al 2022
- “Red Teaming Language Models to Reduce Harms: Methods, Scaling Behaviors, and Lessons Learned”, Et Al 2022
- “Effidit: Your AI Writing Assistant”, Et Al 2022
- “DIRECTOR: Generator-Classifiers For Supervised Language Modeling”, Et Al 2022
- “RankGen: Improving Text Generation With Large Ranking Models”, Et Al 2022
- “Controllable Natural Language Generation With Contrastive Prefixes”, Et Al 2022
- “Using Natural Language Prompts for Machine Translation”, 2022
- “A Contrastive Framework for Neural Text Generation”, Et Al 2022
- “Typical Decoding for Natural Language Generation”, Et Al 2022
- “FIGARO: Generating Symbolic Music With Fine-Grained Artistic Control”, Et Al 2022
- “A Survey of Controllable Text Generation Using Transformer-based Pre-trained Language Models”, Et Al 2022
- “NeuroLogic A*Esque Decoding: Constrained Text Generation With Lookahead Heuristics”, Et Al 2021
- “Program Synthesis With Large Language Models”, Et Al 2021
- “Scarecrow: A Framework for Scrutinizing Machine Text”, Et Al 2021
- “Time-Aware Language Models As Temporal Knowledge Bases”, Et Al 2021
- “Choose-Your-Own-Adventure AI Dungeon Games”, Gwern 2021
- “Machine Translation Decoding beyond Beam Search”, Et Al 2021
- “Controllable Generation from Pre-trained Language Models via Inverse Prompting”, Et Al 2021
- “Improving Diversity of Neural Text Generation via Inverse Probability Weighting”, Et Al 2021
- “There Once Was a Really Bad Poet, It Was Automated but You Didn’t Know It”, Et Al 2021
- “MAUVE: Measuring the Gap Between Neural Text and Human Text Using Divergence Frontiers”, Et Al 2021
- “Prefix-Tuning: Optimizing Continuous Prompts for Generation”, 2021
- “Bot-Adversarial Dialogue for Safe Conversational Agents”, Et Al 2021
- “Collaborative Storytelling With Large-scale Neural Language Models”, Et Al 2020
- “NeuroLogic Decoding: (Un)supervised Neural Text Generation With Predicate Logic Constraints”, Et Al 2020
- “Interacting With GPT-2 to Generate Controlled and Believable Musical Sequences in ABC Notation”, Geerlings & Meroño-2020
- “MEGATRON-CNTRL: Controllable Story Generation With External Knowledge Using Large-Scale Language Models”, Et Al 2020
- “Weird AI Yankovic: Generating Parody Lyrics”, 2020
- “A Systematic Characterization of Sampling Algorithms for Open-ended Language Generation”, Et Al 2020
- “GeDi: Generative Discriminator Guided Sequence Generation”, Et Al 2020
- “Mirostat: A Neural Text Decoding Algorithm That Directly Controls Perplexity”, Et Al 2020
- “Progressive Generation of Long Text”, Et Al 2020
- “GPT-3 Creative Fiction”, Gwern 2020
- “This Word Does Not Exist”, 2020
- “True_poetry: Poetry Generator by GPT-2 With Meter and Rhyme Constraints”, Summers-2020
- “Blender: A State-of-the-art Open Source Chatbot”, Et Al 2020
- “Trading Off Diversity and Quality in Natural Language Generation”, Et Al 2020
- “Rapformer: Conditional Rap Lyrics Generation With Denoising Autoencoders”, Et Al 2020
- “A Hundred Visions and Revisions”, 2020
- “Top-K Training of GANs: Improving GAN Performance by Throwing Away Bad Samples”, Et Al 2020
- “Towards a Human-like Open-Domain Chatbot”, Et Al 2020
- “Controlling Text Generation With Plug and Play Language Models”, Et Al 2019
- “Plug and Play Language Models: A Simple Approach to Controlled Text Generation”, Et Al 2019
- “CTRL: A Conditional Transformer Language Model For Controllable Generation”, Et Al 2019
- “Neural Text Generation With Unlikelihood Training”, Et Al 2019
- “GROVER: Defending Against Neural Fake News”, Et Al 2019
- “The Curious Case of Neural Text Degeneration”, Et Al 2019
- “Good News, Everyone! Context Driven Entity-aware Captioning for News Images”, Et Al 2019
- “GPT-2 Neural Network Poetry”, Gwern & 2019
- “Controlling Linguistic Style Aspects in Neural Language Generation”, 2017
- “Language Generation With Recurrent Generative Adversarial Networks without Pre-training”, Et Al 2017
- “A Deep Reinforced Model for Abstractive Summarization”, Et Al 2017
- “Learning to Generate Reviews and Discovering Sentiment”, Et Al 2017
- “Improving Neural Machine Translation With Conditional Sequence Generative Adversarial Nets”, Et Al 2017
- “Tuning Recurrent Neural Networks With Reinforcement Learning”, Et Al 2017
- “Google’s Multilingual Neural Machine Translation System: Enabling Zero-Shot Translation”, Et Al 2016
- “WaveNet: A Generative Model for Raw Audio”, Et Al 2016
- “Sequence Level Training With Recurrent Neural Networks”, Et Al 2015
- “Generative Concatenative Nets Jointly Learn to Write and Classify Reviews”, Et Al 2015
- “Semi-supervised Sequence Learning”, 2015
- “RNN Metadata for Mimicking Author Style”, Gwern 2015
- “Scheduled Sampling for Sequence Prediction With Recurrent Neural Networks”, Et Al 2015
- “Pixels Still Beat Text: Attacking the OpenAI CLIP Model With Text Patches and Adversarial Pixel Perturbations”
- “Apple or IPod? Easy Fix for Adversarial Textual Attacks on OpenAI’s CLIP Model!”
- Wikipedia
- Miscellaneous
- Link Bibliography
See Also
Links
“Tractable Control for Autoregressive Language Generation”, Et Al 2023
“Characterizing Attribution and Fluency Tradeoffs for Retrieval-Augmented Large Language Models”, Et Al 2023
“Characterizing Attribution and Fluency Tradeoffs for Retrieval-Augmented Large Language Models”
“Witscript 3: A Hybrid AI System for Improvising Jokes in a Conversation”, 2023
“Witscript 3: A Hybrid AI System for Improvising Jokes in a Conversation”
“Fast Inference from Transformers via Speculative Decoding”, Et Al 2022
“The CRINGE Loss: Learning What Language Not to Model”, Et Al 2022
“Contrastive Search Is What You Need For Neural Text Generation”, 2022
“Contrastive Search Is What You Need For Neural Text Generation”
“Help Me Write a Poem: Instruction Tuning As a Vehicle for Collaborative Poetry Writing (CoPoet)”, Et Al 2022
“Help me write a poem: Instruction Tuning as a Vehicle for Collaborative Poetry Writing (CoPoet)”
“Arithmetic Sampling: Parallel Diverse Decoding for Large Language Models”, Et Al 2022
“Arithmetic Sampling: Parallel Diverse Decoding for Large Language Models”
“Most Language Models Can Be Poets Too: An AI Writing Assistant and Constrained Text Generation Studio”, Et Al 2022
“Ask Me Anything (AMA): A Simple Strategy for Prompting Language Models”, Et Al 2022
“Ask Me Anything (AMA): A simple strategy for prompting language models”
“Out of One, Many: Using Language Models to Simulate Human Samples”, Et Al 2022
“Out of One, Many: Using Language Models to Simulate Human Samples”
“Red Teaming Language Models to Reduce Harms: Methods, Scaling Behaviors, and Lessons Learned”, Et Al 2022
“Red Teaming Language Models to Reduce Harms: Methods, Scaling Behaviors, and Lessons Learned”
“Effidit: Your AI Writing Assistant”, Et Al 2022
“DIRECTOR: Generator-Classifiers For Supervised Language Modeling”, Et Al 2022
“DIRECTOR: Generator-Classifiers For Supervised Language Modeling”
“RankGen: Improving Text Generation With Large Ranking Models”, Et Al 2022
“RankGen: Improving Text Generation with Large Ranking Models”
“Controllable Natural Language Generation With Contrastive Prefixes”, Et Al 2022
“Controllable Natural Language Generation with Contrastive Prefixes”
“Using Natural Language Prompts for Machine Translation”, 2022
“A Contrastive Framework for Neural Text Generation”, Et Al 2022
“Typical Decoding for Natural Language Generation”, Et Al 2022
“FIGARO: Generating Symbolic Music With Fine-Grained Artistic Control”, Et Al 2022
“FIGARO: Generating Symbolic Music with Fine-Grained Artistic Control”
“A Survey of Controllable Text Generation Using Transformer-based Pre-trained Language Models”, Et Al 2022
“A Survey of Controllable Text Generation using Transformer-based Pre-trained Language Models”
“NeuroLogic A*Esque Decoding: Constrained Text Generation With Lookahead Heuristics”, Et Al 2021
“NeuroLogic A*esque Decoding: Constrained Text Generation with Lookahead Heuristics”
“Program Synthesis With Large Language Models”, Et Al 2021
“Scarecrow: A Framework for Scrutinizing Machine Text”, Et Al 2021
“Time-Aware Language Models As Temporal Knowledge Bases”, Et Al 2021
“Choose-Your-Own-Adventure AI Dungeon Games”, Gwern 2021
“Machine Translation Decoding beyond Beam Search”, Et Al 2021
“Controllable Generation from Pre-trained Language Models via Inverse Prompting”, Et Al 2021
“Controllable Generation from Pre-trained Language Models via Inverse Prompting”
“Improving Diversity of Neural Text Generation via Inverse Probability Weighting”, Et Al 2021
“Improving Diversity of Neural Text Generation via Inverse Probability Weighting”
“There Once Was a Really Bad Poet, It Was Automated but You Didn’t Know It”, Et Al 2021
“There Once Was a Really Bad Poet, It Was Automated but You Didn’t Know It”
“MAUVE: Measuring the Gap Between Neural Text and Human Text Using Divergence Frontiers”, Et Al 2021
“MAUVE: Measuring the Gap Between Neural Text and Human Text using Divergence Frontiers”
“Prefix-Tuning: Optimizing Continuous Prompts for Generation”, 2021
“Prefix-Tuning: Optimizing Continuous Prompts for Generation”
“Bot-Adversarial Dialogue for Safe Conversational Agents”, Et Al 2021
“Collaborative Storytelling With Large-scale Neural Language Models”, Et Al 2020
“Collaborative Storytelling with Large-scale Neural Language Models”
“NeuroLogic Decoding: (Un)supervised Neural Text Generation With Predicate Logic Constraints”, Et Al 2020
“NeuroLogic Decoding: (Un)supervised Neural Text Generation with Predicate Logic Constraints”
“Interacting With GPT-2 to Generate Controlled and Believable Musical Sequences in ABC Notation”, Geerlings & Meroño-2020
“Interacting with GPT-2 to Generate Controlled and Believable Musical Sequences in ABC Notation”
“MEGATRON-CNTRL: Controllable Story Generation With External Knowledge Using Large-Scale Language Models”, Et Al 2020
“Weird AI Yankovic: Generating Parody Lyrics”, 2020
“A Systematic Characterization of Sampling Algorithms for Open-ended Language Generation”, Et Al 2020
“A Systematic Characterization of Sampling Algorithms for Open-ended Language Generation”
“GeDi: Generative Discriminator Guided Sequence Generation”, Et Al 2020
“Mirostat: A Neural Text Decoding Algorithm That Directly Controls Perplexity”, Et Al 2020
“Mirostat: A Neural Text Decoding Algorithm that Directly Controls Perplexity”
“Progressive Generation of Long Text”, Et Al 2020
“GPT-3 Creative Fiction”, Gwern 2020
“This Word Does Not Exist”, 2020
“True_poetry: Poetry Generator by GPT-2 With Meter and Rhyme Constraints”, Summers-2020
“true_poetry: Poetry generator by GPT-2 with meter and rhyme constraints”
“Blender: A State-of-the-art Open Source Chatbot”, Et Al 2020
“Trading Off Diversity and Quality in Natural Language Generation”, Et Al 2020
“Trading Off Diversity and Quality in Natural Language Generation”
“Rapformer: Conditional Rap Lyrics Generation With Denoising Autoencoders”, Et Al 2020
“Rapformer: Conditional Rap Lyrics Generation with Denoising Autoencoders”
“A Hundred Visions and Revisions”, 2020
“Top-K Training of GANs: Improving GAN Performance by Throwing Away Bad Samples”, Et Al 2020
“Top-K Training of GANs: Improving GAN Performance by Throwing Away Bad Samples”
“Towards a Human-like Open-Domain Chatbot”, Et Al 2020
“Controlling Text Generation With Plug and Play Language Models”, Et Al 2019
“Controlling Text Generation with Plug and Play Language Models”
“Plug and Play Language Models: A Simple Approach to Controlled Text Generation”, Et Al 2019
“Plug and Play Language Models: A Simple Approach to Controlled Text Generation”
“CTRL: A Conditional Transformer Language Model For Controllable Generation”, Et Al 2019
“CTRL: A Conditional Transformer Language Model For Controllable Generation”
“Neural Text Generation With Unlikelihood Training”, Et Al 2019
“GROVER: Defending Against Neural Fake News”, Et Al 2019
“The Curious Case of Neural Text Degeneration”, Et Al 2019
“Good News, Everyone! Context Driven Entity-aware Captioning for News Images”, Et Al 2019
“Good News, Everyone! Context driven entity-aware captioning for news images”
“GPT-2 Neural Network Poetry”, Gwern & 2019
“Controlling Linguistic Style Aspects in Neural Language Generation”, 2017
“Controlling Linguistic Style Aspects in Neural Language Generation”
“Language Generation With Recurrent Generative Adversarial Networks without Pre-training”, Et Al 2017
“Language Generation with Recurrent Generative Adversarial Networks without Pre-training”
“A Deep Reinforced Model for Abstractive Summarization”, Et Al 2017
“Learning to Generate Reviews and Discovering Sentiment”, Et Al 2017
“Improving Neural Machine Translation With Conditional Sequence Generative Adversarial Nets”, Et Al 2017
“Improving Neural Machine Translation with Conditional Sequence Generative Adversarial Nets”
“Tuning Recurrent Neural Networks With Reinforcement Learning”, Et Al 2017
“Tuning Recurrent Neural Networks with Reinforcement Learning”
“Google’s Multilingual Neural Machine Translation System: Enabling Zero-Shot Translation”, Et Al 2016
“Google’s Multilingual Neural Machine Translation System: Enabling Zero-Shot Translation”
“WaveNet: A Generative Model for Raw Audio”, Et Al 2016
“Sequence Level Training With Recurrent Neural Networks”, Et Al 2015
“Generative Concatenative Nets Jointly Learn to Write and Classify Reviews”, Et Al 2015
“Generative Concatenative Nets Jointly Learn to Write and Classify Reviews”
“Semi-supervised Sequence Learning”, 2015
“RNN Metadata for Mimicking Author Style”, Gwern 2015
“Scheduled Sampling for Sequence Prediction With Recurrent Neural Networks”, Et Al 2015
“Scheduled Sampling for Sequence Prediction with Recurrent Neural Networks”
“Pixels Still Beat Text: Attacking the OpenAI CLIP Model With Text Patches and Adversarial Pixel Perturbations”
“Apple or IPod? Easy Fix for Adversarial Textual Attacks on OpenAI’s CLIP Model!”
“Apple or iPod? Easy Fix for Adversarial Textual Attacks on OpenAI's CLIP Model!”
Wikipedia
Miscellaneous
-
/doc/ai/nn/sampling/2022-arora-figure4-amapromptgenerationscalingvskshotwithmodelsize.png
-
/doc/ai/nn/sampling/2020-roller-facebook-blenderchatbot-ratedperformancevshumans.jpg
-
http://mi.eng.cam.ac.uk/projects/cued-rnnlm/papers/Interspeech15.pdf
-
https://homepages.inf.ed.ac.uk/abmayne/publications/sennrich2016NAACL.pdf
-
https://workshop2015.iwslt.org/downloads/IWSLT_2015_RP_13.pdf
Link Bibliography
-
https://arxiv.org/abs/2210.14140
: “Contrastive Search Is What You Need For Neural Text Generation”, Yixuan Su, Nigel Collier -
https://arxiv.org/abs/2210.13669
: “Help Me Write a Poem: Instruction Tuning As a Vehicle for Collaborative Poetry Writing (CoPoet)”, Tuhin Chakrabarty, Vishakh Padmakumar, He He -
https://arxiv.org/abs/2210.15458#google
: “Arithmetic Sampling: Parallel Diverse Decoding for Large Language Models”, Luke Vilnis, Yury Zemlyanskiy, Patrick Murray, Alexandre Passos, Sumit Sanghai -
https://aclanthology.org/2022.cai-1.2.pdf
: “Most Language Models Can Be Poets Too: An AI Writing Assistant and Constrained Text Generation Studio”, Allen Roush, Sanjay Basu, Akshay Moorthy, Dmitry Dubovoy -
https://arxiv.org/abs/2210.02441
: “Ask Me Anything (AMA): A Simple Strategy for Prompting Language Models”, -
https://www.anthropic.com/red_teaming.pdf
: “Red Teaming Language Models to Reduce Harms: Methods, Scaling Behaviors, and Lessons Learned”, -
https://arxiv.org/abs/2202.11822#google
: “Using Natural Language Prompts for Machine Translation”, Xavier Garcia, Orhan Firat -
https://arxiv.org/abs/2107.01294#allen
: “Scarecrow: A Framework for Scrutinizing Machine Text”, Yao Dou, Maxwell Forbes, Rik Koncel-Kedziorski, Noah A. Smith, Yejin Choi -
cyoa
: “Choose-Your-Own-Adventure AI Dungeon Games”, gwern -
https://arxiv.org/abs/2101.00190
: “Prefix-Tuning: Optimizing Continuous Prompts for Generation”, Xiang Lisa Li, Percy Liang -
https://aclanthology.org/2021.naacl-main.235.pdf#facebook
: “Bot-Adversarial Dialogue for Safe Conversational Agents”, Jing Xu, Da Ju, Margaret Li, Y-Lan Boureau, Jason Weston, Emily Dinan -
gpt-3
: “GPT-3 Creative Fiction”, gwern -
https://www.thisworddoesnotexist.com/
: “This Word Does Not Exist”, Thomas Dimson -
https://ai.facebook.com/blog/state-of-the-art-open-source-chatbot
: “Blender: A State-of-the-art Open Source Chatbot”, Stephen Roller, Jason Weston, Emily Dinan -
https://arxiv.org/abs/2004.03965
: “Rapformer: Conditional Rap Lyrics Generation With Denoising Autoencoders”, Nikola I. Nikolov, Eric Malmi, Curtis G. Northcutt, Loreto Parisi -
https://www.uber.com/blog/pplm/
: “Controlling Text Generation With Plug and Play Language Models”, Rosanne Liu, Sumanth Dathathri, Andrea Madotto, Piero Molino, Jason Yosinski -
https://arxiv.org/abs/1909.05858#salesforce
: “CTRL: A Conditional Transformer Language Model For Controllable Generation”, Nitish Shirish Keskar, Bryan McCann, Lav R. Varshney, Caiming Xiong, Richard Socher (Salesforce) -
gpt-2
: “GPT-2 Neural Network Poetry”, gwern, Shawn Presser -
rnn-metadata
: “RNN Metadata for Mimicking Author Style”, gwern