“‘NN Sampling’ Tag”,2019-10-29 (inner monologue (AI), poetry by AI, RL exploration, AlphaGo, preference learning; backlinks)⁠:

Bibliography for tag ai/nn/sampling, most recent first: 2 related tags, 118 annotations, & 17 links (parent).

• See Also
• Gwern
  • “Research Ideas”, Gwern2017
  • “GPT-3 Creative Fiction”, Gwern2020
  • “Choose-Your-Own-Adventure AI Dungeon Games”, Gwern2021
  • “RNN Metadata for Mimicking Author Style”, Gwern2015
• Links
  • “Do LLMs Estimate Uncertainty Well in Instruction-Following?”, Heo et al 2024
  • “SimpleStrat: Diversifying Language Model Generation With Stratification”, Wong et al 2024
  • “Me, Myself, and AI: The Situational Awareness Dataset (SAD) for LLMs”, Laine et al 2024
  • “Be like a Goldfish, Don’t Memorize! Mitigating Memorization in Generative LLMs”, Hans et al 2024
  • “Superposed Decoding: Multiple Generations from a Single Autoregressive Inference Pass”, Shen et al 2024
  • “Probabilistic Inference in Language Models via Twisted Sequential Monte Carlo”, Zhao et al 2024
  • “Σ-GPTs: A New Approach to Autoregressive Models”, Pannatier et al 2024
  • “LLM Evaluators Recognize and Favor Their Own Generations”, Panickssery et al 2024
  • “Re-Evaluating GPT-4’s Bar Exam Performance”, Martínez2024
  • “Quiet-STaR: Language Models Can Teach Themselves to Think Before Speaking”, Zelikman et al 2024
  • “Monitoring AI-Modified Content at Scale: A Case Study on the Impact of ChatGPT on AI Conference Peer Reviews”, Liang et al 2024
  • “Chain-Of-Thought Reasoning Without Prompting”, Wang & Zhou2024
  • “The Non-Effect of Sampling Temperature on Problem Solving in GPT-3.5/GPT-4”, Renze & Guven2024
  • “Blending Is All You Need: Cheaper, Better Alternative to Trillion-Parameters LLM”, Lu et al 2024
  • “GIVT: Generative Infinite-Vocabulary Transformers”, Tschannen et al 2023
  • “Universal Self-Consistency for Large Language Model Generation”, Chen et al 2023
  • “Controlled Text Generation via Language Model Arithmetic”, Dekoninck et al 2023
  • “Language Model Inversion”, Morris et al 2023
  • “SEDD: Discrete Diffusion Modeling by Estimating the Ratios of the Data Distribution”, Lou et al 2023
  • “Let Models Speak Ciphers: Multiagent Debate through Embeddings”, Pham et al 2023
  • “Contrastive Decoding Improves Reasoning in Large Language Models”, O’Brien & Lewis2023
  • “Accelerating LLM Inference With Staged Speculative Decoding”, Spector & Re2023
  • “Efficient Guided Generation for Large Language Models”, Willard & Louf2023
  • “Copy Is All You Need”, Lan et al 2023
  • “Stay on Topic With Classifier-Free Guidance”, Sanchez et al 2023
  • “Sequential Monte Carlo Steering of Large Language Models Using Probabilistic Programs”, Lew et al 2023
  • “How Language Model Hallucinations Can Snowball”, Zhang et al 2023
  • “Tractable Control for Autoregressive Language Generation”, Zhang et al 2023
  • “MUX-PLMs: Pre-Training Language Models With Data Multiplexing”, Murahari et al 2023
  • “Characterizing Attribution and Fluency Tradeoffs for Retrieval-Augmented Large Language Models”, Aksitov et al 2023
  • “DataMUX: Data Multiplexing for Neural Networks”, Murahari et al 2023
  • “Witscript 3: A Hybrid AI System for Improvising Jokes in a Conversation”, Toplyn2023
  • “A Survey on Text Generation Using Generative Adversarial Networks”, Rosa & Papa2022
  • “Fast Inference from Transformers via Speculative Decoding”, Leviathan et al 2022
  • “The CRINGE Loss: Learning What Language Not to Model”, Adolphs et al 2022
  • “Contrastive Decoding: Open-Ended Text Generation As Optimization”, Li et al 2022
  • “Help Me Write a Poem: Instruction Tuning As a Vehicle for Collaborative Poetry Writing (CoPoet)”, Chakrabarty et al 2022
  • “Contrastive Search Is What You Need For Neural Text Generation”, Su & Collier2022
  • “Arithmetic Sampling: Parallel Diverse Decoding for Large Language Models”, Vilnis et al 2022
  • “Most Language Models Can Be Poets Too: An AI Writing Assistant and Constrained Text Generation Studio”, Roush et al 2022
  • “Ask Me Anything (AMA): A Simple Strategy for Prompting Language Models”, Arora et al 2022
  • “Out of One, Many: Using Language Models to Simulate Human Samples”, Argyle et al 2022
  • “Red Teaming Language Models to Reduce Harms: Methods, Scaling Behaviors, and Lessons Learned”, Ganguli et al 2022
  • “Effidit: Your AI Writing Assistant”, Shi et al 2022
  • “DIRECTOR: Generator-Classifiers For Supervised Language Modeling”, Arora et al 2022
  • “RankGen: Improving Text Generation With Large Ranking Models”, Krishna et al 2022
  • “Time Control: Language Modeling via Stochastic Processes”, Wang et al 2022
  • “Controllable Natural Language Generation With Contrastive Prefixes”, Qian et al 2022
  • “Using Natural Language Prompts for Machine Translation”, Garcia & Firat2022
  • “A Contrastive Framework for Neural Text Generation”, Su et al 2022
  • “Typical Decoding for Natural Language Generation”, Meister et al 2022
  • “FIGARO: Generating Symbolic Music With Fine-Grained Artistic Control”, Rütte et al 2022
  • “A Survey of Controllable Text Generation Using Transformer-Based Pre-Trained Language Models”, Zhang et al 2022
  • “FRUIT: Faithfully Reflecting Updated Information in Text”, IV et al 2021
  • “NeuroLogic A^✱esque Decoding: Constrained Text Generation With Lookahead Heuristics”, Lu et al 2021
  • “Relating Neural Text Degeneration to Exposure Bias”, Chiang & Chen2021
  • “Program Synthesis With Large Language Models”, Austin et al 2021
  • “Scarecrow: A Framework for Scrutinizing Machine Text”, Dou et al 2021
  • “Time-Aware Language Models As Temporal Knowledge Bases”, Dhingra et al 2021
  • “Machine Translation Decoding beyond Beam Search”, Leblond et al 2021
  • “Controllable Generation from Pre-Trained Language Models via Inverse Prompting”, Zou et al 2021
  • “Improving Diversity of Neural Text Generation via Inverse Probability Weighting”, Zhang et al 2021
  • “There Once Was a Really Bad Poet, It Was Automated but You Didn’t Know It”, Wang et al 2021
  • “A^✱ Search Without Expansions: Learning Heuristic Functions With Deep Q-Networks”, Agostinelli et al 2021
  • “MAUVE: Measuring the Gap Between Neural Text and Human Text Using Divergence Frontiers”, Pillutla et al 2021
  • “Prefix-Tuning: Optimizing Continuous Prompts for Generation”, Li & Liang2021
  • “Bot-Adversarial Dialogue for Safe Conversational Agents”, Xu et al 2021
  • “Collaborative Storytelling With Large-Scale Neural Language Models”, Nichols et al 2020
  • “NeuroLogic Decoding: (Un)supervised Neural Text Generation With Predicate Logic Constraints”, Lu et al 2020
  • “Interacting With GPT-2 to Generate Controlled and Believable Musical Sequences in ABC Notation”, Geerlings & Meroño-Peñuela2020
  • “Training Independent Subnetworks for Robust Prediction”, Havasi et al 2020
  • “MEGATRON-CNTRL: Controllable Story Generation With External Knowledge Using Large-Scale Language Models”, Xu et al 2020
  • “Weird AI Yankovic: Generating Parody Lyrics”, Riedl2020
  • “A Systematic Characterization of Sampling Algorithms for Open-Ended Language Generation”, Nadeem et al 2020
  • “GeDi: Generative Discriminator Guided Sequence Generation”, Krause et al 2020
  • “Mirostat: A Neural Text Decoding Algorithm That Directly Controls Perplexity”, Basu et al 2020
  • “Progressive Generation of Long Text”, Tan et al 2020
  • “This Word Does Not Exist”, Dimson2020
  • “True_poetry: Poetry Generator by GPT-2 With Meter and Rhyme Constraints”, Summers-Stay2020
  • “Blender: A State-Of-The-Art Open Source Chatbot”, Roller et al 2020
  • “Trading Off Diversity and Quality in Natural Language Generation”, Zhang et al 2020
  • “Rapformer: Conditional Rap Lyrics Generation With Denoising Autoencoders”, Nikolov et al 2020
  • “A Hundred Visions and Revisions”, Binder2020
  • “Top-K Training of GANs: Improving GAN Performance by Throwing Away Bad Samples”, Sinha et al 2020
  • “Towards a Human-Like Open-Domain Chatbot”, Adiwardana et al 2020
  • “Controlling Text Generation With Plug and Play Language Models”, Liu et al 2019
  • “Plug and Play Language Models: A Simple Approach to Controlled Text Generation”, Dathathri et al 2019
  • “CTRL: A Conditional Transformer Language Model For Controllable Generation”, Keskar et al 2019
  • “Neural Text Generation With Unlikelihood Training”, Welleck et al 2019
  • “GROVER: Defending Against Neural Fake News”, Zellers et al 2019
  • “The Curious Case of Neural Text Degeneration”, Holtzman et al 2019
  • “Good News, Everyone! Context Driven Entity-Aware Captioning for News Images”, Biten et al 2019
  • “GPT-2 Neural Network Poetry”, Gwern & Presser2019
  • “Insertion Transformer: Flexible Sequence Generation via Insertion Operations”, Stern et al 2019
  • “Blockwise Parallel Decoding for Deep Autoregressive Models”, Stern et al 2018
  • “Language GANs Falling Short”, Caccia et al 2018
  • “Discriminator Rejection Sampling”, Azadi et al 2018
  • “OCD: Optimal Completion Distillation for Sequence Learning”, Sabour et al 2018
  • “Controlling Linguistic Style Aspects in Neural Language Generation”, Ficler & Goldberg2017
  • “Six Challenges for Neural Machine Translation”, Koehn & Knowles2017
  • “Language Generation With Recurrent Generative Adversarial Networks without Pre-Training”, Press et al 2017
  • “A Deep Reinforced Model for Abstractive Summarization”, Paulus et al 2017
  • “Learning to Generate Reviews and Discovering Sentiment”, Radford et al 2017
  • “Improving Neural Machine Translation With Conditional Sequence Generative Adversarial Nets”, Yang et al 2017
  • “Tuning Recurrent Neural Networks With Reinforcement Learning”, Jaques et al 2017
  • “Google’s Multilingual Neural Machine Translation System: Enabling Zero-Shot Translation”, Johnson et al 2016
  • “WaveNet: A Generative Model for Raw Audio”, Oord et al 2016
  • “Sequence Level Training With Recurrent Neural Networks”, Ranzato et al 2015
  • “Generative Concatenative Nets Jointly Learn to Write and Classify Reviews”, Lipton et al 2015
  • “Semi-Supervised Sequence Learning”, Dai & Le2015
  • “Scheduled Sampling for Sequence Prediction With Recurrent Neural Networks”, Bengio et al 2015
  • “Controlling GPT-3 With Logit Bias”
  • “Feature: Beam Search for Improving Global Quality of New Text Samples”
  • “Exclude Top Choices (XTC): A Sampler That Boosts Creativity, Breaks Writing Clichés, and Inhibits Non-Verbatim Repetition”
  • “Prompting Diverse Ideas: Increasing AI Idea Variance”
  • “Pixels Still Beat Text: Attacking the OpenAI CLIP Model With Text Patches and Adversarial Pixel Perturbations”
  • “Me, Myself, and AI: the Situational Awareness Dataset (SAD) for LLMs”
  • “Apple or IPod? Easy Fix for Adversarial Textual Attacks on OpenAI’s CLIP Model!”
  • Sort By Magic
    • artistic-ai
    • text-optimization
    • controllable
  • Wikipedia
• Miscellaneous
• Bibliography

Backlinks: