RNN Metadata for Mimicking Author Style
Poems
GPT-3 Creative Fiction
twdne#text
[Transclude the forward-link's
context]
A Very Unlikely Chess Game
Update: Upgrading to 1.5B GPT-2, and adding 22 new subreddit-bots
GPT-3: Language Models are Few-Shot Learners
Better Language Models and Their Implications
https://research.google/blog/transformer-a-novel-neural-network-architecture-for-language-understanding/
The Illustrated Transformer
The Illustrated GPT-2 (Visualizing Transformer Language Models)
The Transformer—Attention Is All You Need.
https://blog.floydhub.com/the-transformer-in-pytorch/
https://e2eml.school/transformers.html
Attention Is All You Need
The Annotated Transformer
Self-Attention with Relative Position Representations
Character-Level Language Modeling with Deeper Self-Attention
Transformer-XL: Attentive Language Models Beyond a Fixed-Length Context
Transformer-XL—Combining Transformers and RNNs Into a State-Of-The-Art Language Model
Understanding BERT Transformer: Attention Isn’t All You Need
Transformers are a very exciting family of machine learning architectures
https://amaarora.github.io/2020/02/18/annotatedGPT2.html
The Transformer Family: Attention and Self-Attention • Multi-Head Self-Attention • Transformer • Adaptive Computation Time (ACT) • Improved Attention Span: (Longer Attention Span (Transformer-XL) / Adaptive Attention Span / Localized Attention Span (Image Transformer)) • Less Time and Memory Cost: (Sparse Attention Matrix Factorization (Sparse Transformers) / Locality-Sensitive Hashing (Reformer)) • Make It Recurrent (Universal Transformer) • Stabilization for RL (GTrXL)
The Bottom-up Evolution of Representations in the Transformer: A Study with Machine Translation and Language Modeling Objectives
Karpathy/minGPT: A Minimal PyTorch Re-Implementation of the OpenAI GPT (Generative Pretrained Transformer) Training
RASP: Thinking Like Transformers
‘self-attention’ directory
‘MLP NN’ directory
GPT-1: Improving Language Understanding with Unsupervised Learning
Language Modeling State-of-the-art leaderboards
Language Models are Unsupervised Multitask Learners
Humans Who Are Not Concentrating Are Not General Intelligences
Gpt-2-Samples
LM Explorer (alpha)
GPT-2: 6-Month Follow-Up
GPT-2: 1.5B Release
OpenGPT-2: We Replicated GPT-2-1.5b Because You Can Too
https://colab.research.google.com/drive/1BXry0kcm869-RVHHiY6NZmY9uBzbkf1Q
GROVER: Defending Against Neural Fake News
XLNet: Generalized Autoregressive Pretraining for Language Understanding
MegatronLM: Training Billion+ Parameter Language Models Using GPU Model Parallelism
T5: Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
https://colab.research.google.com/drive/1-ROO7L09EupLFLQM-TWgDHa5-FIOdLLh
DialoGPT: Large-Scale Generative Pre-training for Conversational Response Generation
This Waifu Does Not Exist
Talking to Myself or How I Trained GPT-2-1.5b for Rubber Ducking Using My Facebook Chat Data: Using Only Google Colab
https://www.thisstorydoesnotexist.com/
https://web.archive.org/web/20200110223938/https://stackroboflow.com/
Howl
An Eternal Howl
https://www.reddit.com/r/slatestarcodex/comments/as8ke7/an_eternal_howl/
GPT-2 Howl
GPT-2 Writes a Shelley Poem
GPT-2 As Step Toward General Intelligence
First line of famous poems continued by GPT-2
gpt-2-poetry
Ask GPT-2
Ask GPT-2
FridAI: ‘Water, water, everywhere’, as read by Artificial Intelligence
The Poetry Machine
GPT-based Generation for Classical Chinese Poetry
Three More GPT-2 Poems
https://www.reddit.com/r/MachineLearning/comments/coc09l/p_these_lyrics_do_not_exist/
Testing The Limits of GROVER The Neural Fake News Detector. Can It Write Fiction? Can It Write Riddles?
https://www.reddit.com/r/SubSimulatorGPT2Meta/comments/ccvspt/update_experimenting_with_generating_hybrid/
CTRL: A Conditional Transformer Language Model For Controllable Generation
Conditional Transformer Language Model for Controllable Generation
https://papergains.co/pdfs/Transformer_Poetry-978-1-7341647-0-1.pdf#page=3
345M-GPT-2 After James Wright: Can AI Generate Convincing Contemporary Poetry?
GPT-2 AI Poetry Generation: Writing like Donne
Writing the Next American Hit: Using GPT-2 to Explore the Possibility of Creating Successful AI-Generated Song Lyrics Possibility of Creating Successful AI-Generated Song Lyric
How to Train It
Nshepperd/gpt-2: Code for the Paper "Language Models Are Unsupervised Multitask Learners"
ConnorJL/GPT2: An Implementation of Training for GPT-2, Supports TPUs
Replicating GPT-2-1.5B
Addendum: Evaluation of My Model
A Corpus of Poetry from Project Gutenberg
Dataset Search
Poems from Poetryfoundation.org
A Small Module Meant for Use in Text Generators That Lets You Filter Strings for Bad Words
Success
Unhandled Arguments Checked After Execution, Not Before
The Curious Case of Neural Text Degeneration
https://www.trentonbricken.com/Tail-Free-Sampling/
The Unreasonable Effectiveness of Recurrent Neural Networks
2019-03-06-gwern-gpt2-poetry-projectgutenberg-network-519407.tar.xz
2019-03-06-gpt2-poetry-1000samples.txt
https://x.com/theshawwn
Kaggle: Your Home for Data Science
rnn-metadata#inline-metadata-trick
[Transclude the forward-link's
context]
2019-10-18-Poetryfoundation-Formatted.txt
2019-10-17-117m-poetry-cleanprojectgutenberg-samples.txt
2019-10-19-117m-poetryfoundation-samples.txt
2019-10-19-gwern-gpt2-poetry-pgclean-117m.tar.xz
2019-03-06-gwern-gpt2-poetry-prefix-projectgutenberg-network-224474.tar.xz
2019-03-06-gpt2-poetry-prefix-1000samples.txt
To a Skylark by Percy Bysshe Shelley
Gwern’s AI-Generated Poetry
Overview for Starspawn0
Semantic projection: recovering human knowledge of multiple, distinct object features from word embeddings
Distributional Vectors Encode Referential Attributes
Dynamic Word Embeddings for Evolving Semantic Discovery
Verb Physics: Relative Physical Knowledge of Actions and Objects
Language Models Represent Space and Time
Language Encodes Geographical Information
Grounding the Ungrounded: Estimating Locations of Unknown Place Names from Linguistic Associations and Grounded Representations
Books by Pope, Alexander (Sorted by Popularity)
2019-03-16-gpt2-poetry-prefix-jabberwocky-100samples.txt
The Jingle Book by Carolyn Wells
https://openai.com/index/better-language-models/#update
UniLM: Unified Language Model Pre-training for Natural Language Understanding and Generation
Fitting Larger Networks into Memory: TLDR; We Release the Python/Tensorflow Package Openai/gradient-Checkpointing, That Lets You Fit 10× Larger Neural Nets into Memory at the Cost of an Additional 20% Computation Time
Generating Long Sequences with Sparse Transformers
Training Deep Nets with Sublinear Memory Cost
Memory-Efficient Backpropagation through Time
MuseNet: a deep neural network that can generate 4-minute musical compositions with 10 different instruments, and can combine styles from country to Mozart to the Beatles
Why Momentum Really Works
Cyclical Learning Rates for Training Neural Networks
SGDR: Stochastic Gradient Descent with Warm Restarts
Averaging Weights Leads to Wider Optima and Better Generalization
2019-05-13-gwern-gpt2-poetry-345m.tar.xz
2019-05-13-gpt2-poetry-345m-5000samples.txt
Reassuring
This Is a Python Script As Described in XKCD #1263: ‘Reassuring’. It Generates Thousands of Reassuring Parables about Things Humans Are Better Than Computers at Every Second.
2019-05-24-gpt2-poetry-yeatssecondcoming-500completions.txt
https://www.awanderingmind.blog/posts/2024-01-14-tao-te-ching-by-an-llm.html
https://x.com/HW
https://web.archive.org/web/20200209040154/https://decaut.org/situ/index.php/ttc-compilation/
2019-07-19-taotehching-ch1-1ksamples.txt
2019-07-21-gwern-gpt2-345m-taotehching-all.tar.xz
2019-07-21-taotehching-all-1ksamples.txt
2019-07-22-gpt2-345m-taotehching-all-ch181.tar.xz
Release Strategies and the Social Impacts of Language Models
Swarm Training: We Demonstrate a New Technique to Train ML Models Using Dozens of Independent TPUs.
2020-02-09-gpt21.5b-poetry-model-500522-1msamples.txt
2019-12-13-gwern-gpt-2-1.5b-poetry-model-500522.tar.xz
Shawwn/gpt-2: Code for the Paper "Language Models Are Unsupervised Multitask Learners"
Pricing
Danbooru2019 Is a Large-Scale Anime Image Database With 3.69m+ Images Annotated With 108m+ Tags; It Can Be Useful for Machine Learning Purposes such as Image Recognition and Generation.
[Transclude the forward-link's
context]
The Google SRE Handbook: Chapter 4—Service Level Objectives
HOGWILD!: A Lock-Free Approach to Parallelizing Stochastic Gradient Descent
TensorFlow Research Cloud (TRC): Accelerate your cutting-edge machine learning research with free Cloud TPUs
GPT-1: Improving Language Understanding by Generative Pre-Training § Model specifications
ftfy: fixes text for you
Adafactor: Adaptive Learning Rates with Sublinear Memory Cost
U B U W E B :: Racter
Language Models Are Unsupervised Multitask Learners § Experiments
2019-12-13-gpt21.5b-poetry-samples-topp090.txt
2019-12-15-gpt21.5b-poetry-samples-topp090.txt
2019-12-16-gpt21.5b-poetry-samples-topp080.txt
2019-12-18-gpt21.5b-poetry-samples-topp080.txt
Greg Brockman: OpenAI and AGI
Figure F.1: Four Uncurated Completions from a Context Suggesting the Model Compose a Poem in the Style of Wallace Stevens With the Title ‘Shadows on the Way’
https://github.com/karpathy/char-rnn/issues/138
https://news.ycombinator.com/item?id=21335120
true_poetry: Poetry generator by GPT-2 with meter and rhyme constraints
MuZero: Mastering Atari, Go, Chess and Shogi by Planning with a Learned Model
Neural Text Generation with Unlikelihood Training
Do Massively Pretrained Language Models Make Better Storytellers?
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension
Why Tool AIs Want to Be Agent AIs
Deep reinforcement learning from human preferences
CAN: Creative Adversarial Networks, Generating "Art" by Learning About Styles and Deviating from Style Norms
AlphaStar: Mastering the Real-Time Strategy Game StarCraft II
https://www.reddit.com/r/slatestarcodex/comments/b1b47h/gwerns_aigenerated_poetry/
https://news.ycombinator.com/item?id=19399467
https://news.ycombinator.com/item?id=21456403
Some Pretty Impressive Machine-Learning Generated Poetry Courtesy of GPT-2
Hark! from Those Shadowy Depths Thy Voice / Mournfully Echoes
On the Significance of Gwern’s Poem Generator
OpenAI’s New Language AI Is Available to Try Yourself
Generates Rhyming Poetry Using Huggingface GPT-2
A Hundred Visions and Revisions
RoBERTa: A Robustly Optimized BERT Pretraining Approach
Simonepri/lm-Scorer: 📃Language Model Based Sentences Scoring Library
How to Fine-Tune GPT-2 on Podcast Transcripts
These WWDC Boxed Lunches Aren't Real
https://web.archive.org/web/20220526054159/http://bkkaggle.github.io/blog/algpt2/2020/06/22/ALGPT2-part-1
https://web.archive.org/web/20210131134147/https://bkkaggle.github.io/blog/algpt2/2020/07/17/ALGPT2-part-2.html
The Average Fourth Grader Is a Better Poet Than…
The First Sally (A), Or, Trurl’s Electronic Bard
Seduced, Shaggy Samson Snored: The Fictional Machine That Generated Poems, and the Real People Who Had to Translate Them
Ramon Lull’s Thinking Machine
How to Build a State-Of-The-Art Conversational AI With Transfer Learning by Thomas Wolf
Computer Generated Foundation
https://www.reddit.com/r/SubSimulatorGPT2/comments/btfhks/what_is_rsubsimulatorgpt2/
A Chinese Room Writes a Sequel to Blindsight
How To Make Custom AI-Generated Text With GPT-2
Minimaxir/gpt-2-Keyword-Generation: Method to Encode Text for GPT-2 to Generate Text Based on Provided Keywords
Evaluation Metrics for Language Modeling
Lessons Learned from Building an AI Writing App
Excavate
Introducing Aspects of Creativity in Automatic Poetry Generation
Smart Vet: Autocompleting Sentences in Veterinary Medical Records
Deepfake Bot Submissions to Federal Public Comment Websites Cannot Be Distinguished from Human Submissions
This Word Does Not Exist [Github]
https://towardsdatascience.com/how-to-fine-tune-gpt-2-so-you-can-generate-long-form-creative-writing-7a5ae1314a61
This AI Poet Mastered Rhythm, Rhyme, and Natural Language to Write Like Shakespeare
Deep-speare: A Joint Neural Model of Poetic Language, Meter and Rhyme
Progressive Generation of Long Text
AdapterHub - 625 Adapters for 71 Text Tasks and 97 Languages
AdapterHub: A Framework for Adapting Transformers
Collaborative Storytelling with Large-scale Neural Language Models
Controllable Neural Text Generation
This Article Provides an Overview of Recent Methods to Fine-Tune Large Pre-Trained Language Models
Making Pre-trained Language Models Better Few-shot Learners
Prefix-Tuning: Optimizing Continuous Prompts for Generation
GPT Understands, Too
The Power of Scale for Parameter-Efficient Prompt Tuning
Entailment as Few-Shot Learner
Controllable Generation from Pre-trained Language Models via Inverse Prompting
https://gaotianyu.xyz/prompting/
Cutting Down on Prompts and Parameters: Simple Few-Shot Learning with Language Models
DART: Differentiable Prompt Makes Pre-trained Language Models Better Few-shot Learners
PPT: Pre-trained Prompt Tuning for Few-shot Learning
Towards a Unified View of Parameter-Efficient Transfer Learning
2019-12-18-skylion-archiveofourown-fanfics-textscrape.tar.xz
https://archive.org/details/@entropy11235813
2020-01-14-gpt2-1558m-archiveofourownao3.tar.xz
AI Dungeon 2
2020-02-03-gpt21.5b-archiveofourownao3-model-510427-samples-topp090.txt
https://x.com/astraliteheart
Expanding the Frontiers of AI Creativity
😇A PyTorch Implementation of the DeepMoji Model: State-Of-The-Art Deep Learning Model for Analyzing Sentiment, Emotion, Sarcasm Etc
This Pony Does Not Exist
Yzhou359/MakeItTalk
2021-05-05-astraliteheart-purplesmartai-mylittleponygpt215b-twilightsparkledialogue.png
2021-05-05-astraliteheart-purplesmartai-mylittleponygpt215b-twilightsparkledialogue-torchmojiemotionalvoicecontrol.jpg
2021-05-05-astraliteheart-purplesmartai-mylittleponygpt215b-twilightsparkledialogue-voicedialogue.png
2021-05-05-astraliteheart-purplesmartai-mylittleponygpt215b-gpuloadgraph.png
End to End Agent Conversation Demo
My Little Pony: Friendship Is Magic Fanfiction
Library Genesis
Best Science Fiction (3506 Books)
The Best Fantasy Books
2020-08-20-astraliteheart-gpt215b-sffuberset.tar.xz
2021-03-14-astraliteheart-tts-mlp.tar.xz
https://x.com/me_irl/status/1217818112957014022
2020-02-03-gpt21.5b-videogamewalkthrough-model-174925-samples-topp090.txt
OpenAI Text Generator GPT-2 Creates Video Game Walkthrough for ‘Most Tedious Game in History’
2020-01-16-gpt-2-1558m-shawnpresser-videogamewalkthrough.tar.xz
https://x.com/theshawwn/status/1212156603140648961
2025-03-08-2019-12-18-shawnpresser-gpt-2-117m-rdota2.tar.xz
