Bibliography (13):

  1. https://academictorrents.com/details/0d366035664fdf51cfbe9f733953ba325776e667/tech

  2. EleutherAI

  3. https://pile.eleuther.ai/

  4. https://openwebtext2.readthedocs.io/en/latest/

  5. Compressive Transformers for Long-Range Sequence Modeling

  6. Language Models are Unsupervised Multitask Learners

  7. GPT-3: Language Models are Few-Shot Learners

  8. The Pile: An 800GB Dataset of Diverse Text for Language Modeling

  9. Wikipedia Bibliography:

    1. Common Crawl

    2. PubMed

    3. GitHub

    4. Europarl Corpus

    5. Hacker news