Bibliography (22):

1. https://github.com/ThomasScialom/T0_continual_learning

2. https://huggingface.co/ThomasNLG/CT0-11B

3. Memorization Without Overfitting: Analyzing the Training Dynamics of Large Language Models

4. Continual Learning with Foundation Models: An Empirical Study of Latent Replay

5. Don’t Stop Learning: Towards Continual Learning for the CLIP Model

6. Lifelong Pretraining: Continually Adapting Language Models to Emerging Corpora

7. DualPrompt: Complementary Prompting for Rehearsal-free Continual Learning

8. ​ ‘instruct-tuning LLMs’ directory

9. T0: Multitask Prompted Training Enables Zero-Shot Task Generalization

10. A Call to Build Models Like We Build Open-Source Software

11. https://arxiv.org/pdf/2205.12393.pdf#page=5

12. https://arxiv.org/pdf/2205.12393.pdf#page=7

13. https://arxiv.org/pdf/2205.12393.pdf#page=4

14. https://arxiv.org/pdf/2205.12393.pdf#page=8

15. Effect of scale on catastrophic forgetting in neural networks

16. T5: Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer

17. Continual Pre-Training Mitigates Forgetting in Language and Vision

18. An Empirical Investigation of the Role of Pre-training in Lifelong Learning

19. https://arxiv.org/pdf/2205.12393.pdf#page=15

20. Attention Is All You Need

21. Wikipedia Bibliography:

    1. https://en.wikipedia.org/wiki/Catastrophic_interference  :

       https://en.wikipedia.org/wiki/Catastrophic_interference

    2. Neural scaling law  :

       https://en.wikipedia.org/wiki/Neural_scaling_law