Bibliography (31):

1. Contrastive Representation Learning: A Framework and Review

2. CLIP: Connecting Text and Images: We’re introducing a neural network called CLIP which efficiently learns visual concepts from natural language supervision. CLIP can be applied to any visual classification benchmark by simply providing the names of the visual categories to be recognized, similar to the ‘zero-shot’ capabilities of GPT-2 and GPT-3

3. LAION-400M: Open Dataset of CLIP-Filtered 400 Million Image-Text Pairs

4. An Open Source Implementation of CLIP

5. ​ ‘end-to-end’ directory

6. https://github.com/LAION-AI/scaling-laws-openclip

7. ImageNet: A Large-Scale Hierarchical Image Database

8. ImageNet Large Scale Visual Recognition Challenge

9. Do ImageNet Classifiers Generalize to ImageNet?

10. The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Generalization

11. ImageNet-Sketch: Learning Robust Global Representations by Penalizing Local Predictive Power

12. ObjectNet: A large-scale bias-controlled dataset for pushing the limits of object recognition models

13. ImageNet-A: Natural Adversarial Examples

14. WIT: Wikipedia-based Image Text Dataset for Multimodal Multilingual Machine Learning

15. Microsoft COCO: Common Objects in Context

16. https://paperswithcode.com/dataset/flickr30k

17. https://arxiv.org/pdf/2212.07143.pdf#page=27

18. https://pytorch.org/docs/stable/notes/ddp.html

19. https://apps.fz-juelich.de/jsc/hps/juwels/configuration.html#hardware-configuration-of-the-system-name-booster-module

20. https://hpc.stability.ai/

21. Scaling Laws for Neural Language Models

22. Scaling Vision Transformers

23. Vision Transformer: An Image is Worth 16×16 Words: Transformers for Image Recognition at Scale

24. https://arxiv.org/pdf/2212.07143.pdf#page=22

25. Wikipedia Bibliography:

    1. Neural scaling law  :

       https://en.wikipedia.org/wiki/Neural_scaling_law

    2. Power law

    3. OpenAI

    4. Pareto front

    5. Ampere (microarchitecture)

    6. https://en.wikipedia.org/wiki/JUWELS  :

       https://en.wikipedia.org/wiki/JUWELS

    7. https://en.wikipedia.org/wiki/Forschungszentrum_J%C3%BClich#Supercomputers  :

       https://en.wikipedia.org/wiki/Forschungszentrum_J%C3%BClich#Supercomputers