Bibliography (22):

  1. AI and Compute

  2. T5: Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer

  3. Towards a Human-like Open-Domain Chatbot

  4. GShard: Scaling Giant Models with Conditional Computation and Automatic Sharding

  5. Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity

  6. GPT-3: Language Models are Few-Shot Learners

  7. The Evolved Transformer

  8. https://mlcommons.org/

  9. https://www.hpcwire.com/2019/03/19/aws-upgrades-its-gpu-backed-ai-inference-platform/

  10. https://aws.amazon.com/blogs/aws/amazon-ec2-update-inf1-instances-with-aws-inferentia-chips-for-high-performance-cost-effective-inferencing/

  11. https://arxiv.org/pdf/2104.10350.pdf#page=6

  12. Attention Is All You Need

  13. Energy and Policy Considerations for Deep Learning in NLP

  14. The Evolved Transformer

  15. https://arxiv.org/pdf/2104.10350#page=21&org=google

  16. https://arxiv.org/pdf/2104.10350.pdf#page=9

  17. https://arxiv.org/pdf/2104.10350.pdf#page=3

  18. https://www.gstatic.com/gumdrop/sustainability/google-2020-environmental-report.pdf

  19. https://arxiv.org/pdf/2104.10350.pdf#page=14

  20. GShard: Scaling Giant Models with Conditional Computation and Automatic Sharding

  21. Wikipedia Bibliography:

    1. https://en.wikipedia.org/wiki/Proxy_(statistics)  :

      https://en.wikipedia.org/wiki/Proxy_(statistics)

    2. BLEU