Bibliography (20):

1. https://www.youtube.com/watch?v=_PlcQUvNeCg

2. The Deep Learning Revolution and Its Implications for Computer Architecture and Chip Design

3. Understanding sources of inefficiency in general-purpose chips

4. Ten Lessons From Three Generations Shaped Google’s TPUv4i

5. ZeRO-Offload: Democratizing Billion-Scale Model Training

6. Startup Tenstorrent shows AI is changing computing and vice versa: Tenstorrent is one of the rush of AI chip makers founded in 2016 and finally showing product. The new wave of chips represent a substantial departure from how traditional computer chips work, but also point to ways that neural network design may change in the years to come

7. Google details new AI accelerator chips

8. Hardware Beyond Backpropagation: a Photonic Co-Processor for Direct Feedback Alignment

9. 2019 recent trends in GPU price per FLOPS

10. The Design Process for Google’s Training Chips: TPUv2 and TPUv3

11. When will computer hardware match the human brain?

12. Wikipedia Bibliography:

    1. https://en.wikipedia.org/wiki/Design_Automation_Conference  :

       https://en.wikipedia.org/wiki/Design_Automation_Conference

    2. Bill Dally  :

       https://en.wikipedia.org/wiki/Bill_Dally

    3. Nvidia

    4. Volta (microarchitecture)

    5. https://en.wikipedia.org/wiki/Dot_product  :

       https://en.wikipedia.org/wiki/Dot_product

    6. https://en.wikipedia.org/wiki/Matrix_multiplication  :

       https://en.wikipedia.org/wiki/Matrix_multiplication

    7. https://en.wikipedia.org/wiki/Multiply%E2%80%93accumulate_operation  :

       https://en.wikipedia.org/wiki/Multiply%E2%80%93accumulate_operation

    8. Hidden Markov model

    9. Graphics processing unit