“AI Chips: What They Are and Why They Matter—An AI Chips Reference”, Saif M. Khan, Alexander Mann2020-04 (AI hardware; backlinks)⁠:

Artificial intelligence will play an important role in national and international security in the years to come. As a result, the US government is considering how to control the diffusion of AI-related information and technologies. Because general-purpose AI software, datasets, and algorithms are not effective targets for controls, the attention naturally falls on the computer hardware necessary to implement modern AI systems. The success of modern AI techniques relies on computation on a scale unimaginable even a few years ago. Training a leading AI algorithm can require a month of computing time and cost $100 million. This enormous computational power is delivered by computer chips that not only pack the maximum number of transistors—basic computational devices that can be switched between on (1) and off (0) states—but also are tailor-made to efficiently perform specific calculations required by AI systems. Such leading-edge, specialized “AI chips” are essential for cost-effectively implementing AI at scale; trying to deliver the same AI application using older AI chips or general-purpose chips can cost tens to thousands of times more. The fact that the complex supply chains needed to produce leading-edge AI chips are concentrated in the United States and a small number of allied democracies provides an opportunity for export control policies.

This report presents the above story in detail. It explains how AI chips work, why they have proliferated, and why they matter. It also shows why leading-edge chips are more cost-effective than older generations, and why chips specialized for AI are more cost-effective than general-purpose chips. As part of this story, the report surveys semiconductor industry and AI chip design

trends shaping the evolution of chips in general and AI chips in particular. It also presents a consolidated discussion of technical and economic trends that result in the critical cost-effectiveness tradeoffs for AI applications. In this paper, AI refers to cutting-edge computationally-intensive AI systems, such as deep neural networks. DNNs are responsible for most recent AI breakthroughs, like DeepMind’s AlphaGo, which beat the world champion Go player. As suggested above, we use “AI chips” to refer to certain types of computer chips that attain high efficiency and speed for AI-specific calculations at the expense of low efficiency and speed for other calculations.

This paper focuses on AI chips and why they are essential for the development and deployment of AI at scale. It does not focus on details of the supply chain for such AI chips or the best targets within the supply chain for export controls (CSET has published preliminary results on this topic). Forthcoming CSET reports will analyze the semiconductor supply chain, national competitiveness, the prospects of China’s semiconductor industry for supply chain localization, and policies the United States and its allies can pursue to maintain their advantages in the production of AI chips, recommending how this advantage can be used to ensure beneficial development and adoption of AI technologies.

Key points:

1. Industry Trends Favor AI Chips over General-Purpose Chips

2. AI Chip Basics

3. Why Cutting-Edge AI Chips are Necessary for AI

4. Implications for National AI Competitiveness

Table of contents:

• Introduction and Summary

• The Laws of Chip Innovation

  • Transistor Shrinkage: Moore’s Law

  • Efficiency and Speed Improvements

  • Increasing Transistor Density Unlocks Improved Designs for Efficiency and Speed

  • Transistor Design is Reaching Fundamental Size Limits

• The Slowing of Moore’s Law and the Decline of General-Purpose Chips

  • The Economies of Scale of General-Purpose Chips

  • Costs are Increasing Faster than the Semiconductor Market

  • The Semiconductor Industry’s Growth Rate is Unlikely to Increase

• Chip Improvements as Moore’s Law Slows

  • Transistor Improvements Continue, but are Slowing

  • Improved Transistor Density Enables Specialization

• The AI Chip Zoo

  • AI Chip Types

  • AI Chip Benchmarks

• The Value of State-of-the-Art AI Chips

  • The Efficiency of State-of-the-Art AI Chips Translates into Cost-Effectiveness

  • Compute-Intensive AI Algorithms are Bottlenecked by Chip Costs and Speed

• U.S. and Chinese AI Chips and Implications for National Competitiveness

• Appendix A: Basics of Semiconductors and Chips

• Appendix B: How AI Chips Work

  • Parallel Computing

  • Low-Precision Computing

  • Memory Optimization

  • Domain-Specific Languages

• Appendix C: AI Chip Benchmarking Studies

• Appendix D: Chip Economics Model

  • Chip Transistor Density, Design Costs, and Energy Costs

  • Foundry, Assembly, Test and Packaging Costs

• Acknowledgments

See Also:

• ChinAI #137: Year 3 of ChinAI: Reflections on the newsworthiness of machine translation

• U.S. vs. China Rivalry Boosts Tech—and Tensions: Militarized AI threatens a new arms race

• Slowing Moore’s Law: How It Could Happen

• AI and Efficiency: We’re releasing an analysis showing that since 2012 the amount of compute needed to train a neural net to the same performance on ImageNet classification has been decreasing by a factor of 2 every 16 months

• The Computational Limits of Deep Learning

• Research community dynamics behind popular AI benchmarks

• Ten Lessons From Three Generations Shaped Google’s TPUv4i

• The Role Of RAW POWER In INTELLIGENCE

• The Node Is Nonsense: There are better ways to measure progress than the old Moore’s law metric

• Sustainable AI: Environmental Implications, Challenges and Opportunities

Backlinks: