“2019 Recent Trends in GPU Price per FLOPS”, Asya Bergal2020-03-25 (AI hardware, experience curves; backlinks; similar)⁠:

	Release Prices	95th-percentile Active Prices	95th-percentile Active Prices (pre-crypto price rise)
	2007-11–2y2010-0114ya	2011-03–8y2020-01	2011-03–5y2016-12
$ / single-precision FLOPS	12.5	17	16
	9/2014–1/2020	1/2015–1/2020	1/2015–12/2016
$ / half-precision FLOPS	8	10	8
$ / half-precision FMA FLOPS	4	4.5	—

Release price data seems to generally support the trends we found in active prices, with the notable exception of trends in GPU price / single-precision FLOPS, which cannot be explained solely by the different start dates.⁵⁸ We think the best estimate of the overall trend for prices at which people recently bought GPUs is the 95^th-percentile active price data from 2011^–₉2020, since release price data does not account for existing GPUs becoming cheaper over time. The pre-crypto trends are similar to the overall trends, suggesting that the trends we are seeing are not anomalous due to cryptocurrency.

Given that, we guess that GPU prices as a whole have fallen at rates that would yield an order of magnitude over roughly:

• 17 years for single-precision FLOPS

• 10 years for half-precision FLOPS

• 5 years for half-precision fused multiply-add FLOPS

Half-precision FLOPS seem to have become cheaper substantially faster than single-precision in recent years. This may be a “catching up” effect as more of the space on GPUs was allocated to half-precision computing, rather than reflecting more fundamental technological progress.

[Keywords: AI Timelines, Featured Articles, Hardware and AI Timelines, Hardware progress]

Backlinks:

Similar Links: