“TorToise: Better Speech Synthesis through Scaling”, James Betker2023-05-12 (CLIP, DALL·E 2, AI scaling)⁠:

In recent years, the field of image generation has been revolutionized by the application of autoregressive transformers and DDPMs. These approaches model the process of image generation as a step-wise probabilistic processes and leverage large amounts of compute and data to learn the image distribution. This methodology of improving performance need not be confined to images.

This paper describes a way to apply advances in the image generative domain to speech synthesis. The result is TorToise—an expressive, multi-voice text-to-speech system.

All model code and trained weights have been open-sourced at Github.