“Stochastic Backpropagation and Approximate Inference in Deep Generative Models”, Danilo Jimenez Rezende, Shakir Mohamed, Daan Wierstra2014-01-16 (autoencoder NN; backlinks)⁠:

We marry ideas from deep neural networks and approximate Bayesian inference to derive a generalized class of deep, directed generative models, endowed with a new algorithm for scalable inference and learning.

Our algorithm introduces a recognition model to represent approximate posterior distributions, and that acts as a stochastic encoder of the data. We develop stochastic back-propagation—rules for back-propagation through stochastic variables—and use this to develop an algorithm that allows for joint optimization of the parameters of both the generative and recognition model.

We demonstrate on several real-world data sets that the model generates realistic samples, provides accurate imputations of missing data and is a useful tool for high-dimensional data visualization.

Backlinks: