“Bridging the Gaps Between Residual Learning, Recurrent Neural Networks and Visual Cortex”, Qianli Liao, Tomaso Poggio2016-04-13 (RNN, neuroscience; similar)⁠:

We discuss relations between Residual Networks (ResNet), Recurrent Neural Networks (RNNs) and the primate visual cortex. We begin with the observation that a special type of shallow RNN is exactly equivalent to a very deep ResNet with weight sharing among the layers.

A direct implementation of such a RNN, although having orders of magnitude fewer parameters, leads to a performance similar to the corresponding ResNet. We propose (1) a generalization of both RNN and ResNet architectures and (2) the conjecture that a class of moderately deep RNNs is a biologically-plausible model of the ventral stream in visual cortex.

We demonstrate the effectiveness of the architectures by testing them on the CIFAR-10 and ImageNet dataset.

Similar Links: