“V1T: Large-Scale Mouse V1 Response Prediction Using a Vision Transformer”, Bryan M. Li, Isabel M. Cornacchia, Nathalie L. Rochefort, Arno Onken2023-02-06 (Transformer, neuroscience)⁠:

Accurate predictive models of the visual cortex neural response to natural visual stimuli remain a challenge in computational neuroscience. In this work, we introduce V1T, a novel Vision Transformer based architecture that learns a shared visual and behavioral representation across animals.

We evaluate our model on two large datasets recorded from mouse primary visual cortex and:

outperform previous convolution-based models by more than 12.7% in prediction performance. Moreover, we show that the attention weights learned by the Transformer correlate with the population receptive fields.

Our model thus sets a new benchmark for neural response prediction and captures characteristic features of the visual cortex.