Bibliography (11):

1. Replacing softmax with ReLU in Vision Transformers

2. Vision Transformer: An Image is Worth 16×16 Words: Transformers for Image Recognition at Scale

3. Pay Less Attention with Lightweight and Dynamic Convolutions

4. Attention Is All You Need

5. Layer Normalization

6. Wikipedia Bibliography:

   1. Transformer (deep learning architecture)

   2. Convolutional neural network

   3. https://en.wikipedia.org/wiki/Computer_vision  :

      https://en.wikipedia.org/wiki/Computer_vision

   4. Softmax function

   5. Rectifier (neural networks)

   6. https://en.wikipedia.org/wiki/Convolutional_neural_network#Types_of_layers  :

      https://en.wikipedia.org/wiki/Convolutional_neural_network#Types_of_layers