Bibliography (10):

1. https://github.com/ironjr/grokfast

2. https://arxiv.org/pdf/2405.20233#page=3

3. https://arxiv.org/pdf/2405.20233#page=12

4. Decoupled Weight Decay Regularization

5. Omnigrok: Grokking Beyond Algorithmic Data

6. https://arxiv.org/pdf/2405.20233#page=2

7. Wikipedia Bibliography:

   1. Gradient descent

   2. Stochastic gradient descent

   3. https://en.wikipedia.org/wiki/Stochastic_gradient_descent#Adam  :

      https://en.wikipedia.org/wiki/Stochastic_gradient_descent#Adam

   4. Ridge regression § Tikhonov regularization  :

      https://en.wikipedia.org/wiki/Ridge_regression#Tikhonov_regularization