Bibliography (67):

1. https://x.com/DimitrisPapail/status/1678407512637284352

2. https://github.com/lee-ny/teaching_arithmetic

3. https://github.com/karpathy/nanoGPT

4. Language Models are Unsupervised Multitask Learners

5. Chain-of-Thought Prompting Elicits Reasoning in Large Language Models

6. https://arxiv.org/pdf/2307.03381.pdf#page=3

7. https://arxiv.org/pdf/2307.03381.pdf#page=4

8. https://arxiv.org/pdf/2307.03381.pdf#page=5

9. https://arxiv.org/pdf/2307.03381.pdf#page=7

10. https://arxiv.org/pdf/2307.03381.pdf#page=8

11. https://x.com/BlinkDL_AI/status/1677593798531223552

12. https://arxiv.org/pdf/2307.03381.pdf#page=9

13. https://arxiv.org/pdf/2307.03381.pdf#page=11

14. https://arxiv.org/pdf/2307.03381.pdf#page=13

15. https://arxiv.org/pdf/2307.03381.pdf#page=15

16. https://arxiv.org/pdf/2307.03381.pdf#page=16

17. https://arxiv.org/pdf/2307.03381.pdf#page=17

18. https://arxiv.org/pdf/2307.03381.pdf#page=18

19. https://arxiv.org/pdf/2307.03381.pdf#page=19

20. https://arxiv.org/pdf/2307.03381.pdf#page=20

21. https://arxiv.org/pdf/2307.03381.pdf#page=21

22. https://arxiv.org/pdf/2307.03381.pdf#page=23

23. https://arxiv.org/pdf/2307.03381.pdf#page=26

24. https://arxiv.org/pdf/2307.03381.pdf#page=27

25. https://arxiv.org/pdf/2307.03381.pdf#page=30

26. https://arxiv.org/pdf/2307.03381.pdf#page=31

27. FLAN: Scaling Instruction-Finetuned Language Models

28. Attention Is All You Need

29. Program Induction by Rationale Generation: Learning to Solve and Explain Algebraic Word Problems

30. Training Verifiers to Solve Math Word Problems

31. Show Your Work: Scratchpads for Intermediate Computation with Language Models

32. MultiArith: Solving General Arithmetic Word Problems

33. Neural Programmer-Interpreters

34. Towards Synthesizing Complex Programs from Input-Output Examples

35. Efficient Architecture Search by Network Transformation

36. Investigating the Limitations of the Transformers with Simple Arithmetic Tasks

37. How well do Large Language Models perform in Arithmetic tasks?

38. Impact of Pretraining Term Frequencies on Few-Shot Reasoning

39. GPT-3: Language Models are Few-Shot Learners

40. Least-to-Most Prompting Enables Complex Reasoning in Large Language Models

41. Learning to Execute

42. https://aclanthology.org/2021.emnlp-main.563.pdf

43. There Once Was a Really Bad Poet, It Was Automated but You Didn’t Know It

44. Limitations of Language Models in Arithmetic and Symbolic Induction

45. Let’s Verify Step by Step

46. Solving math word problems with process & outcome-based feedback

47. Do NLP Models Know Numbers? Probing Numeracy in Embeddings

48. What is my math transformer doing? – 3 results on interpretability and generalization

49. Linear algebra with transformers

50. Solving Linear Algebra by Program Synthesis

51. How does GPT-2 compute greater-than?: Interpreting mathematical abilities in a pre-trained language model

52. The Shape of Learning Curves: a Review

53. The Algebraic Combinatorial Approach for Low-Rank Matrix Completion

54. Exploring Length Generalization in Large Language Models

55. Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?

56. 2023-lee-figure1-numberformattingforgpt2arithmetic.jpg

57. https://arxiv.org/pdf/2307.03381.pdf#page=14

58. ​ GPT-3 Creative Fiction § BPEs

59. https://github.com/openai/tiktoken

60. Evaluating Transformer Language Models on Arithmetic Operations Using Number Decomposition

61. It’s Not Just Size That Matters: Small Language Models Are Also Few-Shot Learners

62. TinyStories: How Small Can Language Models Be and Still Speak Coherent English?

63. Models In a Spelling Bee: Language Models Implicitly Learn the Character Composition of Tokens

64. Wikipedia Bibliography:

    1. https://en.wikipedia.org/wiki/GPT-3  :

       https://en.wikipedia.org/wiki/GPT-3

    2. https://en.wikipedia.org/wiki/Matrix_completion  :

       https://en.wikipedia.org/wiki/Matrix_completion

    3. Recurrent neural network

    4. Word embedding