“Int-4 LLaMa Is Not Enough—Int-3 and Beyond: More Compression, Easier to Build Apps on LLMs That Run Locally”, Nolano.org 2023

“Int-4 LLaMa is not enough—Int-3 and beyond: More compression, easier to build apps on LLMs that run locally”⁠, nolano.org2023-03-13 (GPT; similar; bibliography)

“BMT: Binarized Neural Machine Translation”, Zhang Et Al 2023

“BMT: Binarized Neural Machine Translation”⁠, Yichi Zhang, Ankush Garg, Yuan Cao, Łukasz Lew, Behrooz Ghorbani, Zhiru Zhang, Orhan Firat2023-02-09 (knowledge distillation⁠, Transformer NN; similar; bibliography)

“SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Models”, Xiao Et Al 2022

“SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Models”⁠, Guangxuan Xiao, Ji Lin, Mickael Seznec, Julien Demouth, Song Han2022-11-18 (GPT; similar; bibliography)

“Who Says Elephants Can’t Run: Bringing Large Scale MoE Models into Cloud Scale Production”, Kim Et Al 2022

“Who Says Elephants Can’t Run: Bringing Large Scale MoE Models into Cloud Scale Production”⁠, Young Jin Kim, Rawn Henry, Raffy Fahim, Hany Hassan Awadalla2022-11-18 (MoE NN; similar)

“Efficiently Scaling Transformer Inference”, Pope Et Al 2022

“Efficiently Scaling Transformer Inference”⁠, Reiner Pope, Sholto Douglas, Aakanksha Chowdhery, Jacob Devlin, James Bradbury, Anselm Levskaya, Jonathan Heek2022-11-09 (self-attention⁠, PaLM⁠, AI scaling/hardware; similar; bibliography)

“GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers”, Frantar Et Al 2022

“GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers”⁠, Elias Frantar, Saleh Ashkboos, Torsten Hoefler, Dan Alistarh2022-10-31 (GPT; backlinks; similar; bibliography)

“Fast DistilBERT on CPUs”, Shen Et Al 2022

“Fast DistilBERT on CPUs”⁠, Haihao Shen, Ofir Zafrir, Bo Dong, Hengyu Meng, Xinyu Ye, Zhe Wang, Yi Ding, Hanwen Chang, Guy Boudoukh2022-10-27 (knowledge distillation⁠, Transformer NN; similar)

“GLM-130B: An Open Bilingual Pre-trained Model”, Zeng Et Al 2022

“GLM-130B: An Open Bilingual Pre-trained Model”⁠, Aohan Zeng, Xiao Liu, Zhengxiao Du, Zihan Wang, Hanyu Lai, Ming Ding, Zhuoyi Yang, Yifan Xu, Wendi Zheng2022-10-05 (T5 Transformer⁠, AI scaling; similar; bibliography)

“FP8 Formats for Deep Learning”, Micikevicius Et Al 2022

“FP8 Formats for Deep Learning”⁠, Paulius Micikevicius, Dusan Stosic, Neil Burgess, Marius Cornea, Pradeep Dubey, Richard Grisenthwaite2022-09-12 (GPT; similar)

“Is Integer Arithmetic Enough for Deep Learning Training?”, Ghaffari Et Al 2022

“Is Integer Arithmetic Enough for Deep Learning Training?”⁠, Alireza Ghaffari, Marzieh S. Tahaei, Mohammadreza Tayaranian, Masoud Asgharian, Vahid Partovi Nia2022-07-18 (AI scaling/hardware; similar)

“On-Device Training Under 256KB Memory”, Lin Et Al 2022

“On-Device Training Under 256KB Memory”⁠, Ji Lin, Ligeng Zhu, Wei-Ming Chen, Wei-Chen Wang, Chuang Gan, Song Han2022-06-30 (AI scaling; similar; bibliography)

“How to Train Accurate BNNs for Embedded Systems?”, Putter & Corporaal2022

“How to train accurate BNNs for embedded systems?”⁠, Floran de Putter, Henk Corporaal2022-06-24 (similar)

“Director: Deep Hierarchical Planning from Pixels”, Hafner Et Al 2022

“Director: Deep Hierarchical Planning from Pixels”⁠, Danijar Hafner, Kuang-Huei Lee, Ian Fischer, Pieter Abbeel2022-06-08 (RL exploration⁠, model-based RL; similar; bibliography)

“8-bit Numerical Formats for Deep Neural Networks”, Noune Et Al 2022

“8-bit Numerical Formats for Deep Neural Networks”⁠, Badreddine Noune, Philip Jones, Daniel Justus, Dominic Masters, Carlo Luschi2022-06-06 (AI scaling/hardware; similar)

“ZeroQuant: Efficient and Affordable Post-Training Quantization for Large-Scale Transformers”, Yao Et Al 2022

“ZeroQuant: Efficient and Affordable Post-Training Quantization for Large-Scale Transformers”⁠, Zhewei Yao, Reza Yazdani Aminabadi, Minjia Zhang, Xiaoxia Wu, Conglong Li, Yuxiong He2022-06-04 (knowledge distillation⁠, GPT⁠, AI scaling/hardware; similar; bibliography)

“XTC: Extreme Compression for Pre-trained Transformers Made Simple and Efficient”, Wu Et Al 2022

“XTC: Extreme Compression for Pre-trained Transformers Made Simple and Efficient”⁠, Xiaoxia Wu, Zhewei Yao, Minjia Zhang, Conglong Li, Yuxiong He2022-06-04 (Transformer NN; similar; bibliography)

“Matryoshka Representations for Adaptive Deployment”, Kusupati Et Al 2022

“Matryoshka Representations for Adaptive Deployment”⁠, Aditya Kusupati, Gantavya Bhatt, Aniket Rege, Matthew Wallingford, Aditya Sinha, Vivek Ramanujan, William Howard-Snyder2022-05-26 (similar; bibliography)

“Maximizing Communication Efficiency for Large-scale Training via 0/​1 Adam”, Lu Et Al 2022

“Maximizing Communication Efficiency for Large-scale Training via 0/1 Adam”⁠, Yucheng Lu, Conglong Li, Minjia Zhang, Christopher De Sa, Yuxiong He2022-02-12 (AI scaling/hardware; similar; bibliography)

“Is Programmable Overhead Worth The Cost? How Much Do We Pay for a System to Be Programmable? It Depends upon Who You Ask”, Bailey2022

“Is Programmable Overhead Worth The Cost? How much do we pay for a system to be programmable? It depends upon who you ask”⁠, Brian Bailey2022-01-13 (AI scaling/hardware⁠, CS/hardware; backlinks; similar; bibliography)

“Boosted Dense Retriever”, Lewis Et Al 2021

“Boosted Dense Retriever”⁠, Patrick Lewis, Barlas Oğuz, Wenhan Xiong, Fabio Petroni, Wen-tau Yih, Sebastian Riedel2021-12-14 (retrieval AI; similar)

“FQ-ViT: Fully Quantized Vision Transformer without Retraining”, Lin Et Al 2021

“FQ-ViT: Fully Quantized Vision Transformer without Retraining”⁠, Yang Lin, Tianyu Zhang, Peiqin Sun, Zheng Li, Shuchang Zhou2021-11-27 (Transformer NN; similar; bibliography)

“𝜇NCA: Texture Generation With Ultra-Compact Neural Cellular Automata”, Mordvintsev & Niklasson2021

“𝜇NCA: Texture Generation with Ultra-Compact Neural Cellular Automata”⁠, Alexander Mordvintsev, Eyvind Niklasson2021-11-26 (cellular automata⁠, design; backlinks; similar)

“Prune Once for All: Sparse Pre-Trained Language Models”, Zafrir Et Al 2021

“Prune Once for All: Sparse Pre-Trained Language Models”⁠, Ofir Zafrir, Ariel Larey, Guy Boudoukh, Haihao Shen, Moshe Wasserblat2021-11-10 (knowledge distillation⁠, NN pruning; similar; bibliography)

“8-bit Optimizers via Block-wise Quantization”, Dettmers Et Al 2021

“8-bit Optimizers via Block-wise Quantization”⁠, Tim Dettmers, Mike Lewis, Sam Shleifer, Luke Zettlemoyer2021-10-06 (similar; bibliography)

“Understanding and Overcoming the Challenges of Efficient Transformer Quantization”, Bondarenko Et Al 2021

“Understanding and Overcoming the Challenges of Efficient Transformer Quantization”⁠, Yelysei Bondarenko, Markus Nagel, Tijmen Blankevoort2021-09-27 (Transformer NN; similar; bibliography)

“A Winning Hand: Compressing Deep Networks Can Improve Out-Of-Distribution Robustness”, Diffenderfer Et Al 2021

“A Winning Hand: Compressing Deep Networks Can Improve Out-Of-Distribution Robustness”⁠, James Diffenderfer, Brian R. Bartoldson, Shreya Chaganti, Jize Zhang, Bhavya Kailkhura2021-06-16 (NN pruning; similar)

“Efficient Deep Learning: A Survey on Making Deep Learning Models Smaller, Faster, and Better”, Menghani2021

“Efficient Deep Learning: A Survey on Making Deep Learning Models Smaller, Faster, and Better”⁠, Gaurav Menghani2021-06-16 (knowledge distillation; similar)

“Ten Lessons From Three Generations Shaped Google’s TPUv4i”, Jouppi Et Al 2021

“Ten Lessons From Three Generations Shaped Google’s TPUv4i”⁠, Norman P. Jouppi, Doe Hyun Yoon, Matthew Ashcraft, Mark Gottscho, Thomas B. Jablin, George Kurian, James Laudon2021-06-14 (RNN⁠, AI scaling/hardware; similar; bibliography)

“High-performance, Distributed Training of Large-scale Deep Learning Recommendation Models (DLRMs)”, Mudigere Et Al 2021

“High-performance, Distributed Training of Large-scale Deep Learning Recommendation Models (DLRMs)”⁠, Dheevatsa Mudigere, Yuchen Hao, Jianyu Huang, Andrew Tulloch, Srinivas Sridharan, Xing Liu, Mustafa Ozdal2021-04-12 (AI scaling/hardware; similar)

“1-bit Adam: Communication Efficient Large-Scale Training With Adam’s Convergence Speed”, Tang Et Al 2021

“1-bit Adam: Communication Efficient Large-Scale Training with Adam’s Convergence Speed”⁠, Hanlin Tang, Shaoduo Gan, Ammar Ahmad Awan, Samyam Rajbhandari, Conglong Li, Xiangru Lian, Ji Liu, Ce Zhang2021-02-04 (AI scaling; similar; bibliography)

“ES-ENAS: Blackbox Optimization over Hybrid Spaces via Combinatorial and Continuous Evolution”, Song Et Al 2021

“ES-ENAS: Blackbox Optimization over Hybrid Spaces via Combinatorial and Continuous Evolution”⁠, Xingyou Song, Krzysztof Choromanski, Jack Parker-Holder, Yunhao Tang, Daiyi Peng, Deepali Jain, Wenbo Gao2021-01-19 (NN pruning⁠, meta-learning; similar)

“Switch Transformers: Scaling to Trillion Parameter Models With Simple and Efficient Sparsity”, Fedus Et Al 2021

“Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity”⁠, William Fedus, Barret Zoph, Noam Shazeer2021-01-11 (T5 Transformer⁠, MoE NN; similar; bibliography)

“A Primer in BERTology: What We Know about How BERT Works”, Rogers Et Al 2020

“A Primer in BERTology: What we know about how BERT works”⁠, Anna Rogers, Olga Kovaleva, Anna Rumshisky2020-11-09 (knowledge distillation⁠, NN pruning⁠, Transformer NN; similar)

“L2L: Training Large Neural Networks With Constant Memory Using a New Execution Algorithm”, Pudipeddi Et Al 2020

“L2L: Training Large Neural Networks with Constant Memory using a New Execution Algorithm”⁠, Bharadwaj Pudipeddi, Maral Mesmakhosroshahi, Jinwen Xi, Sujeeth Bharadwaj2020-10-16 (GPT⁠, AI scaling/hardware; similar)

“RegDeepDanbooru: Yet Another Deep Danbooru Project”, Zyddnys 2020

“RegDeepDanbooru: Yet another Deep Danbooru project”⁠, zyddnys2020-10-11 (Danbooru AI; backlinks)

“TernaryBERT: Distillation-aware Ultra-low Bit BERT”, Zhang Et Al 2020

“TernaryBERT: Distillation-aware Ultra-low Bit BERT”⁠, Wei Zhang, Lu Hou, Yichun Yin, Lifeng Shang, Xiao Chen, Xin Jiang, Qun Liu2020-09-27 (knowledge distillation⁠, Transformer NN; similar)

“HOBFLOPS CNNs: Hardware Optimized Bitslice-Parallel Floating-Point Operations for Convolutional Neural Networks”, Garland & Gregg2020

“HOBFLOPS CNNs: Hardware Optimized Bitslice-Parallel Floating-Point Operations for Convolutional Neural Networks”⁠, James Garland, David Gregg2020-07-11 (AI scaling/hardware; similar)

“Bayesian Bits: Unifying Quantization and Pruning”, Baalen Et Al 2020

“Bayesian Bits: Unifying Quantization and Pruning”⁠, Mart van Baalen, Christos Louizos, Markus Nagel, Rana Ali Amjad, Ying Wang, Tijmen Blankevoort, Max Welling2020-05-14 (NN pruning; similar)

“General Purpose Text Embeddings from Pre-trained Language Models for Scalable Inference”, Du Et Al 2020

“General Purpose Text Embeddings from Pre-trained Language Models for Scalable Inference”⁠, Jingfei Du, Myle Ott, Haoran Li, Xing Zhou, Veselin Stoyanov2020-04-29 (knowledge distillation⁠, Transformer NN; similar)

“Lite Transformer With Long-Short Range Attention”, Wu Et Al 2020

“Lite Transformer with Long-Short Range Attention”⁠, Zhanghao Wu, Zhijian Liu, Ji Lin, Yujun Lin, Song Han2020-04-24 (NN pruning⁠, multi-scale Transformer; backlinks; similar)

“Training With Quantization Noise for Extreme Model Compression”, Fan Et Al 2020

“Training with Quantization Noise for Extreme Model Compression”⁠, Angela Fan, Pierre Stock, Benjamin Graham, Edouard Grave, Remi Gribonval, Herve Jegou, Arm, Joulin2020-04-15 (similar; bibliography)

“Train Large, Then Compress: Rethinking Model Size for Efficient Training and Inference of Transformers”, Li Et Al 2020

“Train Large, Then Compress: Rethinking Model Size for Efficient Training and Inference of Transformers”⁠, Zhuohan Li, Eric Wallace, Sheng Shen, Kevin Lin, Kurt Keutzer, Dan Klein, Joseph E. Gonzalez2020-02-26 (NN pruning⁠, AI scaling; backlinks; similar)

“SWAT: Sparse Weight Activation Training”, Raihan & Aamodt2020

“SWAT: Sparse Weight Activation Training”⁠, Md Aamir Raihan, Tor M. Aamodt2020-01-07 (similar; bibliography)

“QUARL: Quantized Reinforcement Learning (ActorQ)”, Lam Et Al 2019

“QUARL: Quantized Reinforcement Learning (ActorQ)”⁠, Maximilian Lam, Sharad Chitlangia, Srivatsan Krishnan, Zishen Wan, Gabriel Barth-Maron, Aleksandra Faust2019-10-02 (model-free RL; similar; bibliography)

“SCaNN: Accelerating Large-Scale Inference With Anisotropic Vector Quantization”, Guo Et Al 2019

“SCaNN: Accelerating Large-Scale Inference with Anisotropic Vector Quantization”⁠, Ruiqi Guo, Philip Sun, Erik Lindgren, Quan Geng, David Simcha, Felix Chern, Sanjiv Kumar2019-08-27 (similar)

“And the Bit Goes Down: Revisiting the Quantization of Neural Networks”, Stock Et Al 2019

“And the Bit Goes Down: Revisiting the Quantization of Neural Networks”⁠, Pierre Stock, Arm, Joulin, Rémi Gribonval, Benjamin Graham, Hervé Jégou2019-07-12 (similar)

“Rethinking Floating Point for Deep Learning”, Johnson2018

“Rethinking floating point for deep learning”⁠, Jeff Johnson2018-11-01 (similar)

“Learning Recurrent Binary/​Ternary Weights”, Ardakani Et Al 2018

“Learning Recurrent Binary/Ternary Weights”⁠, Arash Ardakani, Zhengyun Ji, Sean C. Smithson, Brett H. Meyer, Warren J. Gross2018-09-28 (RNN; similar)

“Rethinking Numerical Representations for Deep Neural Networks”, Hill Et Al 2018

“Rethinking Numerical Representations for Deep Neural Networks”⁠, Parker Hill, Babak Zamirai, Shengshuo Lu, Yu-Wei Chao, Michael Laurenzano, Mehrzad Samadi, Marios Papaefthymiou2018-08-07 (similar)

“Highly Scalable Deep Learning Training System With Mixed-Precision: Training ImageNet in 4 Minutes”, Jia Et Al 2018

“Highly Scalable Deep Learning Training System with Mixed-Precision: Training ImageNet in 4 Minutes”⁠, Xianyan Jia, Shutao Song, Wei He, Yangzihao Wang, Haidong Rong, Feihu Zhou, Liqiang Xie, Zhenyu Guo, Yuanzhou Yang2018-07-30 (AI scaling/hardware; similar)

“Quantization Mimic: Towards Very Tiny CNN for Object Detection”, Wei Et Al 2018

“Quantization Mimic: Towards Very Tiny CNN for Object Detection”⁠, Yi Wei, Xinyu Pan, Hongwei Qin, Wanli Ouyang, Junjie Yan2018-05-06 (similar)

“Training Imagenet in 3 Hours for $25; and CIFAR-10 for $0.26”, Howard2018

“Training Imagenet in 3 hours for $25; and CIFAR-10 for $0.26”⁠, Jeremy Howard2018-04-30 (experience curves; backlinks; similar; bibliography)

“High-Accuracy Low-Precision Training”, Sa Et Al 2018

“High-Accuracy Low-Precision Training”⁠, Christopher De Sa, Megan Leszczynski, Jian Zhang, Alana Marzoev, Christopher R. Aberger, Kunle Olukotun2018-03-09 (similar)

“Training Wide Residual Networks for Deployment Using a Single Bit for Each Weight”, McDonnell2018

“Training wide residual networks for deployment using a single bit for each weight”⁠, Mark D. McDonnell2018-02-23 (similar; bibliography)

“Universal Deep Neural Network Compression”, Choi Et Al 2018

“Universal Deep Neural Network Compression”⁠, Yoojin Choi, Mostafa El-Khamy, Jungwon Lee2018-02-07 (similar)

“Deep Gradient Compression: Reducing the Communication Bandwidth for Distributed Training”, Lin Et Al 2017

“Deep Gradient Compression: Reducing the Communication Bandwidth for Distributed Training”⁠, Yujun Lin, Song Han, Huizi Mao, Yu Wang, William J. Dally2017-12-05 (AI scaling/hardware; similar; bibliography)

“Shift: A Zero FLOP, Zero Parameter Alternative to Spatial Convolutions”, Wu Et Al 2017

“Shift: A Zero FLOP, Zero Parameter Alternative to Spatial Convolutions”⁠, Bichen Wu, Alvin Wan, Xiangyu Yue, Peter Jin, Sicheng Zhao, Noah Golmant, Amir Gholaminejad, Joseph Gonzalez2017-11-22 (similar; bibliography)

“Training Simplification and Model Simplification for Deep Learning: A Minimal Effort Back Propagation Method”, Sun Et Al 2017

“Training Simplification and Model Simplification for Deep Learning: A Minimal Effort Back Propagation Method”⁠, Xu Sun, Xuancheng Ren, Shuming Ma, Bingzhen Wei, Wei Li, Jingjing Xu, Houfeng Wang, Yi Zhang2017-11-17 (NN pruning; similar)

“Compressing Word Embeddings via Deep Compositional Code Learning”, Shu & Nakayama2017

“Compressing Word Embeddings via Deep Compositional Code Learning”⁠, Raphael Shu, Hideki Nakayama2017-11-03 (similar)

“Learning Discrete Weights Using the Local Reparameterization Trick”, Shayer Et Al 2017

“Learning Discrete Weights Using the Local Reparameterization Trick”⁠, Oran Shayer, Dan Levi, Ethan Fetaya2017-10-21 (similar)

“TensorQuant—A Simulation Toolbox for Deep Neural Network Quantization”, Loroch Et Al 2017

“TensorQuant—A Simulation Toolbox for Deep Neural Network Quantization”⁠, Dominik Marek Loroch, Norbert Wehn, Franz-Josef Pfreundt, Janis Keuper2017-10-13 (backlinks; similar)

“Mixed Precision Training”, Micikevicius Et Al 2017

“Mixed Precision Training”⁠, Paulius Micikevicius, Sharan Narang, Jonah Alben, Gregory Diamos, Erich Elsen, David Garcia, Boris Ginsburg2017-10-10 (GAN⁠, RNN⁠, AI scaling/hardware; similar)

“Bolt: Accelerated Data Mining With Fast Vector Compression”, Blalock & Guttag2017

“Bolt: Accelerated Data Mining with Fast Vector Compression”⁠, Davis W. Blalock, John V. Guttag2017-06-30 (retrieval AI; similar)

“Google’s Neural Machine Translation System: Bridging the Gap between Human and Machine Translation”, Wu Et Al 2016

“Google’s Neural Machine Translation System: Bridging the Gap between Human and Machine Translation”⁠, Yonghui Wu, Mike Schuster, Zhifeng Chen, Quoc V. Le, Mohammad Norouzi, Wolfgang Macherey, Maxim Krikun2016-09-26 (RNN⁠, LM tokenization; similar)

“Convolutional Networks for Fast, Energy-Efficient Neuromorphic Computing”, Esser Et Al 2016

“Convolutional Networks for Fast, Energy-Efficient Neuromorphic Computing”⁠, Steven K. Esser, Paul A. Merolla, John V. Arthur, Andrew S. Cassidy, Rathinakumar Appuswamy, Alexander Andreopoulos2016-03-28 (CNN⁠, AI scaling/hardware; similar)

Wikipedia

1. Bfloat16 floating-point format

2. Half-precision floating-point format

3. Minifloat

4. Quantization (signal processing)