“Organic Reaction Mechanism Classification Using Machine Learning”, Burés & Larrosa2023

“Organic reaction mechanism classification using machine learning”⁠, Jordi Burés, Igor Larrosa2023-01-25 (RNN⁠, science; similar; bibliography)

“DataMUX: Data Multiplexing for Neural Networks”, Murahari Et Al 2023

“DataMUX: Data Multiplexing for Neural Networks”⁠, Vishvak Murahari, Carlos E. Jimenez, Runzhe Yang, Karthik R. Narasimhan2023-01-13 (NN sparsity⁠, Transformer NN; backlinks; similar)

“Merging Enzymatic and Synthetic Chemistry With Computational Synthesis Planning”, Levin Et Al 2022

“Merging enzymatic and synthetic chemistry with computational synthesis planning”⁠, Itai Levin, Mengjie Liu, Christopher A. Voigt, Connor W. Coley2022-12-14 (biology⁠, model-based RL; similar; bibliography)

“Magic3D: High-Resolution Text-to-3D Content Creation”, Lin Et Al 2022

“Magic3D: High-Resolution Text-to-3D Content Creation”⁠, Chen-Hsuan Lin, Jun Gao, Luming Tang, Towaki Takikawa, Xiaohui Zeng, Xun Huang, Karsten Kreis, Sanja Fidler2022-11-18 (similar)

“How Much Does Attention Actually Attend? Questioning the Importance of Attention in Pretrained Transformers”, Hassid Et Al 2022

“How Much Does Attention Actually Attend? Questioning the Importance of Attention in Pretrained Transformers”⁠, Michael Hassid, Hao Peng, Daniel Rotem, Jungo Kasai, Ivan Montero, Noah A. Smith, Roy Schwartz2022-11-07 (self-attention; backlinks; similar; bibliography)

“The Unreasonable Effectiveness of Fully-Connected Layers for Low-Data Regimes”, Kocsis Et Al 2022

“The Unreasonable Effectiveness of Fully-Connected Layers for Low-Data Regimes”⁠, Peter Kocsis, Peter Súkeník, Guillem Brasó, Matthias Nießner, Laura Leal-Taixé, Ismail Elezi2022-10-11 (CNN⁠, knowledge distillation⁠, active learning; similar)

“DreamFusion: Text-to-3D Using 2D Diffusion”, Poole Et Al 2022

“DreamFusion: Text-to-3D using 2D Diffusion”⁠, Ben Poole, Ajay Jain, Jonathan T. Barron, Ben Mildenhall2022-09-29 (diffusion model; similar)

“Random Initializations Performing above Chance and How to Find Them”, Benzing Et Al 2022

“Random initializations performing above chance and how to find them”⁠, Frederik Benzing, Simon Schug, Robert Meier, Johannes von Oswald, Yassir Akram, Nicolas Zucchet, Laurence Aitchison2022-09-15 (similar)

“Scaling Laws vs Model Architectures: How Does Inductive Bias Influence Scaling?”, Tay Et Al 2022

“Scaling Laws vs Model Architectures: How does Inductive Bias Influence Scaling?”⁠, Yi Tay, Mostafa Dehghani, Samira Abnar, Hyung Won Chung, William Fedus, Jinfeng Rao, Sharan Narang, Vinh Q. Tran2022-07-21 (sparse Transformer⁠, MoE NN; similar; bibliography)

“Why Do Tree-based Models Still Outperform Deep Learning on Tabular Data?”, Grinsztajn Et Al 2022

“Why do tree-based models still outperform deep learning on tabular data?”⁠, Léo Grinsztajn, Edouard Oyallon, Gaël Varoquaux2022-07-18 (ML dataset⁠, Transformer NN⁠, AI scaling⁠, tabular ML; similar)

“Revisiting Pretraining Objectives for Tabular Deep Learning”, Rubachev Et Al 2022

“Revisiting Pretraining Objectives for Tabular Deep Learning”⁠, Ivan Rubachev, Artem Alekberov, Yury Gorishniy, Artem Babenko2022-07-07 (tabular ML; similar)

“RHO-LOSS: Prioritized Training on Points That Are Learnable, Worth Learning, and Not Yet Learnt”, Mindermann Et Al 2022

“RHO-LOSS: Prioritized Training on Points that are Learnable, Worth Learning, and Not Yet Learnt”⁠, Sören Mindermann, Jan Brauner, Muhammed Razzak, Mrinank Sharma, Andreas Kirsch, Winnie Xu, Benedikt Höltgen2022-06-14 (CNN⁠, Transformer NN⁠, active learning⁠, meta-learning; similar; bibliography)

“MLP-3D: A MLP-like 3D Architecture With Grouped Time Mixing”, Qiu Et Al 2022

“MLP-3D: A MLP-like 3D Architecture with Grouped Time Mixing”⁠, Zhaofan Qiu, Ting Yao, Chong-Wah Ngo, Tao Mei2022-06-13 (video analysis; similar)

“Sparse Mixers: Combining MoE and Mixing to Build a More Efficient BERT”, Lee-Thorp & Ainslie2022

“Sparse Mixers: Combining MoE and Mixing to build a more efficient BERT”⁠, James Lee-Thorp, Joshua Ainslie2022-05-24 (MoE NN; similar; bibliography)

“Towards Understanding Grokking: An Effective Theory of Representation Learning”, Liu Et Al 2022

“Towards Understanding Grokking: An Effective Theory of Representation Learning”⁠, Ziming Liu, Ouail Kitouni, Niklas Nolte, Eric J. Michaud, Max Tegmark, Mike Williams2022-05-20 (AI scaling; backlinks; similar)

“Deep Learning Meets Nonparametric Regression: Are Weight-Decayed DNNs Locally Adaptive?”, Zhang & Wang2022

“Deep Learning meets Nonparametric Regression: Are Weight-Decayed DNNs Locally Adaptive?”⁠, Kaiqi Zhang, Yu-Xiang Wang2022-04-20 (similar)

“MLP-ASR: Sequence-length Agnostic All-MLP Architectures for Speech Recognition”, Sakuma Et Al 2022

“MLP-ASR: Sequence-length agnostic all-MLP architectures for speech recognition”⁠, Jin Sakuma, Tatsuya Komatsu, Robin Scheibler2022-02-17 (backlinks; similar)

“Mixing and Shifting: Exploiting Global and Local Dependencies in Vision MLPs”, Zheng Et Al 2022

“Mixing and Shifting: Exploiting Global and Local Dependencies in Vision MLPs”⁠, Huangjie Zheng, Pengcheng He, Weizhu Chen, Mingyuan Zhou2022-02-14 (similar; bibliography)

“PNLP-Mixer: an Efficient All-MLP Architecture for Language”, Fusco Et Al 2022

“pNLP-Mixer: an Efficient all-MLP Architecture for Language”⁠, Francesco Fusco, Damian Pascual, Peter Staar2022-02-09 (backlinks; similar)

“Data-driven Emergence of Convolutional Structure in Neural Networks”, Ingrosso & Goldt2022

“Data-driven emergence of convolutional structure in neural networks”⁠, Alessandro Ingrosso, Sebastian Goldt2022-02-01 (neuroscience; backlinks; similar)

“When Shift Operation Meets Vision Transformer: An Extremely Simple Alternative to Attention Mechanism (ShiftViT)”, Wang Et Al 2022

“When Shift Operation Meets Vision Transformer: An Extremely Simple Alternative to Attention Mechanism (ShiftViT)”⁠, Guangting Wang, Yucheng Zhao, Chuanxin Tang, Chong Luo, Wenjun Zeng2022-01-26 (backlinks; similar; bibliography)

“ConvMixer: Patches Are All You Need?”, Trockman & Kolter2022

“ConvMixer: Patches Are All You Need?”⁠, Asher Trockman, J. Zico Kolter2022-01-24 (CNN; backlinks; similar; bibliography)

“MAXIM: Multi-Axis MLP for Image Processing”, Tu Et Al 2022

“MAXIM: Multi-Axis MLP for Image Processing”⁠, Zhengzhong Tu, Hossein Talebi, Han Zhang, Feng Yang, Peyman Milanfar, Alan Bovik, Yinxiao Li2022-01-09 (similar)

“Grokking: Generalization Beyond Overfitting on Small Algorithmic Datasets [paper]”, Power Et Al 2022

“Grokking: Generalization Beyond Overfitting on Small Algorithmic Datasets [paper]”⁠, Alethea Power, Yuri Burda, Harri Edwards, Igor Babuschkin, Vedant Misra2022-01-06 (AI scaling; similar)

“The GatedTabTransformer: An Enhanced Deep Learning Architecture for Tabular Modeling”, Cholakov & Kolev2022

“The GatedTabTransformer: An enhanced deep learning architecture for tabular modeling”⁠, Radostin Cholakov, Todor Kolev2022 (tabular ML; similar)

“MLP Architectures for Vision-and-Language Modeling: An Empirical Study”, Nie Et Al 2021

“MLP Architectures for Vision-and-Language Modeling: An Empirical Study”⁠, Yixin Nie, Linjie Li, Zhe Gan, Shuohang Wang, Chenguang Zhu, Michael Zeng, Zicheng Liu, Mohit Bansal2021-12-08 (AI scaling; backlinks; similar)

“Noether Networks: Meta-Learning Useful Conserved Quantities”, Alet Et Al 2021

“Noether Networks: Meta-Learning Useful Conserved Quantities”⁠, Ferran Alet, Dylan Doblar, Allan Zhou, Joshua Tenenbaum, Kenji Kawaguchi, Chelsea Finn2021-12-06 (CNN⁠, video analysis⁠, meta-learning; similar)

“Zero-Shot Text-Guided Object Generation With Dream Fields”, Jain Et Al 2021

“Zero-Shot Text-Guided Object Generation with Dream Fields”⁠, Ajay Jain, Ben Mildenhall, Jonathan T. Barron, Pieter Abbeel, Ben Poole2021-12-02 (CLIP; similar)

“MorphMLP: A Self-Attention Free, MLP-Like Backbone for Image and Video”, Zhang Et Al 2021

“MorphMLP: A Self-Attention Free, MLP-Like Backbone for Image and Video”⁠, David Junhao Zhang, Kunchang Li, Yunpeng Chen, Yali Wang, Shashwat Chandra, Yu Qiao, Luoqi Liu, Mike Zheng Shou2021-11-24 (video analysis; backlinks; similar)

“MetaFormer Is Actually What You Need for Vision”, Yu Et Al 2021

“MetaFormer is Actually What You Need for Vision”⁠, Weihao Yu, Mi Luo, Pan Zhou, Chenyang Si, Yichen Zhou, Xinchao Wang, Jiashi Feng, Shuicheng Yan2021-11-22 (backlinks; similar; bibliography)

“PointMixer: MLP-Mixer for Point Cloud Understanding”, Choe Et Al 2021

“PointMixer: MLP-Mixer for Point Cloud Understanding”⁠, Jaesung Choe, Chunghyun Park, Francois Rameau, Jaesik Park, In So Kweon2021-11-22 (backlinks; similar)

“Deep Learning without Shortcuts: Shaping the Kernel With Tailored Rectifiers”, Zhang Et Al 2021

“Deep Learning without Shortcuts: Shaping the Kernel with Tailored Rectifiers”⁠, Guodong Zhang, Aleksandar Botev, James Martens2021-11-18 (similar)

“ZerO Initialization: Initializing Residual Networks With Only Zeros and Ones”, Zhao Et Al 2021

“ZerO Initialization: Initializing Residual Networks with only Zeros and Ones”⁠, Jiawei Zhao, Florian Schäfer, Anima Anandkumar2021-10-25 (backlinks; similar)

“ADOP: Approximate Differentiable One-Pixel Point Rendering”, Rückert Et Al 2021

“ADOP: Approximate Differentiable One-Pixel Point Rendering”⁠, Darius Rückert, Linus Franke, Marc Stamminger2021-10-13 (video analysis; similar)

“Exploring the Limits of Large Scale Pre-training”, Abnar Et Al 2021

“Exploring the Limits of Large Scale Pre-training”⁠, Samira Abnar, Mostafa Dehghani, Behnam Neyshabur, Hanie Sedghi2021-10-05 (AI scaling; similar; bibliography)

“Rapid Training of Deep Neural Networks without Skip Connections or Normalization Layers Using Deep Kernel Shaping”, Martens Et Al 2021

“Rapid training of deep neural networks without skip connections or normalization layers using Deep Kernel Shaping”⁠, James Martens, Andy Ballard, Guillaume Desjardins, Grzegorz Swirszcz, Valentin Dalibard, Jascha Sohl-Dickstein2021-10-05 (similar)

“Sparse MLP for Image Recognition: Is Self-Attention Really Necessary?”, Tang Et Al 2021

“Sparse MLP for Image Recognition: Is Self-Attention Really Necessary?”⁠, Chuanxin Tang, Yucheng Zhao, Guangting Wang, Chong Luo, Wenxuan Xie, Wenjun Zeng2021-09-12 (backlinks; similar; bibliography)

“ConvMLP: Hierarchical Convolutional MLPs for Vision”, Li Et Al 2021

“ConvMLP: Hierarchical Convolutional MLPs for Vision”⁠, Jiachen Li, Ali Hassani, Steven Walton, Humphrey Shi2021-09-09 (backlinks; similar; bibliography)

“Sparse-MLP: A Fully-MLP Architecture With Conditional Computation”, Lou Et Al 2021

“Sparse-MLP: A Fully-MLP Architecture with Conditional Computation”⁠, Yuxuan Lou, Fuzhao Xue, Zangwei Zheng, Yang You2021-09-05 (MoE NN; backlinks; similar)

“Hire-MLP: Vision MLP via Hierarchical Rearrangement”, Guo Et Al 2021

“Hire-MLP: Vision MLP via Hierarchical Rearrangement”⁠, Jianyuan Guo, Yehui Tang, Kai Han, Xinghao Chen, Han Wu, Chao Xu, Chang Xu, Yunhe Wang2021-08-30 (similar; bibliography)

“RaftMLP: How Much Can Be Done Without Attention and With Less Spatial Locality?”, Tatsunami & Taki2021

“RaftMLP: How Much Can Be Done Without Attention and with Less Spatial Locality?”⁠, Yuki Tatsunami, Masato Taki2021-08-09 (backlinks; similar; bibliography)

“S²-MLPv2: Improved Spatial-Shift MLP Architecture for Vision”, Yu Et Al 2021

“S²-MLPv2: Improved Spatial-Shift MLP Architecture for Vision”⁠, Tan Yu, Xu Li, Yunfeng Cai, Mingming Sun, Ping Li2021-08-02 (similar; bibliography)

“CycleMLP: A MLP-like Architecture for Dense Prediction”, Chen Et Al 2021

“CycleMLP: A MLP-like Architecture for Dense Prediction”⁠, Shoufa Chen, Enze Xie, Chongjian Ge, Runjian Chen, Ding Liang, Ping Luo2021-07-21 (backlinks; similar; bibliography)

“AS-MLP: An Axial Shifted MLP Architecture for Vision”, Lian Et Al 2021

“AS-MLP: An Axial Shifted MLP Architecture for Vision”⁠, Dongze Lian, Zehao Yu, Xing Sun, Shenghua Gao2021-07-18 (backlinks; similar; bibliography)

“Real-time Neural Radiance Caching for Path Tracing”, Müller Et Al 2021

“Real-time Neural Radiance Caching for Path Tracing”⁠, Thomas Müller, Fabrice Rousselle, Jan Novák, Alexander Keller2021-06-23 (CS/algorithm; similar; bibliography)

“Vision Permutator: A Permutable MLP-Like Architecture for Visual Recognition”, Hou Et Al 2021

“Vision Permutator: A Permutable MLP-Like Architecture for Visual Recognition”⁠, Qibin Hou, Zihang Jiang, Li Yuan, Ming-Ming Cheng, Shuicheng Yan, Jiashi Feng2021-06-23 (backlinks; similar; bibliography)

“Towards Biologically Plausible Convolutional Networks”, Pogodin Et Al 2021

“Towards Biologically Plausible Convolutional Networks”⁠, Roman Pogodin, Yash Mehta, Timothy P. Lillicrap, Peter E. Latham2021-06-22 (neuroscience; backlinks; similar)

“Well-tuned Simple Nets Excel on Tabular Datasets”, Kadra Et Al 2021

“Well-tuned Simple Nets Excel on Tabular Datasets”⁠, Arlind Kadra, Marius Lindauer, Frank Hutter, Josif Grabocka2021-06-21 (tabular ML; backlinks; similar)

“PairConnect: A Compute-Efficient MLP Alternative to Attention”, Xu Et Al 2021

“PairConnect: A Compute-Efficient MLP Alternative to Attention”⁠, Zhaozhuo Xu, Minghao Yan, Junyan Zhang, Anshumali Shrivastava2021-06-15 (backlinks; similar)

“MLP Singer: Towards Rapid Parallel Korean Singing Voice Synthesis”, Tae Et Al 2021

“MLP Singer: Towards Rapid Parallel Korean Singing Voice Synthesis”⁠, Jaesung Tae, Hyeongju Kim, Younggun Lee2021-06-15 (AI/music⁠, GAN; backlinks; similar)

“S²-MLP: Spatial-Shift MLP Architecture for Vision”, Yu Et Al 2021

“S²-MLP: Spatial-Shift MLP Architecture for Vision”⁠, Tan Yu, Xu Li, Yunfeng Cai, Mingming Sun, Ping Li2021-06-14 (similar; bibliography)

“When Vision Transformers Outperform ResNets without Pre-training or Strong Data Augmentations”, Chen Et Al 2021

“When Vision Transformers Outperform ResNets without Pre-training or Strong Data Augmentations”⁠, Xiangning Chen, Cho-Jui Hsieh, Boqing Gong2021-06-03 (backlinks; similar; bibliography)

“Container: Context Aggregation Network”, Gao Et Al 2021

“Container: Context Aggregation Network”⁠, Peng Gao, Jiasen Lu, Hongsheng Li, Roozbeh Mottaghi, Aniruddha Kembhavi2021-06-02 (backlinks; similar; bibliography)

“MixerGAN: An MLP-Based Architecture for Unpaired Image-to-Image Translation”, Cazenavette & Guevara2021

“MixerGAN: An MLP-Based Architecture for Unpaired Image-to-Image Translation”⁠, George Cazenavette, Manuel Ladron De Guevara2021-05-28 (GAN; backlinks; similar)

“Pay Attention to MLPs”, Liu Et Al 2021

“Pay Attention to MLPs”⁠, Hanxiao Liu, Zihang Dai, David R. So, Quoc V. Le2021-05-17 (multi-scale Transformer; similar; bibliography)

“FNet: Mixing Tokens With Fourier Transforms”, Lee-Thorp Et Al 2021

“FNet: Mixing Tokens with Fourier Transforms”⁠, James Lee-Thorp, Joshua Ainslie, Ilya Eckstein, Santiago Ontanon2021-05-09 (self-attention; similar; bibliography)

“ResMLP: Feedforward Networks for Image Classification With Data-efficient Training”, Touvron Et Al 2021

“ResMLP: Feedforward networks for image classification with data-efficient training”⁠, Hugo Touvron, Piotr Bojanowski, Mathilde Caron, Matthieu Cord, Alaaeldin El-Nouby, Edouard Grave, Gautier Izacard2021-05-07 (knowledge distillation; similar)

“Multi-scale Inference of Genetic Trait Architecture Using Biologically Annotated Neural Networks”, Demetci Et Al 2021

“Multi-scale Inference of Genetic Trait Architecture using Biologically Annotated Neural Networks”⁠, Pinar Demetci, Wei Cheng, Gregory Darnell, Xiang Zhou, Sohini Ramachandran, Lorin Crawford2021-05-06 (heritability; backlinks; similar)

“Do You Even Need Attention? A Stack of Feed-Forward Layers Does Surprisingly Well on ImageNet”, Melas-Kyriazi2021

“Do You Even Need Attention? A Stack of Feed-Forward Layers Does Surprisingly Well on ImageNet”⁠, Luke Melas-Kyriazi2021-05-06 (self-attention; backlinks; similar; bibliography)

“RepMLP: Re-parameterizing Convolutions into Fully-connected Layers for Image Recognition”, Ding Et Al 2021

“RepMLP: Re-parameterizing Convolutions into Fully-connected Layers for Image Recognition”⁠, Xiaohan Ding, Chunlong Xia, Xiangyu Zhang, Xiaojie Chu, Jungong Han, Guiguang Ding2021-05-05 (backlinks; similar; bibliography)

“MLP-Mixer: An All-MLP Architecture for Vision”, Tolstikhin Et Al 2021

“MLP-Mixer: An all-MLP Architecture for Vision”⁠, Ilya Tolstikhin, Neil Houlsby, Alexander Kolesnikov, Lucas Beyer, Xiaohua Zhai, Thomas Unterthiner, Jessica Yung2021-05-04 (AI scaling; similar; bibliography)

“Grokking: Generalization Beyond Overfitting On Small Algorithmic Datasets”, Power Et Al 2021

“Grokking: Generalization Beyond Overfitting On Small Algorithmic Datasets”⁠, Alethea Power, Yuri Burda, Harri Edwards, Igor Babuschkin, Vedant Misra2021-05-01 (similar; bibliography)

“Sifting out the Features by Pruning: Are Convolutional Networks the Winning Lottery Ticket of Fully Connected Ones?”, Pellegrini & Biroli2021

“Sifting out the features by pruning: Are convolutional networks the winning lottery ticket of fully connected ones?”⁠, Franco Pellegrini, Giulio Biroli2021-04-27 (NN pruning; backlinks; similar)

“Fully-Connected Neural Nets”, Branwen2021

“Fully-Connected Neural Nets”⁠, Gwern Branwen2021-04-24 (AI scaling⁠, tabular ML; backlinks; similar; bibliography)

“Revisiting Simple Neural Probabilistic Language Models”, Sun & Iyyer2021

“Revisiting Simple Neural Probabilistic Language Models”⁠, Simeng Sun, Mohit Iyyer2021-04-08 (backlinks; similar)

“KiloNeRF: Speeding up Neural Radiance Fields With Thousands of Tiny MLPs”, Reiser Et Al 2021

“KiloNeRF: Speeding up Neural Radiance Fields with Thousands of Tiny MLPs”⁠, Christian Reiser, Songyou Peng, Yiyi Liao, Andreas Geiger2021-03-25 (knowledge distillation; backlinks; similar)

“Clusterability in Neural Networks”, Filan Et Al 2021

“Clusterability in Neural Networks”⁠, Daniel Filan, Stephen Casper, Shlomi Hod, Cody Wild, Andrew Critch, Stuart Russell2021-03-04 (NN sparsity; similar)

“Neural Geometric Level of Detail: Real-time Rendering With Implicit 3D Shapes”, Takikawa Et Al 2021

“Neural Geometric Level of Detail: Real-time Rendering with Implicit 3D Shapes”⁠, Towaki Takikawa, Joey Litalien, Kangxue Yin, Karsten Kreis, Charles Loop, Derek Nowrouzezahrai, Alec Jacobson2021-01-26 (backlinks; similar)

“Is MLP-Mixer a CNN in Disguise? As Part of This Blog Post, We Look at the MLP Mixer Architecture in Detail and Also Understand Why It Is Not Considered Convolution Free.”

“TabTransformer: Tabular Data Modeling Using Contextual Embeddings”, Huang Et Al 2020

“TabTransformer: Tabular Data Modeling Using Contextual Embeddings”⁠, Xin Huang, Ashish Khetan, Milan Cvitkovic, Zohar Karnin2020-12-11 (tabular ML; similar)

“Scaling down Deep Learning”, Greydanus2020

“Scaling down Deep Learning”⁠, Sam Greydanus2020-12-01 (CNN⁠, NN sparsity⁠, CS/hardware⁠, meta-learning⁠, RL scaling; backlinks; similar; bibliography)

“Image Generators With Conditionally-Independent Pixel Synthesis”, Anokhin Et Al 2020

“Image Generators with Conditionally-Independent Pixel Synthesis”⁠, Ivan Anokhin, Kirill Demochkin, Taras Khakhulin, Gleb Sterkin, Victor Lempitsky, Denis Korzhenkov2020-11-27 (GAN; backlinks; similar)

“Fourier Neural Operator for Parametric Partial Differential Equations”, Li Et Al 2020

“Fourier Neural Operator for Parametric Partial Differential Equations”⁠, Zongyi Li, Nikola Kovachki, Kamyar Azizzadenesheli, Burigede Liu, Kaushik Bhattacharya, Andrew Stuart2020-10-18 (backlinks; similar)

“AFT: An Attention Free Transformer”, Anonymous2020

“AFT: An Attention Free Transformer”⁠, Anonymous2020-09-28 (RNN⁠, Transformer matrix optimization; similar)

“Towards Learning Convolutions from Scratch”, Neyshabur2020

“Towards Learning Convolutions from Scratch”⁠, Behnam Neyshabur2020-07-27 (backlinks; similar)

“Efficient Attention: Breaking The Quadratic Transformer Bottleneck”, Branwen2020

“Efficient Attention: Breaking The Quadratic Transformer Bottleneck”⁠, Gwern Branwen2020-07-25 (self-attention⁠, GPT; backlinks; similar; bibliography)

“SIREN: Implicit Neural Representations With Periodic Activation Functions”, Sitzmann Et Al 2020

“SIREN: Implicit Neural Representations with Periodic Activation Functions”⁠, Vincent Sitzmann, Julien N. P. Martel, Alexander W. Bergman, David B. Lindell, Gordon Wetzstein2020-06-17 (video generation; backlinks; similar)

“Linformer: Self-Attention With Linear Complexity”, Wang Et Al 2020

“Linformer: Self-Attention with Linear Complexity”⁠, Sinong Wang, Belinda Z. Li, Madian Khabsa, Han Fang, Hao Ma2020-06-08 (Transformer matrix optimization; similar)

“A Map of Object Space in Primate Inferotemporal Cortex”, Bao Et Al 2020

“A map of object space in primate inferotemporal cortex”⁠, Pinglei Bao, Liang She, Mason McGill, Doris Y. Tsao2020-06-03 (neuroscience; similar)

“Synthesizer: Rethinking Self-Attention in Transformer Models”, Tay Et Al 2020

“Synthesizer: Rethinking Self-Attention in Transformer Models”⁠, Yi Tay, Dara Bahri, Donald Metzler, Da-Cheng Juan, Zhe Zhao, Che Zheng2020-05-02 (self-attention; similar; bibliography)

“Deep Learning Training in Facebook Data Centers: Design of Scale-up and Scale-out Systems”, Naumov Et Al 2020

“Deep Learning Training in Facebook Data Centers: Design of Scale-up and Scale-out Systems”⁠, Maxim Naumov, John Kim, Dheevatsa Mudigere, Srinivas Sridharan, Xiaodong Wang, Whitney Zhao, Serhat Yilmaz2020-03-20 (AI scaling; similar)

“NeRF: Representing Scenes As Neural Radiance Fields for View Synthesis”, Mildenhall Et Al 2020

“NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis”⁠, Ben Mildenhall, Pratul P. Srinivasan, Matthew Tancik, Jonathan T. Barron, Ravi Ramamoorthi, Ren Ng2020-03-19 (video generation; backlinks; similar)

“ReZero Is All You Need: Fast Convergence at Large Depth”, Bachlechner Et Al 2020

“ReZero is All You Need: Fast Convergence at Large Depth”⁠, Thomas Bachlechner, Bodhisattwa Prasad Majumder, Huanru Henry Mao, Garrison W. Cottrell, Julian McAuley2020-03-10 (backlinks; similar)

“Gesticulator: A Framework for Semantically-aware Speech-driven Gesture Generation”, Kucherenko Et Al 2020

“Gesticulator: A framework for semantically-aware speech-driven gesture generation”⁠, Taras Kucherenko, Patrik Jonell, Sanne van Waveren, Gustav Eje Henter, Simon Alexanderson, Iolanda Leite2020-01-25 (video analysis; backlinks; similar)

“Understanding the Generalization Of ‘Lottery Tickets’ In Neural Networks”, Morcos & Tian2019

“Understanding the generalization of ‘lottery tickets’ in neural networks”⁠, Ari Morcos, Yuandong Tian2019-11-25 (knowledge distillation⁠, AI scaling; backlinks; similar)

“3D Human Pose Estimation via Human Structure-aware Fully Connected Network”, Zhang Et Al 2019

“3D human pose estimation via human structure-aware fully connected network”⁠, Xiaoyan Zhang, Zhenhua Tang, Junhui Hou, Yanbin Hao2019-07-01 (tabular ML; similar)

“Finding the Needle in the Haystack With Convolutions: on the Benefits of Architectural Bias”, D’Ascoli Et Al 2019

“Finding the Needle in the Haystack with Convolutions: on the benefits of architectural bias”⁠, Stéphane d’Ascoli, Levent Sagun, Joan Bruna, Giulio Biroli2019-06-16 (backlinks; similar)

“MoGlow: Probabilistic and Controllable Motion Synthesis Using Normalizing Flows”, Henter Et Al 2019

“MoGlow: Probabilistic and controllable motion synthesis using normalizing flows”⁠, Gustav Eje Henter, Simon Alexanderson, Jonas Beskow2019-05-16 (RNN; backlinks; similar)

“Fixup Initialization: Residual Learning Without Normalization”, Zhang Et Al 2019

“Fixup Initialization: Residual Learning Without Normalization”⁠, Hongyi Zhang, Yann N. Dauphin, Tengyu Ma2019-01-27 (backlinks; similar)

“SwitchNet: a Neural Network Model for Forward and Inverse Scattering Problems”, Khoo & Ying2018

“SwitchNet: a neural network model for forward and inverse scattering problems”⁠, Yuehaw Khoo, Lexing Ying2018-10-23 (backlinks; similar)

“Scalable Training of Artificial Neural Networks With Adaptive Sparse Connectivity Inspired by Network Science”, Mocanu Et Al 2018

“Scalable training of artificial neural networks with adaptive sparse connectivity inspired by network science”⁠, Decebal Constantin Mocanu, Elena Mocanu, Peter Stone, Phuong H. Nguyen, Madeleine Gibescu, Antonio Liotta2018-06-19 (backlinks; similar)

“Deep Learning Generalizes Because the Parameter-function Map Is Biased towards Simple Functions”, Valle-Pérez Et Al 2018

“Deep learning generalizes because the parameter-function map is biased towards simple functions”⁠, Guillermo Valle-Pérez, Chico Q. Camargo, Ard A. Louis2018-05-22 (CNN⁠, AI scaling⁠, Bayes; similar)

“NAIS-Net: Stable Deep Networks from Non-Autonomous Differential Equations”, Ciccone Et Al 2018

“NAIS-Net: Stable Deep Networks from Non-Autonomous Differential Equations”⁠, Marco Ciccone, Marco Gallieri, Jonathan Masci, Christian Osendorfer, Faustino Gomez2018-04-19 (backlinks; similar)

“Improving Palliative Care With Deep Learning”, An Et Al 2018

“Improving palliative care with deep learning”⁠, An, Avati, Kenneth Jung, Stephanie Harman, Lance Downing, Andrew Ng, Nigam H. Shah2018 (tabular ML⁠, biology; similar)

“Repurposing High-Throughput Image Assays Enables Biological Activity Prediction for Drug Discovery”, Simm Et Al 2018

“Repurposing High-Throughput Image Assays Enables Biological Activity Prediction for Drug Discovery”⁠, Jaak Simm, Günter Klambauer, Adam Arany, Marvin Steijaert, Jörg Kurt Wegner, Emmanuel Gustin, Vladimir Chupakhin2018 (tabular ML; similar)

“Learning to Play Chess With Minimal Lookahead and Deep Value Neural Networks”, Sabatelli2017 (page 3)

“Learning to Play Chess with Minimal Lookahead and Deep Value Neural Networks”⁠, Matthia Sabatelli2017-10-30 (CNN⁠, AI chess; similar; bibliography)

“Sharp Models on Dull Hardware: Fast and Accurate Neural Machine Translation Decoding on the CPU”, Devlin2017

“Sharp Models on Dull Hardware: Fast and Accurate Neural Machine Translation Decoding on the CPU”⁠, Jacob Devlin2017-05-04 (RNN; backlinks; similar)

“The Shattered Gradients Problem: If Resnets Are the Answer, Then What Is the Question?”, Balduzzi Et Al 2017

“The Shattered Gradients Problem: If resnets are the answer, then what is the question?”⁠, David Balduzzi, Marcus Frean, Lennox Leary, J. P. Lewis, Kurt Wan-Duo Ma, Brian McWilliams2017-02-28 (backlinks; similar)

“Gender-From-Iris or Gender-From-Mascara?”, Kuehlkamp Et Al 2017

“Gender-From-Iris or Gender-From-Mascara?”⁠, Andrey Kuehlkamp, Benedict Becker, Kevin Bowyer2017-02-04 (CNN; backlinks; similar)

“Skip Connections Eliminate Singularities”, Orhan & Pitkow2017

“Skip Connections Eliminate Singularities”⁠, A. Emin Orhan, Xaq Pitkow2017-01-31 (backlinks; similar)

“Learning to Optimize”, Li & Malik2016

“Learning to Optimize”⁠, Ke Li, Jitendra Malik2016-06-06 (meta-learning⁠, model-free RL; backlinks; similar)

“Do Deep Convolutional Nets Really Need to Be Deep and Convolutional?”, Urban Et Al 2016

“Do Deep Convolutional Nets Really Need to be Deep and Convolutional?”⁠, Gregor Urban, Krzysztof J. Geras, Samira Ebrahimi Kahou, Ozlem Aslan, Shengjie Wang, Rich Caruana, Abdelrahman Mohamed2016-03-17 (CNN⁠, knowledge distillation⁠, AI scaling; backlinks; similar)

“Network Morphism”, Wei Et Al 2016

“Network Morphism”⁠, Tao Wei, Changhu Wang, Yong Rui, Chang Wen Chen2016-03-05 (CNN; similar)

“Adding Gradient Noise Improves Learning for Very Deep Networks”, Neelakantan Et Al 2015

“Adding Gradient Noise Improves Learning for Very Deep Networks”⁠, Arvind Neelakantan, Luke Vilnis, Quoc V. Le, Ilya Sutskever, Lukasz Kaiser, Karol Kurach, James Martens2015-11-21 (CNN; similar)

“How Far Can We Go without Convolution: Improving Fully-connected Networks”, Lin Et Al 2015

“How far can we go without convolution: Improving fully-connected networks”⁠, Zhouhan Lin, Rol, Memisevic, Kishore Konda2015-11-09 (autoencoder NN; backlinks; similar)

“Tensorizing Neural Networks”, Novikov Et Al 2015

“Tensorizing Neural Networks”⁠, Alexander Novikov, Dmitry Podoprikhin, Anton Osokin, Dmitry Vetrov2015-09-22 (NN sparsity; backlinks; similar)

“Deep Neural Networks for Large Vocabulary Handwritten Text Recognition”, Bluche2015

“Deep Neural Networks for Large Vocabulary Handwritten Text Recognition”⁠, Théodore Bluche2015-05-13 (RNN; backlinks; similar)

“The Loss Surfaces of Multilayer Networks”, Choromanska Et Al 2014

“The Loss Surfaces of Multilayer Networks”⁠, Anna Choromanska, Mikael Henaff, Michael Mathieu, Gérard Ben Arous, Yann LeCun2014-11-30 (similar)

“One Weird Trick for Parallelizing Convolutional Neural Networks”, Krizhevsky2014

“One weird trick for parallelizing convolutional neural networks”⁠, Alex Krizhevsky2014-04-23

“Do Deep Nets Really Need to Be Deep?”, Ba & Caruana2013

“Do Deep Nets Really Need to be Deep?”⁠, Lei Jimmy Ba, Rich Caruana2013-12-21 (knowledge distillation; backlinks; similar)

“Network In Network”, Lin Et Al 2013

“Network In Network”⁠, Min Lin, Qiang Chen, Shuicheng Yan2013-12-16 (backlinks; similar)

“Deep Big Multilayer Perceptrons for Digit Recognition”, Cireşan Et Al 2012

“Deep Big Multilayer Perceptrons for Digit Recognition”⁠, Dan Claudiu Cireşan, Ueli Meier, Luca Maria Gambardella, Jürgen Schmidhuber2012 (similar)

“Extraction De Séquences Numériques Dans Des Documents Manuscrits Quelconques”, Chatelain2006

“Extraction de séquences numériques dans des documents manuscrits quelconques”⁠, Clément Chatelain2006-12-05 (backlinks; similar)

“Best Practices for Convolutional Neural Networks Applied to Visual Document Analysis”, Simard Et Al 2003

“Best Practices for Convolutional Neural Networks Applied to Visual Document Analysis”⁠, Patrice Y. Simard, Dave Steinkraus, John C. Platt2003 (similar)

“NEAT: Evolving Neural Networks through Augmenting Topologies”, Stanley & Miikkulainen2002

“NEAT: Evolving Neural Networks through Augmenting Topologies”⁠, Kenneth O. Stanley, Risto Miikkulainen2002-06-01 (RL exploration; backlinks; similar)

“Quantitative Analysis of Multivariate Data Using Artificial Neural Networks: A Tutorial Review and Applications to the Deconvolution of Pyrolysis Mass Spectra”, Goodacre Et Al 1996

“Quantitative Analysis of Multivariate Data Using Artificial Neural Networks: A Tutorial Review and Applications to the Deconvolution of Pyrolysis Mass Spectra”⁠, Royston Goodacre, Mark J. Neal, Douglas B. Kell1996-08-01 (backlinks; similar)

“On the Ability of the Optimal Perceptron to Generalize”, Opper Et Al 1990

“On the ability of the optimal perceptron to generalize”⁠, M. Opper, W. Kinzel, J. Kleinz, R. Nehl1990 (similar)

“Learning To Tell Two Spirals Apart”, Lang & Witbrock1988

“Learning To Tell Two Spirals Apart”⁠, Kevin J. Lang, Michael J. Witbrock1988-01 (backlinks; similar)