Computer Optimization: Your Computer Is Faster Than You Think
Collapse or Thrive? Perils and Promises of Synthetic Data in a Self-Generating World
Robin Hanson: Prediction Markets, the Future of Civilization, and Polymathy—#66 § Opposition to DL
Simultaneous linear connectivity of neural networks modulo permutation
Tweets to Citations: Unveiling the Impact of Social Media Influencers on AI Research Visibility
Outliers with Opposing Signals Have an Outsized Effect on Neural Network Optimization
Proving Linear Mode Connectivity of Neural Networks via Optimal Transport
Monarch Mixer: A Simple Sub-Quadratic GEMM-Based Architecture
Dynamical versus Bayesian Phase Transitions in a Toy Model of Superposition
Going Beyond Linear Mode Connectivity: The Layerwise Linear Feature Connectivity
Combining Human Expertise with Artificial Intelligence: Experimental Evidence from Radiology
The Architecture of a Biologically Plausible Language Organ
Adam Accumulation to Reduce Memory Footprints of both Activations and Gradients for Large-scale DNN Training
Protecting Society from AI Misuse: When are Restrictions on Capabilities Warranted?
The Forward-Forward Algorithm: Some Preliminary Investigations
Self-Stabilization: The Implicit Bias of Gradient Descent at the Edge of Stability
Do Current Multi-Task Optimization Methods in Deep Learning Even Help?
Selective neutralization and deterring of cockroaches with laser automated by machine vision
Git Re-Basin: Merging Models modulo Permutation Symmetries
AniWho: A Quick and Accurate Way to Classify Anime Character Faces in Images
Zeus: Understanding and Optimizing GPU Energy Consumption of DNN Training
Learning with Combinatorial Optimization Layers: a Probabilistic Approach
Hidden Progress in Deep Learning: SGD Learns Parities Near the Computational Limit
High-performing neural network models of visual cortex benefit from high latent dimensionality
Perceptein: A synthetic protein-level neural network in mammalian cells
Predicting Word Learning in Children from the Performance of Computer Vision Systems
Wav2Vec-Aug: Improved self-supervised training with limited data
The Slingshot Mechanism: An Empirical Study of Adaptive Optimizers and the Grokking Phenomenon
Greedy Bayesian Posterior Approximation with Deep Ensembles
Generating Scientific Claims for Zero-Shot Scientific Fact Checking
Deep Lexical Hypothesis: Identifying personality structure in natural language
Towards Scaling Difference Target Propagation by Learning Backprop Targets
M5 accuracy competition: Results, findings, and conclusions
Formal Analysis of Art: Proxy Learning of Visual Concepts from Style Through Language Models
Silent Bugs in Deep Learning Frameworks: An Empirical Study of Keras and TensorFlow
Pushing the frontiers of density functionals by solving the fractional electron problem
Deep learning enables genetic analysis of the human thoracic aorta
Why Do Self-Supervised Models Transfer? Investigating the Impact of Invariance on Downstream Tasks
BC-Z: Zero-Shot Task Generalization with Robotic Imitation Learning
Learning in High Dimension Always Amounts to Extrapolation
The Role of Permutation Invariance in Linear Mode Connectivity of Neural Networks
The structure of genotype-phenotype maps makes fitness landscapes navigable
Learning through atypical "phase transitions" in overparameterized neural networks
DART: Differentiable Prompt Makes Pre-trained Language Models Better Few-shot Learners
Introducing Triton: Open-Source GPU Programming for Neural Networks
A connectivity-constrained computational account of topographic organization in primate high-level visual cortex
A Diverse Corpus for Evaluating and Developing English Math Word Problem Solvers
Coarse-to-Fine Q-attention: Efficient Learning for Visual Robotic Manipulation via Discretisation
Randomness In Neural Network Training: Characterizing The Impact of Tooling
Revisiting Model Stitching to Compare Neural Representations
Artificial intelligence in China’s revolution in military affairs
VICReg: Variance-Invariance-Covariance Regularization for Self-Supervised Learning
Understanding by Understanding Not: Modeling Negation in Language Models
PAWS: Semi-Supervised Learning of Visual Features by Non-Parametrically Predicting View Assignments with Support Samples
Epistemic Autonomy: Self-supervised Learning in the Mammalian Hippocampus
Positive-Negative Momentum: Manipulating Stochastic Gradient Noise to Improve Generalization
Contrasting Contrastive Self-Supervised Representation Learning Models
Characterizing and Improving the Robustness of Self-Supervised Learning through Background Augmentations
GWAS in almost 195,000 individuals identifies 50 previously unidentified genetic loci for eye color
Predictive Coding Can Do Exact Backpropagation on Any Neural Network
Barlow Twins: Self-Supervised Learning via Redundancy Reduction
WIT: Wikipedia-based Image Text Dataset for Multimodal Multilingual Machine Learning
The inverse variance–flatness relation in stochastic gradient descent is critical for finding flat minima
Gradient Descent on Neural Networks Typically Occurs at the Edge of Stability
Rip van Winkle’s Razor: A Simple Estimate of Overfit to Test Data
DirectPred: Understanding self-supervised Learning Dynamics without Contrastive Pairs
MLGO: a Machine Learning Guided Compiler Optimizations Framework
Facial recognition technology can expose political orientation from naturalistic facial images
Sixteen facial expressions occur in similar contexts worldwide
Real-time Synthesis of Imagined Speech Processes from Minimally Invasive Recordings of Neural Activity
Selective Eye-gaze Augmentation To Enhance Imitation Learning In Atari Games
Recent advances in neurotechnologies with broad potential for neuroscience research
Voting for Authorship Attribution Applied to Dark Web Data
Twenty Years Beyond the Turing Test: Moving Beyond the Human Judges Too
Hypersim: A Photorealistic Synthetic Dataset for Holistic Indoor Scene Understanding
Do Wide and Deep Networks Learn the Same Things? Uncovering How Neural Network Representations Vary with Width and Depth
Open-Domain Question Answering Goes Conversational via Question Rewriting
Rank-Smoothed Pairwise Learning In Perceptual Quality Assessment
Large Associative Memory Problem in Neurobiology and Machine Learning
Bootstrap your own latent (BYOL): A new approach to self-supervised Learning
Can You Put it All Together: Evaluating Conversational Agents’ Ability to Blend Skills
The large learning rate phase of deep learning: the catapult mechanism
The Next Decade in AI: Four Steps Towards Robust Artificial Intelligence
The Secret History of Facial Recognition: Sixty years ago, a sharecropper’s son invented a technology to identify faces. Then the record of his role all but vanished. Who was Woody Bledsoe, and who was he working for?
Can the Brain Do Backpropagation? -Exact Implementation of Backpropagation in Predictive Coding Networks
2019 AI Alignment Literature Review and Charity Comparison
Libri-Light: A Benchmark for ASR with Limited or No Supervision
12-in-1: Multi-Task Vision and Language Representation Learning
Machine Learning for Scent: Learning Generalizable Perceptual Representations of Small Molecules
KuroNet: Pre-Modern Japanese Kuzushiji Character Recognition with Deep Learning
Approximate Inference in Discrete Distributions with Monte Carlo Tree Search and Value Functions
Best practices for the human evaluation of automatically generated text
RandAugment: Practical automated data augmentation with a reduced search space
Large-scale Pretraining for Neural Machine Translation with Tens of Billions of Sentence Pairs
ALBERT: A Lite BERT for Self-supervised Learning of Language Representations
Neural networks are a priori biased towards Boolean functions with low entropy
Simple, Scalable Adaptation for Neural Machine Translation
A Step Toward Quantifying Independently Reproducible Machine Learning Research
Does Machine Translation Affect International Trade? Evidence from a Large Digital Platform
Can One Concurrently Record Electrical Spikes from Every Neuron in a Mammalian Brain?
Massively Multilingual Neural Machine Translation in the Wild: Findings and Challenges
Optimizing color for camouflage and visibility using deep learning: the effects of the environment and the observer’s visual system
SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systems
Universal quantum control through deep reinforcement learning
Analysing Mathematical Reasoning Abilities of Neural Models
Reinforcement Learning for Recommender Systems: A Case Study on Youtube
Stochastic Optimization of Sorting Networks via Continuous Relaxations
Surprises in High-Dimensional Ridgeless Least Squares Interpolation
DROP: A Reading Comprehension Benchmark Requiring Discrete Reasoning Over Paragraphs
A Replication Study: Machine Learning Models Are Capable of Predicting Sexual Orientation From Facial Images
Unmasking Clever Hans Predictors and Assessing What Machines Really Learn
What makes a good conversation? How controllable attributes affect human judgments
Evaluation and Accurate Diagnoses of Pediatric Diseases Using Artificial Intelligence
High-Performance Medicine: the Convergence of Human and Artificial Intelligence
Identifying Facial Phenotypes of Genetic Disorders Using Deep Learning
Reinventing the Wheel: Discovering the Optimal Rolling Shape With PyTorch
An Empirical Study of Example Forgetting during Deep Neural Network Learning
CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledge
Depth with Nonlinearity Creates No Bad Local Minima in ResNets
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Searching for Efficient Multi-Scale Architectures for Dense Image Prediction
Machine Learning to Predict Osteoporotic Fracture Risk from Genotypes
Accelerated Reinforcement Learning for Sentence Generation by Vocabulary Prediction
Searching Toward Pareto-Optimal Device-Aware Neural Architectures
A Study of Reinforcement Learning for Neural Machine Translation
Modeling Visual Context is Key to Augmenting Object Detection Datasets
Towards Automated Deep Learning: Efficient Joint Neural Architecture and Hyperparameter Search
Automatically Composing Representation Transformations as a Means for Generalization
Differentiable Learning-to-Normalize via Switchable Normalization
Neural Tangent Kernel: Convergence and Generalization in Neural Networks
Meta-Learning Transferable Active Learning Policies by Deep Reinforcement Learning
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding
Adafactor: Adaptive Learning Rates with Sublinear Memory Cost
Averaging Weights Leads to Wider Optima and Better Generalization
SentEval: An Evaluation Toolkit for Universal Sentence Representations
Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge
End-to-end deep image reconstruction from human brain activity
Back to Basics: Benchmarking Canonical Evolution Strategies for Playing Atari
Differentiable Dynamic Programming for Structured Prediction and Attention
UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction
Semantic projection: recovering human knowledge of multiple, distinct object features from word embeddings
Clinically Applicable Deep Learning for Diagnosis and Referral in Retinal Disease
Prediction of cardiovascular risk factors from retinal fundus photographs via deep learning
Three-dimensional visualization and a deep-learning model reveal complex fungal parasite networks in behaviorally manipulated ants
Rethinking generalization requires revisiting old ideas: statistical mechanics approaches and complex learning behavior
AlphaGo Zero: Mastering the game of Go without human knowledge
Super-Convergence: Very Fast Training of Neural Networks Using Large Learning Rates
Cut, Paste and Learn: Surprisingly Easy Synthesis for Instance Detection
Verb Physics: Relative Physical Knowledge of Actions and Objects
Driver Identification Using Automobile Sensor Data from a Single Turn
StreetStyle: Exploring world-wide clothing styles from millions of photos
WebVision Challenge: Visual Learning and Understanding With Web Data
A Broad-Coverage Challenge Corpus for Sentence Understanding through Inference
RACE: Large-scale ReAding Comprehension Dataset From Examinations
Data-efficient Deep Reinforcement Learning for Dexterous Manipulation
Understanding Synthetic Gradients and Decoupled Neural Interfaces
Reluplex: An Efficient SMT Solver for Verifying Deep Neural Networks
Dermatologist-Level Classification of Skin Cancer With Deep Neural Networks
Machine Learning for Systems and Systems for Machine Learning
CLEVR: A Diagnostic Dataset for Compositional Language and Elementary Visual Reasoning
Deep Learning Reinvents the Hearing Aid: Finally, wearers of hearing aids can pick out a voice in a crowded room
MS MARCO: A Human Generated MAchine Reading COmprehension Dataset
SQuAD: 100,000+ Questions for Machine Comprehension of Text
Synthesizing the preferred inputs for neurons in neural networks via deep generator networks
Toward Deeper Understanding of Neural Networks: The Power of Initialization and a Dual View on Expressivity
"Why Should I Trust You?": Explaining the Predictions of Any Classifier
Mastering the game of Go with deep neural networks and tree search
Learning to Compose Neural Networks for Question Answering
How a Japanese Cucumber Farmer Is Using Deep Learning and TensorFlow
Data-dependent Initializations of Convolutional Neural Networks
Online Batch Selection for Faster Training of Neural Networks
Deep DPG (DDPG): Continuous control with deep reinforcement learning
Towards AI-Complete Question Answering: A Set of Prerequisite Toy Tasks
Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification
Qualitatively characterizing neural network optimization problems
Exact solutions to the nonlinear dynamics of learning in deep linear neural networks
Distributed Representations of Words and Phrases and their Compositionality
Whatever next? Predictive brains, situated agents, and the future of cognitive science
Artist Agent: A Reinforcement Learning Approach to Automatic Stroke Generation in Oriental Ink Painting
HOGWILD!: A Lock-Free Approach to Parallelizing Stochastic Gradient Descent
Large-scale deep unsupervised learning using graphics processors
Understanding the nature of the general factor of intelligence: The role of individual differences in neural plasticity as an explanatory mechanism
Exponentiated Gradient versus Gradient Descent for Linear Predictors
A Sociological Study of the Official History of the Perceptrons Controversy
Learning and generalization in a two-layer neural network: The role of the Vapnik-Chervonvenkis dimension
A Sociological Study of the Official History of the Perceptrons Controversy [1993]
Memorization Without Generalization in a Multilayered Neural Network
Symbolic and neural learning algorithms: An experimental comparison
Backpropagation Learning For Multilayer Feed-Forward Neural Networks Using The Conjugate Gradient Method
Artificial Neural Networks, Back Propagation, and the Kelley-Bryson Gradient Procedure
International Joint Conference on Neural Networks, January 15–19, 1990: Volume 1: Theory Track, Neural and Cognitive Sciences Track
International Joint Conference on Neural Networks, January 15–19, 1990: Volume 2: Applications Track
Parallel Distributed Processing: Implications for Cognition and Development
Observation of Phase Transitions in Spreading Activation Networks
Storing Infinite Numbers of Patterns in a Spin-Glass Model of Neural Networks
Beyond Regression: New Tools for Prediction and Analysis in the Behavioral Sciences
Principles of Neurodynamics: Perceptrons and the Theory of Brain Mechanisms
Some AI Koans § Http://www.catb.org/esr/jargon/html/koans.html#id3141241
gsutil config: Obtain credentials and create configuration file
Code for Reproducing Results in "Glow: Generative Flow With Invertible 1×1 Convolutions"
Simple versus Short: Higher-Order Degeneracy and Error-Correction
Inferring Neural Activity Before Plasticity As a Foundation for Learning beyond Backpropagation
2022-12-02-gwern-meme-itsafraid-googlereluctancetoproductizedeeplearningresearch.jpg
2022-grand-figure2-semanticprojectionpredictionshumanjudgmentsexamplesofdangersizewitnessanimalscitiesmythologicalcreatures.jpg
2021-santospata-figure1-hippocampusselfsupervisionlearning.jpg
2008-03-03-jonahlehrer-outofthebluecanathinkingrememberingdecisionmakingbiologicallyaccuratebrainbebuiltfromasupercomputer.html
1991-sethi-artificialneuralnetworksandstatisticalpatternrecognition.pdf
http://unremediatedgender.space/2018/Jan/blame-me-for-trying/
https://aleph.se/andart2/math/weird-probability-distributions/
https://juretriglav.si/compressing-global-illumination-with-neural-networks/
https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4553431
https://people.idsia.ch/~juergen/DanNet-triggers-deep-CNN-revolution-2011.html
https://static.googleusercontent.com/media/research.google.com/en//pubs/archive/37648.pdf
https://web.archive.org/web/20220927022638/https://nautil.us/the-man-who-tried-to-redeem-the-world-with-logic-235253/
https://www.chinalawtranslate.com/overview-of-draft-measures-on-generative-ai/
https://www.kaggle.com/code/andy8744/predict-anime-face-using-pre-trained-model/data
https://www.lesswrong.com/posts/QNQuWB3hS5FrGp5yZ/programmatic-backdoors-dnns-can-use-sgd-to-run-arbitrary
https://www.lesswrong.com/posts/RKDQCB6smLWgs2Mhr/multi-component-learning-and-s-curves
https://www.lesswrong.com/posts/XpCnhaAQrssq8tJBG/an-interactive-introduction-to-grokking-and-mechanistic
https://www.neelnanda.io/mechanistic-interpretability/favourite-papers
https://www.protocol.com/china/i-built-bytedance-censorship-machine
https://www.quantamagazine.org/to-be-energy-efficient-brains-predict-their-perceptions-20211115/
https://www.vox.com/future-perfect/23775650/ai-regulation-openai-gpt-anthropic-midjourney-stable
Monarch Mixer: A Simple Sub-Quadratic GEMM-Based Architecture
Combining Human Expertise with Artificial Intelligence: Experimental Evidence from Radiology
https%253A%252F%252Fwww.nber.org%252Fpapers%252Fw31422.html
https%253A%252F%252Farxiv.org%252Fabs%252F2302.06675%2523google.html
Selective neutralization and deterring of cockroaches with laser automated by machine vision
https%253A%252F%252Fwww.tandfonline.com%252Fdoi%252Ffull%252F10.1080%252F00305316.2022.2121777.html
AniWho: A Quick and Accurate Way to Classify Anime Character Faces in Images
Towards Scaling Difference Target Propagation by Learning Backprop Targets
M5 accuracy competition: Results, findings, and conclusions
https%253A%252F%252Fwww.sciencedirect.com%252Fscience%252Farticle%252Fpii%252FS0169207021001874.html
Silent Bugs in Deep Learning Frameworks: An Empirical Study of Keras and TensorFlow
Pushing the frontiers of density functionals by solving the fractional electron problem
%252Fdoc%252Fai%252Fnn%252F2021-kirkpatrick.pdf%2523deepmind.html
Learning through atypical "phase transitions" in overparameterized neural networks
https%253A%252F%252Farxiv.org%252Fabs%252F2106.08254%2523microsoft.html
PAWS: Semi-Supervised Learning of Visual Features by Non-Parametrically Predicting View Assignments with Support Samples
https%253A%252F%252Farxiv.org%252Fabs%252F2104.13963%2523facebook.html
Contrasting Contrastive Self-Supervised Representation Learning Models
Characterizing and Improving the Robustness of Self-Supervised Learning through Background Augmentations
https%253A%252F%252Farxiv.org%252Fabs%252F2103.12719%2523facebook.html
DirectPred: Understanding self-supervised Learning Dynamics without Contrastive Pairs
https%253A%252F%252Farxiv.org%252Fabs%252F2102.06810%2523facebook.html
https%253A%252F%252Farxiv.org%252Fabs%252F2004.11362%2523google.html
2019 AI Alignment Literature Review and Charity Comparison
https%253A%252F%252Fwww.lesswrong.com%252Fposts%252FSmDziGM9hBjW9DKmf%252F2019-ai-alignment-literature-review-and-charity-comparison.html
https%253A%252F%252Farxiv.org%252Fabs%252F1912.03098%2523google.html
ALBERT: A Lite BERT for Self-supervised Learning of Language Representations
https%253A%252F%252Farxiv.org%252Fabs%252F1909.11942%2523google.html
SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systems
Differentiable Learning-to-Normalize via Switchable Normalization
Averaging Weights Leads to Wider Optima and Better Generalization
Back to Basics: Benchmarking Canonical Evolution Strategies for Playing Atari
AlphaGo Zero: Mastering the game of Go without human knowledge
%252Fdoc%252Freinforcement-learning%252Fmodel%252Falphago%252F2017-silver.pdf%2523deepmind.html
Super-Convergence: Very Fast Training of Neural Networks Using Large Learning Rates
https%253A%252F%252Fwww.sciencedirect.com%252Fscience%252Farticle%252Fpii%252FS0896627317303653.html
WebVision Challenge: Visual Learning and Understanding With Web Data
Large-scale deep unsupervised learning using graphics processors
%252Fdoc%252Fai%252Fscaling%252Fhardware%252F2009-raina.pdf.html
Learning and generalization in a two-layer neural network: The role of the Vapnik-Chervonvenkis dimension
A Sociological Study of the Official History of the Perceptrons Controversy [1993]
Memorization Without Generalization in a Multilayered Neural Network
Parallel Distributed Processing: Implications for Cognition and Development
Storing Infinite Numbers of Patterns in a Spin-Glass Model of Neural Networks
Wikipedia Bibliography:
