Junxiong Wang

Currently, I am working on the intersection of language models and systems.

Efficient architectures (i.e., linear RNN models) and systems for modeling long sequence.
Fast video frame retrieval with adaptive indexing using LLMs.

Previously, I worked on large-scale data processing systems and distributed computing systems.

I am incredibly fortunate to have worked on

Entity retrieval in the Siri Information Intelligence Knowledge Platform at Apple.
Automatic indexing for big data systems in the Data Systems Group at Microsoft Research.

Junxiong Wang*, Daniele Paliotta*, Avner May, Alexander M. Rush, Tri Dao
The Mamba in the Llama: Distilling and Accelerating Hybrid Models
Models, Video, Code, Blog
Neural Information Processing Systems (NeurIPS), 2024
A shorter version at ICML 2024, 2nd Workshop on Efficient Systems for Foundation Models (ES-FoMo)
Junxiong Wang, Tushaar Gangavarapu, Jing Nathan Yan, Alexander M. Rush
MambaByte: Token-free Selective State Space Model
Models, Video
Conference on Language Modeling (CoLM), 2024
Junxiong Wang, Jing Nathan Yan, Albert Gu, Alexander M. Rush
Pretraining Without Attention
Findings of Empirical Methods in Natural Language Processing (EMNLP), 2023
First non-attention bidirectional model which achieves BERT-level transfer learning on the GLUE benchmark
Models, Slides

Junxiong Wang, Ali Mousavi, Omar Attia, Saloni Potdar, Alexander M. Rush, Umar Farooq Minhas, Yunyao Li
Disambiguation via Fusion Entity Decoding
North American Chapter of the Association for Computational Linguistics (NAACL), 2024

Junxiong Wang, Mitchell Gray, Immanuel Trummer, Ahmet Kara, Dan Olteanu
ADOPT: Adaptively Optimizing Attribute Orders for Worst-Case Optimal Join Algorithms via Reinforcement Learning
International Conference on Very Large Data Bases (VLDB), 2023

Junxiong Wang et al. Demonstrating ADOPT: Adaptively Optimizing Attribute Orders for Worst-Case Optimal Joins via Reinforcement Learning
International Conference on Very Large Data Bases (VLDB), 2023

Wentao Wu, Chi Wang, Tarique Siddiqui, Junxiong Wang, Vivek R. Narasayya, Surajit Chaudhuri, Philip A. Bernstein
Budget-aware Index Tuning with Reinforcement Learning
ACM SIGMOD International Conference on Management of Data (SIGMOD), 2022

Junxiong Wang, Immanuel Trummer, Debabrota Basu
UDO: Universal Database Optimization using Reinforcement Learning
International Conference on Very Large Data Bases (VLDB), 2022

Junxiong Wang et al. Demonstrating UDO: A Unified Approach for Optimizing Transaction Code, Physical Design, and System Parameters via Reinforcement Learning
ACM SIGMOD International Conference on Management of Data (SIGMOD), 2021

Immanuel Trummer, Junxiong Wang, Deepak Maram, Saehan Jo, Samuel Moseley, Joseph Antonakakis
SkinnerDB: Regret-bounded Query Evaluation via Reinforcement Learning
ACM SIGMOD International Conference on Management of Data (SIGMOD), 2019

"Best of SIGMOD", extended version in ACM Transactions on Database Systems (TODS), 2021

Junxiong Wang et al. Procrastinated Tree Search: Black-box Optimization with Delayed, Noisy, and Multi-fidelity Feedback
AAAI Conference on Artificial Intelligence (AAAI), 2022, Slides

Marcos K. Aguilera*, Tudor David*, Rachid Guerraoui*, Junxiong Wang* (* alphabetical order for theory paper convention)
Locking Timestamps Versus Locking Objects
ACM SIGACT-SIGOPS Symposium on Principles of Distributed Computing (PODC), 2018
Associated code here