“‘Instruct-Tuning LLMs’ Tag”,2021-04-18 (ML dataset, NN sampling, inner monologue (AI), AI scaling, Decision Transformer, preference learning, RL scaling; backlinks)⁠:

Bibliography for tag ai/nn/transformer/gpt/instruction-tuning, most recent first: 5 related tags, 79 annotations, & 25 links (parent).

• See Also
• Links
  • “SANA: Efficient High-Resolution Image Synthesis With Linear Diffusion Transformers”, Xie et al 2024
  • “Instruction Following without Instruction Tuning”, Hewitt et al 2024
  • “Hermes 3 Technical Report”, Teknium et al 2024
  • “State Soup: In-Context Skill Learning, Retrieval and Mixing”, Pióro et al 2024
  • “Auto Evol-Instruct: Automatic Instruction Evolving for Large Language Models”, Zeng et al 2024
  • “Instruction Modeling: Instruction Tuning With Loss Over Instructions”, Shi et al 2024
  • “The Instruction Hierarchy: Training LLMs to Prioritize Privileged Instructions”, Wallace et al 2024
  • “Best Practices and Lessons Learned on Synthetic Data for Language Models”, Liu et al 2024
  • “RecurrentGemma: Moving Past Transformers for Efficient Open Language Models”, Botev et al 2024
  • “Length-Controlled AlpacaEval: A Simple Way to Debias Automatic Evaluators”, Dubois et al 2024
  • “COIG-CQIA: Quality Is All You Need for Chinese Instruction Fine-Tuning”, Bai et al 2024
  • “MetaAligner: Conditional Weak-To-Strong Correction for Generalizable Multi-Objective Alignment of Language Models”, Yang et al 2024
  • “Mastering Text, Code and Math Simultaneously via Fusing Highly Specialized Language Models”, Ding et al 2024
  • “StructLM: Towards Building Generalist Models for Structured Knowledge Grounding”, Zhuang et al 2024
  • “How to Train Data-Efficient LLMs”, Sachdeva et al 2024
  • “Rephrasing the Web (WARP): A Recipe for Compute and Data-Efficient Language Modeling”, Maini et al 2024
  • “WaveCoder: Widespread And Versatile Enhanced Instruction Tuning With Refined Data Generation”, Yu et al 2023
  • “VILA: On Pre-Training for Visual Language Models”, Lin et al 2023
  • “Instruction-Tuning Aligns LLMs to the Human Brain”, Aw et al 2023
  • “R-Tuning: Teaching Large Language Models to Refuse Unknown Questions”, Zhang et al 2023
  • “When ‘A Helpful Assistant’ Is Not Really Helpful: Personas in System Prompts Do Not Improve Performances of Large Language Models”, Zheng et al 2023
  • “Language Models Are Super Mario (DARE): Absorbing Abilities from Homologous Models As a Free Lunch”, Yu et al 2023
  • “ChipNeMo: Domain-Adapted LLMs for Chip Design”, Liu et al 2023
  • “Mistral-7B”, Jiang et al 2023
  • “LLMLingua: Compressing Prompts for Accelerated Inference of Large Language Models”, Jiang et al 2023
  • “LLaVA-1.5: Improved Baselines With Visual Instruction Tuning”, Liu et al 2023
  • “UltraFeedback: Boosting Language Models With High-Quality Feedback”, Cui et al 2023
  • “AceGPT, Localizing Large Language Models in Arabic”, Huang et al 2023
  • “Can Programming Languages Boost Each Other via Instruction Tuning?”, Zan et al 2023
  • “DialogStudio: Towards Richest and Most Diverse Unified Dataset Collection for Conversational AI”, Zhang et al 2023
  • “LLaMA-2: Open Foundation and Fine-Tuned Chat Models”, Touvron et al 2023
  • “AlpaGasus: Training A Better Alpaca With Fewer Data”, Chen et al 2023
  • “Instruction Mining: High-Quality Instruction Data Selection for Large Language Models”, Cao et al 2023
  • “Lost in the Middle: How Language Models Use Long Contexts”, Liu et al 2023
  • “On the Exploitability of Instruction Tuning”, Shu et al 2023
  • “ChessGPT: Bridging Policy Learning and Language Modeling”, Feng et al 2023
  • “Dr. LLaMa: Improving Small Language Models in Domain-Specific QA via Generative Data Augmentation”, Guo et al 2023
  • “SELF-ALIGN: Principle-Driven Self-Alignment of Language Models from Scratch With Minimal Human Supervision”, Sun et al 2023
  • “Distilling Step-By-Step! Outperforming Larger Language Models With Less Training Data and Smaller Model Sizes”, Hsieh et al 2023
  • “LaMini-LM: A Diverse Herd of Distilled Models from Large-Scale Instructions”, Wu et al 2023
  • “WizardLM: Empowering Large Language Models to Follow Complex Instructions”, Xu et al 2023
  • “TANGO: Text-To-Audio Generation Using Instruction-Tuned LLM and Latent Diffusion Model”, Ghosal et al 2023
  • “Phoenix: Democratizing ChatGPT across Languages”, Chen et al 2023
  • “How Well Do Large Language Models Perform in Arithmetic Tasks?”, Yuan et al 2023
  • “Larger Language Models Do In-Context Learning Differently”, Wei et al 2023
  • “LLaMa-1: Open and Efficient Foundation Language Models”, Touvron et al 2023
  • “How Does In-Context Learning Help Prompt Tuning?”, Sun et al 2023
  • “Med-PaLM: Large Language Models Encode Clinical Knowledge”, Singhal et al 2022
  • “Self-Instruct: Aligning Language Models With Self-Generated Instructions”, Wang et al 2022
  • “Unnatural Instructions: Tuning Language Models With (Almost) No Human Labor”, Honovich et al 2022
  • “One Embedder, Any Task: Instruction-Finetuned Text Embeddings (INSTRUCTOR)”, Su et al 2022
  • “HALIE: Evaluating Human-Language Model Interaction”, Lee et al 2022
  • “BLOOMZ/mT0: Crosslingual Generalization through Multitask Finetuning”, Muennighoff et al 2022
  • “Help Me Write a Poem: Instruction Tuning As a Vehicle for Collaborative Poetry Writing (CoPoet)”, Chakrabarty et al 2022
  • “FLAN: Scaling Instruction-Finetuned Language Models”, Chung et al 2022
  • “Language Models Are Multilingual Chain-Of-Thought Reasoners”, Shi et al 2022
  • “LINGUIST: Language Model Instruction Tuning to Generate Annotated Utterances for Intent Classification and Slot Tagging”, Rosenbaum et al 2022
  • “Z-Code++: A Pre-Trained Language Model Optimized for Abstractive Summarization”, He et al 2022
  • “Few-Shot Adaptation Works With UnpredicTable Data”, Chan et al 2022
  • “RST: ReStructured Pre-Training”, Yuan & Liu2022
  • “InstructDial: Improving Zero and Few-Shot Generalization in Dialogue through Instruction Tuning”, Gupta et al 2022
  • “CT0: Fine-Tuned Language Models Are Continual Learners”, Scialom et al 2022
  • “Tk-Instruct: Benchmarking Generalization via In-Context Instructions on 1,600+ Language Tasks”, Wang et al 2022
  • “What Language Model Architecture and Pretraining Objective Work Best for Zero-Shot Generalization?”, Wang et al 2022
  • “UnifiedQA-V2: Stronger Generalization via Broader Cross-Format Training”, Khashabi et al 2022
  • “Reasoning Like Program Executors”, Pi et al 2022
  • “ZeroPrompt: Scaling Prompt-Based Pretraining to 1,000 Tasks Improves Zero-Shot Generalization”, Xu et al 2022
  • “ExT5: Towards Extreme Multi-Task Scaling for Transfer Learning”, Aribandi et al 2021
  • “MetaICL: Learning to Learn In Context”, Min et al 2021
  • “T0: Multitask Prompted Training Enables Zero-Shot Task Generalization”, Sanh et al 2021
  • “FLAN: Finetuned Language Models Are Zero-Shot Learners”, Wei et al 2021
  • “Cross-Task Generalization via Natural Language Crowdsourcing Instructions”, Mishra et al 2021
  • “CrossFit: A Few-Shot Learning Challenge for Cross-Task Generalization in NLP”, Ye et al 2021
  • “Adapting Language Models for Zero-Shot Learning by Meta-Tuning on Dataset and Prompt Collections”, Zhong et al 2021
  • “Muppet: Massive Multi-Task Representations With Pre-Finetuning”, Aghajanyan et al 2021
  • “UnifiedQA: Crossing Format Boundaries With a Single QA System”, Khashabi et al 2020
  • “The Natural Language Decathlon: Multitask Learning As Question Answering”, McCann et al 2018
  • “No Robots: Look Ma, an Instruction Dataset That Wasn’t Generated by GPTs!”, HuggingFace2024
  • “The RetroInstruct Guide To Synthetic Text Data”, Pressman2024
• Miscellaneous
• Bibliography

Backlinks: