- See Also
-
Links
- “Can Programming Languages Boost Each Other via Instruction Tuning?”, Zan et al 2023
- “Simple Synthetic Data Reduces Sycophancy in Large Language Models”, Wei et al 2023
- “DialogStudio: Towards Richest and Most Diverse Unified Dataset Collection for Conversational AI”, Zhang et al 2023
- “LLaMA 2: Open Foundation and Fine-Tuned Chat Models”, Touvron et al 2023
- “AlpaGasus: Training A Better Alpaca With Fewer Data”, Chen et al 2023
- “Instruction Mining: High-Quality Instruction Data Selection for Large Language Models”, Cao et al 2023
- “Lost in the Middle: How Language Models Use Long Contexts”, Liu et al 2023
- “On the Exploitability of Instruction Tuning”, Shu et al 2023
- “ChessGPT: Bridging Policy Learning and Language Modeling”, Feng et al 2023
- “Dr. LLaMa: Improving Small Language Models in Domain-Specific QA via Generative Data Augmentation”, Guo et al 2023
- “LaMini-LM: A Diverse Herd of Distilled Models from Large-Scale Instructions”, Wu et al 2023
- “TANGO: Text-to-Audio Generation Using Instruction-Tuned LLM and Latent Diffusion Model”, Ghosal et al 2023
- “Phoenix: Democratizing ChatGPT across Languages”, Chen et al 2023
- “How Well Do Large Language Models Perform in Arithmetic Tasks?”, Yuan et al 2023
- “LLaMa: Open and Efficient Foundation Language Models”, Touvron et al 2023
- “Med-PaLM: Large Language Models Encode Clinical Knowledge”, Singhal et al 2022
- “One Embedder, Any Task: Instruction-Finetuned Text Embeddings (INSTRUCTOR)”, Su et al 2022
- “Unnatural Instructions: Tuning Language Models With (Almost) No Human Labor”, Honovich et al 2022
- “BLOOMZ/mT0: Crosslingual Generalization through Multitask Finetuning”, Muennighoff et al 2022
- “Help Me Write a Poem: Instruction Tuning As a Vehicle for Collaborative Poetry Writing (CoPoet)”, Chakrabarty et al 2022
- “FLAN: Scaling Instruction-Finetuned Language Models”, Chung et al 2022
- “Language Models Are Multilingual Chain-of-Thought Reasoners”, Shi et al 2022
- “LINGUIST: Language Model Instruction Tuning to Generate Annotated Utterances for Intent Classification and Slot Tagging”, Rosenbaum et al 2022
- “Z-Code++: A Pre-trained Language Model Optimized for Abstractive Summarization”, He et al 2022
- “Few-shot Adaptation Works With UnpredicTable Data”, Chan et al 2022
- “RST: ReStructured Pre-training”, Yuan & Liu 2022
- “InstructDial: Improving Zero and Few-shot Generalization in Dialogue through Instruction Tuning”, Gupta et al 2022
- “CT0: Fine-tuned Language Models Are Continual Learners”, Scialom et al 2022
- “Tk-Instruct: Benchmarking Generalization via In-Context Instructions on 1,600+ Language Tasks”, Wang et al 2022
- “What Language Model Architecture and Pretraining Objective Work Best for Zero-Shot Generalization?”, Wang et al 2022
- “UnifiedQA-v2: Stronger Generalization via Broader Cross-Format Training”, Khashabi et al 2022
- “Reasoning Like Program Executors”, Pi et al 2022
- “ZeroPrompt: Scaling Prompt-Based Pretraining to 1,000 Tasks Improves Zero-Shot Generalization”, Xu et al 2022
- “ExT5: Towards Extreme Multi-Task Scaling for Transfer Learning”, Aribandi et al 2021
- “MetaICL: Learning to Learn In Context”, Min et al 2021
- “T0: Multitask Prompted Training Enables Zero-Shot Task Generalization”, Sanh et al 2021
- “FLAN: Finetuned Language Models Are Zero-Shot Learners”, Wei et al 2021
- “CrossFit: A Few-shot Learning Challenge for Cross-task Generalization in NLP”, Ye et al 2021
- “Cross-Task Generalization via Natural Language Crowdsourcing Instructions”, Mishra et al 2021
- “Muppet: Massive Multi-task Representations With Pre-Finetuning”, Aghajanyan et al 2021
- “UnifiedQA: Crossing Format Boundaries With a Single QA System”, Khashabi et al 2020
- Sort By Magic
- Miscellaneous
- Link Bibliography
See Also
Links
“Can Programming Languages Boost Each Other via Instruction Tuning?”, Zan et al 2023
“Can Programming Languages Boost Each Other via Instruction Tuning?”
“Simple Synthetic Data Reduces Sycophancy in Large Language Models”, Wei et al 2023
“Simple synthetic data reduces sycophancy in large language models”
“DialogStudio: Towards Richest and Most Diverse Unified Dataset Collection for Conversational AI”, Zhang et al 2023
“DialogStudio: Towards Richest and Most Diverse Unified Dataset Collection for Conversational AI”
“LLaMA 2: Open Foundation and Fine-Tuned Chat Models”, Touvron et al 2023
“AlpaGasus: Training A Better Alpaca With Fewer Data”, Chen et al 2023
“Instruction Mining: High-Quality Instruction Data Selection for Large Language Models”, Cao et al 2023
“Instruction Mining: High-Quality Instruction Data Selection for Large Language Models”
“Lost in the Middle: How Language Models Use Long Contexts”, Liu et al 2023
“On the Exploitability of Instruction Tuning”, Shu et al 2023
“ChessGPT: Bridging Policy Learning and Language Modeling”, Feng et al 2023
“Dr. LLaMa: Improving Small Language Models in Domain-Specific QA via Generative Data Augmentation”, Guo et al 2023
“Dr. LLaMa: Improving Small Language Models in Domain-Specific QA via Generative Data Augmentation”
“LaMini-LM: A Diverse Herd of Distilled Models from Large-Scale Instructions”, Wu et al 2023
“LaMini-LM: A Diverse Herd of Distilled Models from Large-Scale Instructions”
“TANGO: Text-to-Audio Generation Using Instruction-Tuned LLM and Latent Diffusion Model”, Ghosal et al 2023
“TANGO: Text-to-Audio Generation using Instruction-Tuned LLM and Latent Diffusion Model”
“Phoenix: Democratizing ChatGPT across Languages”, Chen et al 2023
“How Well Do Large Language Models Perform in Arithmetic Tasks?”, Yuan et al 2023
“How well do Large Language Models perform in Arithmetic tasks?”
“LLaMa: Open and Efficient Foundation Language Models”, Touvron et al 2023
“Med-PaLM: Large Language Models Encode Clinical Knowledge”, Singhal et al 2022
“One Embedder, Any Task: Instruction-Finetuned Text Embeddings (INSTRUCTOR)”, Su et al 2022
“One Embedder, Any Task: Instruction-Finetuned Text Embeddings (INSTRUCTOR)”
“Unnatural Instructions: Tuning Language Models With (Almost) No Human Labor”, Honovich et al 2022
“Unnatural Instructions: Tuning Language Models with (Almost) No Human Labor”
“BLOOMZ/mT0: Crosslingual Generalization through Multitask Finetuning”, Muennighoff et al 2022
“BLOOMZ/mT0: Crosslingual Generalization through Multitask Finetuning”
“Help Me Write a Poem: Instruction Tuning As a Vehicle for Collaborative Poetry Writing (CoPoet)”, Chakrabarty et al 2022
“Help me write a poem: Instruction Tuning as a Vehicle for Collaborative Poetry Writing (CoPoet)”
“FLAN: Scaling Instruction-Finetuned Language Models”, Chung et al 2022
“Language Models Are Multilingual Chain-of-Thought Reasoners”, Shi et al 2022
“Language Models are Multilingual Chain-of-Thought Reasoners”
“LINGUIST: Language Model Instruction Tuning to Generate Annotated Utterances for Intent Classification and Slot Tagging”, Rosenbaum et al 2022
“Z-Code++: A Pre-trained Language Model Optimized for Abstractive Summarization”, He et al 2022
“Z-Code++: A Pre-trained Language Model Optimized for Abstractive Summarization”
“Few-shot Adaptation Works With UnpredicTable Data”, Chan et al 2022
“RST: ReStructured Pre-training”, Yuan & Liu 2022
“InstructDial: Improving Zero and Few-shot Generalization in Dialogue through Instruction Tuning”, Gupta et al 2022
“InstructDial: Improving Zero and Few-shot Generalization in Dialogue through Instruction Tuning”
“CT0: Fine-tuned Language Models Are Continual Learners”, Scialom et al 2022
“Tk-Instruct: Benchmarking Generalization via In-Context Instructions on 1,600+ Language Tasks”, Wang et al 2022
“Tk-Instruct: Benchmarking Generalization via In-Context Instructions on 1,600+ Language Tasks”
“What Language Model Architecture and Pretraining Objective Work Best for Zero-Shot Generalization?”, Wang et al 2022
“What Language Model Architecture and Pretraining Objective Work Best for Zero-Shot Generalization?”
“UnifiedQA-v2: Stronger Generalization via Broader Cross-Format Training”, Khashabi et al 2022
“UnifiedQA-v2: Stronger Generalization via Broader Cross-Format Training”
“Reasoning Like Program Executors”, Pi et al 2022
“ZeroPrompt: Scaling Prompt-Based Pretraining to 1,000 Tasks Improves Zero-Shot Generalization”, Xu et al 2022
“ZeroPrompt: Scaling Prompt-Based Pretraining to 1,000 Tasks Improves Zero-Shot Generalization”
“ExT5: Towards Extreme Multi-Task Scaling for Transfer Learning”, Aribandi et al 2021
“ExT5: Towards Extreme Multi-Task Scaling for Transfer Learning”
“MetaICL: Learning to Learn In Context”, Min et al 2021
“T0: Multitask Prompted Training Enables Zero-Shot Task Generalization”, Sanh et al 2021
“T0: Multitask Prompted Training Enables Zero-Shot Task Generalization”
“FLAN: Finetuned Language Models Are Zero-Shot Learners”, Wei et al 2021
“CrossFit: A Few-shot Learning Challenge for Cross-task Generalization in NLP”, Ye et al 2021
“CrossFit: A Few-shot Learning Challenge for Cross-task Generalization in NLP”
“Cross-Task Generalization via Natural Language Crowdsourcing Instructions”, Mishra et al 2021
“Cross-Task Generalization via Natural Language Crowdsourcing Instructions”
“Muppet: Massive Multi-task Representations With Pre-Finetuning”, Aghajanyan et al 2021
“Muppet: Massive Multi-task Representations with Pre-Finetuning”
“UnifiedQA: Crossing Format Boundaries With a Single QA System”, Khashabi et al 2020
“UnifiedQA: Crossing Format Boundaries With a Single QA System”
Sort By Magic
Annotations sorted by machine learning into inferred 'tags'. This provides an alternative way to browse: instead of by date order, one can browse in topic order. The 'sorted' list has been automatically clustered into multiple sections & auto-labeled for easier browsing.
Beginning with the newest annotation, it uses the embedding of each annotation to attempt to create a list of nearest-neighbor annotations, creating a progression of topics. For more details, see the link.
language-models
3.
language-models
generalization
Miscellaneous
-
https://blog.research.google/2021/10/introducing-flan-more-generalizable.html -
https://github.com/bigscience-workshop/architecture-objective -
https://github.com/google-research/t5x/blob/main/docs/models.md#flan-t5-checkpoints -
https://www.databricks.com/blog/2023/03/24/hello-dolly-democratizing-magic-chatgpt-open-models.html
Link Bibliography
-
https://arxiv.org/abs/2308.03958#deepmind: “Simple Synthetic Data Reduces Sycophancy in Large Language Models”, Jerry Wei, Da Huang, Yifeng Lu, Denny Zhou, Quoc V. Le -
https://arxiv.org/abs/2307.08701#samsung: “AlpaGasus: Training A Better Alpaca With Fewer Data”, -
https://arxiv.org/abs/2305.07804: “Dr. LLaMa: Improving Small Language Models in Domain-Specific QA via Generative Data Augmentation”, Zhen Guo, Peiqi Wang, Yanwei Wang, Shangdi Yu -
https://arxiv.org/abs/2304.13731: “TANGO: Text-to-Audio Generation Using Instruction-Tuned LLM and Latent Diffusion Model”, Deepanway Ghosal, Navonil Majumder, Ambuj Mehrish, Soujanya Poria -
https://arxiv.org/abs/2304.02015#alibaba: “How Well Do Large Language Models Perform in Arithmetic Tasks?”, Zheng Yuan, Hongyi Yuan, Chuanqi Tan, Wei Wang, Songfang Huang -
https://arxiv.org/abs/2212.13138#google: “Med-PaLM: Large Language Models Encode Clinical Knowledge”, -
https://arxiv.org/abs/2212.09741: “One Embedder, Any Task: Instruction-Finetuned Text Embeddings (INSTRUCTOR)”, -
https://arxiv.org/abs/2210.13669: “Help Me Write a Poem: Instruction Tuning As a Vehicle for Collaborative Poetry Writing (CoPoet)”, Tuhin Chakrabarty, Vishakh Padmakumar, He He -
https://arxiv.org/abs/2210.11416#google: “FLAN: Scaling Instruction-Finetuned Language Models”, -
https://arxiv.org/abs/2210.03057#google: “Language Models Are Multilingual Chain-of-Thought Reasoners”, -
https://arxiv.org/abs/2208.09770#microsoft: “Z-Code++: A Pre-trained Language Model Optimized for Abstractive Summarization”, -
https://arxiv.org/abs/2205.12393: “CT0: Fine-tuned Language Models Are Continual Learners”, Thomas Scialom, Tuhin Chakrabarty, Smaranda Muresan -
https://arxiv.org/abs/2204.07705: “T<em>k</em>-Instruct: Benchmarking Generalization via In-Context Instructions on 1,600+ Language Tasks”, -
https://arxiv.org/abs/2201.11473#microsoft: “Reasoning Like Program Executors”, Xinyu Pi, Qian Liu, Bei Chen, Morteza Ziyadi, Zeqi Lin, Yan Gao, Qiang Fu, Jian-Guang Lou, Weizhu Chen -
https://arxiv.org/abs/2201.06910: “ZeroPrompt: Scaling Prompt-Based Pretraining to 1,000 Tasks Improves Zero-Shot Generalization”, Hanwei Xu, Yujun Chen, Yulun Du, Nan Shao, Yanggang Wang, Haiyu Li, Zhilin Yang