instruct-tuning LM tag

N/A

[Warning: JS Disabled!]

[For support of website features (tablesorting, collapsible sections, image zooming, link annotation popups/popins & transclusions, sidenotes), please enable JavaScript.]

See Also
Links
Miscellaneous
Link Bibliography

Links

“Med-PaLM: Large Language Models Encode Clinical Knowledge”, Singhal Et Al 2022

“Med-PaLM: Large Language Models Encode Clinical Knowledge”⁠, Karan Singhal, Shekoofeh Azizi, Tao Tu, S. Sara Mahdavi, Jason Wei, Hyung Won Chung, Nathan Scales, Ajay Tanwani2022-12-26 (ML dataset⁠, PaLM⁠, biology; similar; bibliography)

“One Embedder, Any Task: Instruction-Finetuned Text Embeddings (INSTRUCTOR)”, Su Et Al 2022

“One Embedder, Any Task: Instruction-Finetuned Text Embeddings (INSTRUCTOR)”⁠, Hongjin Su, Weijia Shi, Jungo Kasai, Yizhong Wang, Yushi Hu, Mari Ostendorf, Wen-tau Yih, Noah A. Smith2022-12-19 (retrieval AI⁠, T5 Transformer⁠, AI scaling; similar; bibliography)

“Unnatural Instructions: Tuning Language Models With (Almost) No Human Labor”, Honovich Et Al 2022

“Unnatural Instructions: Tuning Language Models with (Almost) No Human Labor”⁠, Or Honovich, Thomas Scialom, Omer Levy, Timo Schick2022-12-19 (ML dataset⁠, T5 Transformer⁠, meta-learning; similar)

“BLOOMZ/mT0: Crosslingual Generalization through Multitask Finetuning”, Muennighoff Et Al 2022

“BLOOMZ/mT0: Crosslingual Generalization through Multitask Finetuning”⁠, Niklas Muennighoff, Thomas Wang, Lintang Sutawika, Adam Roberts, Stella Biderman, Teven Le Scao, M. Saiful Bari2022-11-03 (ML dataset⁠, T5 Transformer⁠, meta-learning; similar)

“Help Me Write a Poem: Instruction Tuning As a Vehicle for Collaborative Poetry Writing (CoPoet)”, Chakrabarty Et Al 2022

“Help me write a poem: Instruction Tuning as a Vehicle for Collaborative Poetry Writing (CoPoet)”⁠, Tuhin Chakrabarty, Vishakh Padmakumar, He He2022-10-25 (NN sampling⁠, LM tokenization⁠, GPT poetry⁠, T5 Transformer; backlinks; similar; bibliography)

“FLAN: Scaling Instruction-Finetuned Language Models”, Chung Et Al 2022

“FLAN: Scaling Instruction-Finetuned Language Models”⁠, Hyung Won Chung, Le Hou, Shayne Longpre, Barret Zoph, Yi Tay, William Fedus, Yunxuan Li, Xuezhi Wang2022-10-20 (PaLM⁠, T5 Transformer⁠, AI scaling; similar; bibliography)

“Language Models Are Multilingual Chain-of-Thought Reasoners”, Shi Et Al 2022

“Language Models are Multilingual Chain-of-Thought Reasoners”⁠, Freda Shi, Mirac Suzgun, Markus Freitag, Xuezhi Wang, Suraj Srivats, Soroush Vosoughi, Hyung Won Chung2022-10-06 (inner monologue (AI)⁠, PaLM⁠, emergence (AI); similar; bibliography)

“LINGUIST: Language Model Instruction Tuning to Generate Annotated Utterances for Intent Classification and Slot Tagging”, Rosenbaum Et Al 2022

“LINGUIST: Language Model Instruction Tuning to Generate Annotated Utterances for Intent Classification and Slot Tagging”⁠, Andy Rosenbaum, Saleh Soltan, Wael Hamza, Yannick Versley, Markus Boese2022-09-20 (similar)

“Few-shot Adaptation Works With UnpredicTable Data”, Chan Et Al 2022

“Few-shot Adaptation Works with UnpredicTable Data”⁠, Jun Shern Chan, Michael Pieler, Jonathan Jao, Jérémy Scheurer, Ethan Perez2022-08-01 (ML dataset⁠, meta-learning; similar)

“RST: ReStructured Pre-training”, Yuan & Liu2022

“RST: reStructured Pre-training”⁠, Weizhe Yuan, Pengfei Liu2022-06-22 (ML dataset⁠, T5 Transformer⁠, AI scaling; similar)

“InstructDial: Improving Zero and Few-shot Generalization in Dialogue through Instruction Tuning”, Gupta Et Al 2022

“InstructDial: Improving Zero and Few-shot Generalization in Dialogue through Instruction Tuning”⁠, Prakhar Gupta, Cathy Jiao, Yi-Ting Yeh, Shikib Mehri, Maxine Eskenazi, Jeffrey P. Bigham2022-05-25 (ML dataset⁠, AI scaling; similar)

“TK-Instruct: Benchmarking Generalization via In-Context Instructions on 1,600+ Language Tasks”, Wang Et Al 2022

“Tk-Instruct: Benchmarking Generalization via In-Context Instructions on 1,600+ Language Tasks”⁠, Yizhong Wang, Swaroop Mishra, Pegah Alipoormolabashi, Yeganeh Kordi, Amirreza Mirzaei, Anjana Arunkumar2022-04-16 (ML dataset⁠, T5 Transformer⁠, meta-learning; backlinks; similar; bibliography)

“What Language Model Architecture and Pretraining Objective Work Best for Zero-Shot Generalization?”, Wang Et Al 2022

“What Language Model Architecture and Pretraining Objective Work Best for Zero-Shot Generalization?”⁠, Thomas Wang, Adam Roberts, Daniel Hesslow, Teven Le Scao, Hyung Won Chung, Iz Beltagy, Julien Launay2022-04-12 (T5 Transformer⁠, AI scaling⁠, meta-learning; backlinks; similar)

“Reasoning Like Program Executors”, Pi Et Al 2022

“Reasoning Like Program Executors”⁠, Xinyu Pi, Qian Liu, Bei Chen, Morteza Ziyadi, Zeqi Lin, Yan Gao, Qiang Fu, Jian-Guang Lou, Weizhu Chen2022-01-27 (inner monologue (AI)⁠, T5 Transformer⁠, AI scaling; similar; bibliography)

“ZeroPrompt: Scaling Prompt-Based Pretraining to 1,000 Tasks Improves Zero-Shot Generalization”, Xu Et Al 2022

“ZeroPrompt: Scaling Prompt-Based Pretraining to 1,000 Tasks Improves Zero-Shot Generalization”⁠, Hanwei Xu, Yujun Chen, Yulun Du, Nan Shao, Yanggang Wang, Haiyu Li, Zhilin Yang2022-01-18 (AI scaling; backlinks; similar; bibliography)

“ExT5: Towards Extreme Multi-Task Scaling for Transfer Learning”, Aribandi Et Al 2021

“ExT5: Towards Extreme Multi-Task Scaling for Transfer Learning”⁠, Vamsi Aribandi, Yi Tay, Tal Schuster, Jinfeng Rao, Huaixiu Steven Zheng, Sanket Vaibhav Mehta, Honglei Zhuang2021-11-22 (T5 Transformer⁠, AI scaling; similar)

“MetaICL: Learning to Learn In Context”, Min Et Al 2021

“MetaICL: Learning to Learn In Context”⁠, Sewon Min, Mike Lewis, Luke Zettlemoyer, Hannaneh Hajishirzi2021-10-29 (meta-learning⁠, RL scaling; similar)

“T0: Multitask Prompted Training Enables Zero-Shot Task Generalization”, Sanh Et Al 2021

“T0: Multitask Prompted Training Enables Zero-Shot Task Generalization”⁠, Victor Sanh, Albert Webson, Colin Raffel, Stephen H. Bach, Lintang Sutawika, Zaid Alyafeai, Antoine Chaffin2021-10-15 (ML dataset⁠, T5 Transformer⁠, meta-learning⁠, RL scaling; backlinks; similar)

“FLAN: Finetuned Language Models Are Zero-Shot Learners”, Wei Et Al 2021

“FLAN: Finetuned Language Models Are Zero-Shot Learners”⁠, Jason Wei, Maarten Bosma, Vincent Y. Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, Andrew M. Dai2021-09-03 (LaMDA⁠, meta-learning⁠, RL scaling; similar)

“CrossFit: A Few-shot Learning Challenge for Cross-task Generalization in NLP”, Ye Et Al 2021

“CrossFit: A Few-shot Learning Challenge for Cross-task Generalization in NLP”⁠, Qinyuan Ye, Bill Yuchen Lin, Xiang Ren2021-04-18 (AI scaling⁠, meta-learning; similar)

“Cross-Task Generalization via Natural Language Crowdsourcing Instructions”, Mishra Et Al 2021

“Cross-Task Generalization via Natural Language Crowdsourcing Instructions”⁠, Swaroop Mishra, Daniel Khashabi, Chitta Baral, Hannaneh Hajishirzi2021-04-18 (backlinks; similar)

“Muppet: Massive Multi-task Representations With Pre-Finetuning”, Aghajanyan Et Al 2021

“Muppet: Massive Multi-task Representations with Pre-Finetuning”⁠, Armen Aghajanyan, Anchit Gupta, Akshat Shrivastava, Xilun Chen, Luke Zettlemoyer, Sonal Gupta2021-01-26 (AI scaling; similar)

Miscellaneous

Link Bibliography

https://arxiv.org/abs/2212.13138#google: “Med-PaLM: Large Language Models Encode Clinical Knowledge”⁠, Karan Singhal, Shekoofeh Azizi, Tao Tu, S. Sara Mahdavi, Jason Wei, Hyung Won Chung, Nathan Scales, Ajay Tanwani:

link-bibliography
https://arxiv.org/abs/2212.09741: “One Embedder, Any Task: Instruction-Finetuned Text Embeddings (INSTRUCTOR)”⁠, Hongjin Su, Weijia Shi, Jungo Kasai, Yizhong Wang, Yushi Hu, Mari Ostendorf, Wen-tau Yih, Noah A. Smith:

link-bibliography
https://arxiv.org/abs/2210.13669: “Help Me Write a Poem: Instruction Tuning As a Vehicle for Collaborative Poetry Writing (CoPoet)”⁠, Tuhin Chakrabarty, Vishakh Padmakumar, He He:

link-bibliography
https://arxiv.org/abs/2210.11416#google: “FLAN: Scaling Instruction-Finetuned Language Models”⁠, Hyung Won Chung, Le Hou, Shayne Longpre, Barret Zoph, Yi Tay, William Fedus, Yunxuan Li, Xuezhi Wang:

link-bibliography
https://arxiv.org/abs/2210.03057#google: “Language Models Are Multilingual Chain-of-Thought Reasoners”⁠, Freda Shi, Mirac Suzgun, Markus Freitag, Xuezhi Wang, Suraj Srivats, Soroush Vosoughi, Hyung Won Chung:

link-bibliography
https://arxiv.org/abs/2204.07705: “T<em>k< / em>-Instruct: Benchmarking Generalization via In-Context Instructions on 1,600+ Language Tasks”⁠, Yizhong Wang, Swaroop Mishra, Pegah Alipoormolabashi, Yeganeh Kordi, Amirreza Mirzaei, Anjana Arunkumar:

link-bibliography
https://arxiv.org/abs/2201.11473#microsoft: “Reasoning Like Program Executors”⁠, Xinyu Pi, Qian Liu, Bei Chen, Morteza Ziyadi, Zeqi Lin, Yan Gao, Qiang Fu, Jian-Guang Lou, Weizhu Chen:

link-bibliography
https://arxiv.org/abs/2201.06910: “ZeroPrompt: Scaling Prompt-Based Pretraining to 1,000 Tasks Improves Zero-Shot Generalization”⁠, Hanwei Xu, Yujun Chen, Yulun Du, Nan Shao, Yanggang Wang, Haiyu Li, Zhilin Yang:

link-bibliography

[Annotation Of The Day]