This page is about natural language task descriptions and prompting language models. Both these methods use natural language to describe a task which is to be performed.

• Zero-shot
  • Radford et al 2019 - Language Models Are Unsupervised Multitask Learners GPT-2
• Few-shot
  • Schick & Schütze 2020 - It's Not Just Size That Matters: Small Language Models Are Also Few-Shot Learners
  • Schick & Schütze 2021 - Exploiting Cloze Questions for Few Shot Text Classification and Natural Language Inference Pre-dates GTP-3
  • Brown et al 2021 - Language Models are Few-Shot Learners GPT-3
  • Gao et al 2021 - Making Pre-trained Language Models Better Few-shot Learners
• Soft-Prompting
  • Qin & Eisner 2021 - Learning How to Ask: Querying LMs with Mixtures of Soft Prompts
• More papers
  • Scao & Rush 2021 - How Many Data Points is a Prompt Worth? Prompts are very helpful in small data regimes, and are worth 100's of datapoints.

• Goldwasser & Roth 2014 - Learning From Natural Instructions
• Mishra et al 2021 - Natural-Instructions: Benchmarking Generalization to New Tasks from Natural Language Instructions

• Invited Talk @ NAACL 2021: Humans Learn From Task Descriptions and So Should Our Models - Hinrich Schütze

• Timo Schick

• Few-Shot Learning
• Zero-Shot Learning