Differences

This shows you the differences between two versions of the page.

--- ml:meta-learning [2022/02/24 01:03] – [Deep Learning Papers] jmflanig
+++ ml:meta-learning [2023/11/09 19:44] (current) – [Related Pages] jmflanig
@@ Line 13: / Line 13: @@
   * [[https://arxiv.org/pdf/1606.04474.pdf|Andrychowicz et al 2016 - Learning to Learn by Gradient Descent by Gradient Descent]]
   * [[https://arxiv.org/pdf/1609.09106.pdf|Ha et al 2016 - HyperNetworks]]
+  * [[https://openreview.net/pdf?id=rJY0-Kcll|Ravi & Larochelle 2016 - Optimization as a Model for Few-Shot Learning]] Proposes "an LSTM-based meta-learner model to learn the exact optimization algorithm used to train another learner neural network classifier in the few-shot regime."
   * [[http://proceedings.mlr.press/v70/chen17e/chen17e.pdf|Chen 2017 - Learning to Learn without Gradient Descent by Gradient Descent]] Learns a black-box optimizer (gradient-free optimizer).  Can be applied to hyperparameter tuning.
+  * MAML: [[https://arxiv.org/pdf/1703.03400.pdf|Finn et al 2017 - Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks]]
+  * [[http://proceedings.mlr.press/v97/finn19a/finn19a.pdf|Finn et al 2019 - Online Meta-Learning]] Introduces online meta-learning, where the learner sees a series of tasks. Proposes Follow The Meta-Leader (FTML), a follow-the-leader-type meta-learning algorithm, which extend MAML to this setting.
   * [[https://arxiv.org/pdf/2012.14905.pdf|Kirsch & Schmidhuber 2021 - Meta Learning Backpropagation And Improving It]]
-  * [[https://arxiv.org/pdf/2201.04182.pdf|Zhmoginov et al 2022 - HyperTransformer: Model Generation for Supervised and Semi-Supervised Few-Shot Learning]]
+  * [[https://arxiv.org/pdf/2201.04182.pdf|Zhmoginov et al 2022 - HyperTransformer: Model Generation for Supervised and Semi-Supervised Few-Shot Learning]] ([[https://www.youtube.com/watch?v=D6osiiEoV0w&ab_channel=YannicKilcher|interview]])
 ===== Non-Deep Learning Papers =====
   * **[[https://arxiv.org/pdf/2003.03384.pdf|Real et al 2020 - AutoML-Zero: Evolving Machine Learning Algorithms From Scratch]]**
   * [[https://arxiv.org/pdf/2101.08809.pdf|Peng et al 2021 - PyGlove: Symbolic Programming for Automated Machine Learning]]
+===== Meta-Learning in NLP =====
+  * [[https://arxiv.org/pdf/1911.03863.pdf|Bansal et al 2019 - Learning to Few-Shot Learn Across Diverse Natural Language Classification Tasks]]
+  * [[https://aclanthology.org/2021.naacl-main.88.pdf|Murty et al 2021 - DReCa: A General Task Augmentation Strategy for Few-Shot Natural Language Inference]]
+  * [[https://arxiv.org/pdf/2111.01322.pdf|Bansal et al 2021 - Diverse Distributions of Self-Supervised Tasks for Meta-Learning in NLP]]
 ===== Related Pages =====
+  * [[application_optimization|Application: Optimization]]
+  * [[Multi-Task Learning]]
   * [[Neural Architecture Search]]
+  * [[nlp:prompting|Prompting and In-Context Learning]]