NLP Wiki

This page is read only. You can view the source, but not change it. Ask your administrator if you think this is wrong.
====== Meta-Learning ======

===== Overviews =====
  * [[https://arxiv.org/pdf/2004.05439.pdf|Hospedales et al 2020 - Meta-Learning in Neural Networks: A Survey]]

===== AutoML =====
  * [[https://arxiv.org/pdf/1908.00709.pdf|2019 - AutoML: A Survey of the State-of-the-Art]]
  * [[https://arxiv.org/pdf/1904.12054.pdf|2019 - Benchmark and Survey of Automated Machine Learning Frameworks]]
  * [[https://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=9074859&casa_token=NTS3U5gh_gwAAAAA:CsB8trekGncSuh5yvHHKFd7XLoCBID8ElEUOf6KkfklHRh9wtllN_3tecjHfIJCZ9rn1qyhS&tag=1|2020 - Automated Machine Learning: The New Wave of Machine Learning]]

===== Deep Learning Papers =====
  * [[https://jlab.soe.ucsc.edu/nlp-wiki/lib/exe/fetch.php?media=papers:learning_to_learn_using_gradient_descent.pdf|Hochreiter & Younger 2001 - Learning to Learn Using Gradient Descent]] Amazing paper from one of the inventors of LSTMs.  Jeurgen talks about it [[https://people.idsia.ch/~juergen/lstm/sld039.htm|here]].
  * [[https://arxiv.org/pdf/1606.04474.pdf|Andrychowicz et al 2016 - Learning to Learn by Gradient Descent by Gradient Descent]]
  * [[https://arxiv.org/pdf/1609.09106.pdf|Ha et al 2016 - HyperNetworks]]
  * [[https://openreview.net/pdf?id=rJY0-Kcll|Ravi & Larochelle 2016 - Optimization as a Model for Few-Shot Learning]] Proposes "an LSTM-based meta-learner model to learn the exact optimization algorithm used to train another learner neural network classifier in the few-shot regime."
  * [[http://proceedings.mlr.press/v70/chen17e/chen17e.pdf|Chen 2017 - Learning to Learn without Gradient Descent by Gradient Descent]] Learns a black-box optimizer (gradient-free optimizer).  Can be applied to hyperparameter tuning.
  * MAML: [[https://arxiv.org/pdf/1703.03400.pdf|Finn et al 2017 - Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks]]
  * [[http://proceedings.mlr.press/v97/finn19a/finn19a.pdf|Finn et al 2019 - Online Meta-Learning]] Introduces online meta-learning, where the learner sees a series of tasks. Proposes Follow The Meta-Leader (FTML), a follow-the-leader-type meta-learning algorithm, which extend MAML to this setting.
  * [[https://arxiv.org/pdf/2012.14905.pdf|Kirsch & Schmidhuber 2021 - Meta Learning Backpropagation And Improving It]]
  * [[https://arxiv.org/pdf/2201.04182.pdf|Zhmoginov et al 2022 - HyperTransformer: Model Generation for Supervised and Semi-Supervised Few-Shot Learning]] ([[https://www.youtube.com/watch?v=D6osiiEoV0w&ab_channel=YannicKilcher|interview]])

===== Non-Deep Learning Papers =====
  * **[[https://arxiv.org/pdf/2003.03384.pdf|Real et al 2020 - AutoML-Zero: Evolving Machine Learning Algorithms From Scratch]]**
  * [[https://arxiv.org/pdf/2101.08809.pdf|Peng et al 2021 - PyGlove: Symbolic Programming for Automated Machine Learning]]

===== Meta-Learning in NLP =====
  * [[https://arxiv.org/pdf/1911.03863.pdf|Bansal et al 2019 - Learning to Few-Shot Learn Across Diverse Natural Language Classification Tasks]]
  * [[https://aclanthology.org/2021.naacl-main.88.pdf|Murty et al 2021 - DReCa: A General Task Augmentation Strategy for Few-Shot Natural Language Inference]]
  * [[https://arxiv.org/pdf/2111.01322.pdf|Bansal et al 2021 - Diverse Distributions of Self-Supervised Tasks for Meta-Learning in NLP]]

===== Related Pages =====
  * [[application_optimization|Application: Optimization]]
  * [[Multi-Task Learning]]
  * [[Neural Architecture Search]]
  * [[nlp:prompting|Prompting and In-Context Learning]]