Differences

This shows you the differences between two versions of the page.

--- ml:ml_overview [2021/06/24 19:25] – jmflanig
+++ ml:ml_overview [2024/11/27 21:56] (current) – [Machine Learning Overview] jmflanig
@@ Line 3: / Line 3: @@
 This is a resource to help you get up to speed in various topics if you're trying to learn ML on your own or broaden your ML knowledge.
+See also [[https://aman.ai/primers/ai/|Aman.ai - AI Fundamentals]]
 ===== Books =====
   * Pattern Recognition and Machine Learning, Bishop, 2006 (Referenced below as Bishop) available [[https://www.microsoft.com/en-us/research/uploads/prod/2006/01/Bishop-Pattern-Recognition-and-Machine-Learning-2006.pdf|here]] or [[https://jlab.soe.ucsc.edu/nlp-wiki/lib/exe/fetch.php?media=book:bishop.pdf|local copy]]
-  * An Introduction to Statistical Learning (Reference below as ISL) available here or [[https://jlab.soe.ucsc.edu/nlp-wiki/lib/exe/fetch.php?media=book:islr-7th.pdf|local copy]]
+  * An Introduction to Statistical Learning (Reference below as ISL) available [[https://www.statlearning.com/|here]] or [[https://jlab.soe.ucsc.edu/nlp-wiki/lib/exe/fetch.php?media=book:islr-7th.pdf|local copy]]
   * [[https://web.stanford.edu/~hastie/ElemStatLearn/|The Elements of Statistical Learning: Data Mining, Inference, and Prediction. 2009]] (Referenced below as ESL) available [[https://web.stanford.edu/~hastie/ElemStatLearn/printings/ESLII_print12_toc.pdf|here]] or [[https://jlab.soe.ucsc.edu/nlp-wiki/lib/exe/fetch.php?media=book:eslii_print12.pdf|local copy]]
   * [[http://ciml.info/|CIML]] (Referenced below as CIML) available [[http://ciml.info/dl/v0_99/ciml-v0_99-all.pdf|here]]
@@ Line 12: / Line 14: @@
     * Machine Learning A Probabilistic Perspective, 2012 (referenced below as Murphy) available [[http://noiselab.ucsd.edu/ECE228/Murphy_Machine_Learning.pdf|here]]
     * Probabilistic Machine Learning: An Introduction, 2021 (reference below as PML1) available [[https://github.com/probml/pml-book/releases/latest/download/book1.pdf|here]] or [[https://jlab.soe.ucsc.edu/nlp-wiki/lib/exe/fetch.php?media=book:pml-book1.pdf|local copy]]
-  * [[https://www.deeplearningbook.org/contents/convnets.html|Deep Learning Book]] (Referenced below as DLBook)
+  * [[https://www.deeplearningbook.org|Deep Learning Book]] (Referenced below as DLBook)
   * [[https://www.jair.org/index.php/jair/article/view/11030|A Primer on Neural Network Models for Natural Language Processing, Yoav Goldberg, 2016]] [[https://jlab.soe.ucsc.edu/nlp-wiki/lib/exe/fetch.php?media=book:nn_primer.pdf|local copy]] (Referenced below as NNPrimer)
   * The Matrix Cookbook, available [[https://svivek.com/teaching/deep-learning-nlp/spring2019/resources/matrixcookbook.pdf|here]]
+  * [[https://www.eleuther.ai/beginners.pdf|Intro to ML]] (note this was released April 1st)
 ===== Courses =====
   * [[https://www.cs.cornell.edu/courses/cs4780/2018fa/lectures/|Machine Learning for Intelligent Systems @ Cornell]]
+  * Very quick intro to machine learning (slides): [[https://indico.physics.lbl.gov/event/569/contributions/1382/attachments/1268/1407/gs-20170913.pdf|Introduction to Machine Learning]] {{papers:quick-ml-intro.pdf|local copy}}
 ===== Overview of Topics =====
-This overview contains links to particular pages in textbooks, lectures, blog posts, and videos covering the topic, listed easiest to hardest to understand, with videos listed at the end.  In other words, for each topic, introductory material is listed first with more advanced material afterwards, although you may find more advanced material easier to understand in some cases. **//The blog posts and some of the videos are introductory and give the overall gist of the method, but may contain mathematical or conceptual errors. Videos that are lectures should be fine.//**
+This overview contains links to particular pages in textbooks, lectures, blog posts, and videos covering the topic, listed easiest to hardest to understand, with videos listed at the end.  In other words, for each topic, introductory material is listed first with more advanced material afterwards, although you may find more advanced material easier to understand in some cases.
+**//The blog posts and some of the videos are introductory and give the overall gist of the method, but may contain mathematical or conceptual errors. Videos that are lectures should be fine.//**
   * **Introduction to Machine Learning** [[https://www.cin.ufpe.br/~cavmj/Machine%20-%20Learning%20-%20Tom%20Mitchell.pdf#page=13|MLBook p. 1-15]] [[https://jlab.soe.ucsc.edu/nlp-wiki/lib/exe/fetch.php?media=book:pml-book1.pdf#page=29|PML p. 1-28]]
     * "Machine Learning is the field of study that gives computers the ability to learn without being explicitly programmed." [[https://en.wikipedia.org/wiki/Arthur_Samuel|Arthur Samuel]] [[https://citeseerx.ist.psu.edu/viewdoc/download;jsessionid=45FE379DC2BFEA630F406F16589305D1?doi=10.1.1.368.2254&rep=rep1&type=pdf|1959]]
   * **Basic Machine Learning Concepts**
-    * **Inductive Bias**
+    * **Inductive Bias** [[https://www.cin.ufpe.br/~cavmj/Machine%20-%20Learning%20-%20Tom%20Mitchell.pdf#page=51|MLBook p. 39-45]]
     * **Overfitting/Underfitting**
-    * **Approximation error vs estimation error** [[https://people.eecs.berkeley.edu/~bartlett/courses/281b-sp08/20.pdf|Bartlett notes]] Sometimes also intuitively called the bias-variance tradeoff.
+    * **Approximation error vs estimation error aka Bias-Variance Tradeoff** [[http://ciml.info/dl/v0_99/ciml-v0_99-all.pdf#page=71|CIML p. 71-72]] [[https://people.eecs.berkeley.edu/~bartlett/courses/281b-sp08/20.pdf|Bartlett notes]] Sometimes also called the bias-variance tradeoff.
     * **Features**
     * **Hyperparameters**
@@ Line 56: / Line 62: @@
   * **Regression**
       * ** *Linear Regression** [[https://machinelearningmastery.com/linear-regression-for-machine-learning/|Blog1]] [[https://ml-cheatsheet.readthedocs.io/en/latest/linear_regression.html|Blog2]] [[https://jlab.soe.ucsc.edu/nlp-wiki/lib/exe/fetch.php?media=book:eslii_print12.pdf#page=62|ESL p. 43-51]] [[https://jlab.soe.ucsc.edu/nlp-wiki/lib/exe/fetch.php?media=book:bishop.pdf#page=157|Bishop p. 137-147]] [[https://jlab.soe.ucsc.edu/nlp-wiki/lib/exe/fetch.php?media=book:islr-7th.pdf#page=68|ISL p. 59-119]]
+        * MAP vs MLE linear regression (MAP adds a regularizer term)
+        * Bayesian linear regression
       * **"Non-Linear" Regression**
         * **Polynomial Regression** [[https://jlab.soe.ucsc.edu/nlp-wiki/lib/exe/fetch.php?media=book:islr-7th.pdf#page=273|ISL p. 266-268]]
@@ Line 75: / Line 83: @@
       * **Tests of Significance** [[http://ciml.info/dl/v0_99/ciml-v0_99-all.pdf#page=67|CIML p. 67-69]]
     * **Data Resampling Methods**
-      * **k-fold Cross Validation**
+      * **k-fold Cross Validation** Be careful using this method on NLP datasets! Due to the non-IID nature of NLP datasets, it is generally not recommended to use k-fold cross validation (can over-estimate performance).  Better to use a thoughtfully-chosen train/dev/test split.
       * ** *Bootstrap Resampling**
       * **Jacknife**
@@ Line 162: / Line 170: @@
     * **SpaCy**
     * **OpenCV**
+===== Related Pages =====
+  * [[Deep Learning]]
+  * [[ML Glossary]] (glossary of slightly more advanced terms)