https://jlab.soe.ucsc.edu/nlp-wiki/ 2026-05-10T20:10:42+00:00 https://jlab.soe.ucsc.edu/nlp-wiki/ https://jlab.soe.ucsc.edu/nlp-wiki/lib/tpl/dokuwiki/images/favicon.ico text/html 2023-06-15T07:36:14+00:00 Anonymous (anonymous@undisclosed.example.com) https://jlab.soe.ucsc.edu/nlp-wiki/doku.php?id=ml:theory:binary_classification&rev=1686814574&do=diff Machine Learning Theory: Binary Classification For an overview of the theory of binary classification, see Neural Network Theory Theoretical Foundations, Foundations of Machine Learning, or Understanding Machine Learning From Theory to Algorithms. PAC Learning Theory Overview of results in PAC learning theory (from the introduction of The sample complexity of agnostic learning under deterministic labels): And later from the related work: text/html 2025-05-29T07:00:03+00:00 Anonymous (anonymous@undisclosed.example.com) https://jlab.soe.ucsc.edu/nlp-wiki/doku.php?id=ml:theory:generalization_in_deep_learning&rev=1748502003&do=diff Generalization in Deep Learning The theory of generalization in deep learning is not well understood, and is an active area of research. Overviews * Lil'log - Are Deep Neural Networks Dramatically Overfitted? Good summary from 2019. * Overview Papers * He and Tao 2022 - Recent Advances in Deep Learning Theory * Textbooks * Roberts & Yaida 2021 - The Principles of Deep Learning Theory: An Effective Theory Approach to Understanding Neural Networks Key Papers * Zhang et al 201… text/html 2023-06-15T07:36:14+00:00 Anonymous (anonymous@undisclosed.example.com) https://jlab.soe.ucsc.edu/nlp-wiki/doku.php?id=ml:theory:learning_curves&rev=1686814574&do=diff Learning Curves This should be merged with Generalization in Deep Learning and Binary Classification. Papers * Hutter 2021 - Learning Curve Theory * Bisla et al 2021 - A Theoretical-Empirical Approach to Estimating Sample Complexity of DNNs (Found by searching “empirical sample complexity” on Google Scholar) text/html 2024-03-06T21:46:50+00:00 Anonymous (anonymous@undisclosed.example.com) https://jlab.soe.ucsc.edu/nlp-wiki/doku.php?id=ml:theory:multi-armed_bandit&rev=1709761610&do=diff Multi-Armed Bandits See Wikipedia - Multi-armed Bandit. Surveys * Bubeck & Cesa-Bianchi 2012 - Regret Analysis of Stochastic and Nonstochastic Multi-armed Bandit Problems Very good survey Theory * Mei et al 2023 - Stochastic Gradient Succeeds for Bandits Related Pages * Reinforcement Learning * Online Learning * Regret Bounds text/html 2023-06-15T07:36:14+00:00 Anonymous (anonymous@undisclosed.example.com) https://jlab.soe.ucsc.edu/nlp-wiki/doku.php?id=ml:theory:regret_bounds&rev=1686814574&do=diff Theory: Online Learning and Regret Bounds Online Learning Surveys and Theses * Shalev-Shwartz 2007 - Online Learning: Theory, Algorithms, and Applications See section 2.4 (page 27 in pdf) for historical references * Battou - Online Learning and Stochastic Approximations Key Papers * Zinkevich 2003 - Online Convex Programming and Generalized Infinitesimal Gradient Ascent See also the CMU tech report Regret Bounds Regret bounds are widely used for proving generalization bounds for on…