Differences

This shows you the differences between two versions of the page.

--- ml:gradient_clipping [2021/03/09 18:43] – created jmflanig
+++ ml:gradient_clipping [2023/06/15 07:36] (current) – external edit 127.0.0.1
@@ Line 3: / Line 3: @@
 ===== Papers =====
-  * 2020 - Why gradient clipping accelerates training: A theoretical justification for adaptivity
+  * [[https://arxiv.org/pdf/1905.11881.pdf|Zhang et al 2020 - Why Gradient Clipping Accelerates Training: A Theoretical Justification for Adaptivity]]
+  * An extreme form of gradient clipping, where everything gets clipped, is the "Manhattan-Learning rule" (see the {{papers:rprop_paper.pdf|Rprop paper}}).  Rprop is an advancement over this.
 ===== Blog Posts =====
   * [[https://towardsdatascience.com/what-is-gradient-clipping-b8e815cdfb48|What is gradient clipping]]
+===== Related Pages =====
+  * [[Optimizers]]