Differences

This shows you the differences between two versions of the page.

--- ml:normalization [2021/03/12 05:43] – [Layer Normalization] jmflanig
+++ ml:normalization [2023/06/15 07:36] (current) – external edit 127.0.0.1
@@ Line 13: / Line 13: @@
     * [[https://arxiv.org/pdf/1510.01378.pdf|Laurent et al 2015 - Batch Normalized Recurrent Neural Networks]]
     * [[https://arxiv.org/pdf/1603.09025.pdf|Cooijmans et al 2016 - Recurrent Batch Normalization]]
-  * [[https://arxiv.org/pdf/1806.02375.pdf|Bjorck et al 2018 - Understanding Batch Normalization]]
+  * [[https://arxiv.org/pdf/1806.02375.pdf|Bjorck et al 2018 - Understanding Batch Normalization]]. See also section 3 of [[https://arxiv.org/pdf/2002.10444.pdf|De & Smith 2020 - Batch Normalization Biases Residual Blocks Towards the Identity Function in Deep Networks]] for a different perspective.
   * [[https://arxiv.org/pdf/2003.07845.pdf|Shen et al 2020 - PowerNorm: Rethinking Batch Normalization in Transformers]]