NLP Wiki

User Tools

Site Tools

Trace: • optimization • nn_initialization • regularization • quantum_machine_learning • conditional_computation
ml:conditional_computation

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revisionPrevious revision
Next revision		Previous revision
ml:conditional_computation [2022/08/01 07:56] – [Papers] jmflanigml:conditional_computation [2025/03/26 03:43] (current) – [Related Pages] jmflanig
Line 1: Line 1:
 ====== Dynamic NNs and Conditional Computation ====== ====== Dynamic NNs and Conditional Computation ======
 Dynamic neural networks use methods such as conditional computation, adaptive computation, dynamic model sparsification, early-exit approaches, etc to build larger models with less compute requirements. Dynamic neural networks use methods such as conditional computation, adaptive computation, dynamic model sparsification, early-exit approaches, etc to build larger models with less compute requirements.
 +
 +===== Overviews =====
 +  * [[https://arxiv.org/pdf/2102.04906.pdf|Han et al 2021 - Dynamic Neural Networks: A Survey]]
 +
 ===== Papers ===== ===== Papers =====
   * [[https://arxiv.org/pdf/2101.03961.pdf|Fedus et al 2021 - Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity]] Uses conditional computation to scale-up the Transformer.  Good overview of conditional computation in the introduction.   * [[https://arxiv.org/pdf/2101.03961.pdf|Fedus et al 2021 - Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity]] Uses conditional computation to scale-up the Transformer.  Good overview of conditional computation in the introduction.
Line 8: Line 12:
   * [[https://dynn-icml2022.github.io/|DyNN Workshop 2022]]   * [[https://dynn-icml2022.github.io/|DyNN Workshop 2022]]
  
 +===== Related Pages =====
 +  * [[Mixture of Expert Models|Mixture of Experts]]
ml/conditional_computation.1659340560.txt.gz · Last modified: 2023/06/15 07:36 (external edit)
Donate Powered by PHP Valid HTML5 Valid CSS Driven by DokuWiki