ml:mixture_of_expert_models

This is an old revision of the document!

Table of Contents

Mixture of Expert (MoE) Models

Mixture of Expert (MoE) Models

Mixture of expert (MoE) models, focusing on sparse MoE models.

Overviews

[[https://arxiv.org/pdf/2209.01667|Fedus et al 2022 - A Review of Sparse Expert Models in Deep Learning]]

MoE Large Language Models

Fedus et al 2021 - Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity

Related Pages

ml/mixture_of_expert_models.1741900587.txt.gz · Last modified: 2025/03/13 21:16 by jmflanig