ml:mixture_of_expert_models

This is an old revision of the document!

Table of Contents

Mixture of Expert (MoE) Models

Mixture of Expert (MoE) Models

Mixture of expert (MoE) models, focusing on sparse MoE models.

Overviews

Fedus et al 2022 - A Review of Sparse Expert Models in Deep Learning

MoE Large Language Models

Fedus et al 2021 - Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity

People

Barret Zoph

Related Pages

Language Models

ml/mixture_of_expert_models.1741900646.txt.gz · Last modified: 2025/03/13 21:17 by jmflanig