ml:state-space_models
Table of Contents
State-Space Models
Overviews
- Survey Papers
- Papers with Good Overviews
- S4 model: Gu et al 2021 - Efficiently Modeling Long Sequences with Structured State Spaces Good intro to state spaces
- Gu & Dao 2023 - Mamba: Linear-Time Sequence Modeling with Selective State Spaces Nice overview of SSMs
Key Papers
- Mega: Ma et al 2022 - Mega: Moving Average Equipped Gated Attention SOTA on long-range arena benchmark. Combines flash attention with state-space models. Still n-squared runtime however
- Orvieto et al 2023 - Resurrecting Recurrent Neural Networks for Long Sequences Gives a nice history
- Gu & Dao 2023 - Mamba: Linear-Time Sequence Modeling with Selective State Spaces First rejected at ICLR (bad reviewing) and then accepted to COLM (reviews)
- Mamba: The Hard Way Sasha Rush's tutorial implementation
Papers
Analysis and Mechanistic Interpretability
Theoretical Properties
People
Related Pages
ml/state-space_models.txt · Last modified: 2025/08/22 17:58 by jmflanig