Zhang et al 2019 - A Lightweight Recurrent Network for Sequence Modeling Related to the Transformer, LRNs are a drop-in replacement to other RNNs, which remove the sequential natural of RNN processing. It essentially uses a Key-Query-Value attention mechanism instead of the recurrence.