RWKV
"Receptance Weighted Key Value" — architecture that trains in parallel like a transformer but runs as an RNN at inference, with constant memory regardless of sequence length.
Driven by Bo Peng and a large community since 2021. Token mixing via a linear decay mechanism instead of full attention. v5/v6 ("Eagle"/"Finch") approach transformer quality on several benchmarks with significantly lower inference cost for long contexts.
Philosophically similar to Mamba (the SSM family) — both attack the transformer's quadratic cost. RWKV is still more of an outsider but driven by open research without big labs.