This is an automated archive made by the Lemmit Bot.

The original was posted on /r/machinelearning by /u/Potential_Duty_6095 on 2025-06-07 08:34:07+00:00.

Super new research, from the authors of FlashAttention and Mamba(2):

https://arxiv.org/abs/2506.04761

Long Story Short: They extend Mamba2 to have state that can is not fixed and can grow in time, directly increasing Long Range Performance. This seem a sweet point between traditional Mamba2 where the state is fixed sized, being an bottleneck for long sequences, and Attention which is stateless, but need to store past KV pairs! All with specialised Triton kernels!