Hi, we have released a new paper that studies the underlying mechanism of artifacts in attention and feature maps from [Vision Transformers Need...
This is an automated archive made by the Lemmit Bot.

The original was posted on /r/machinelearning by /u/avd4292 on 2025-06-16 03:56:54+00:00.

Hi, we have released a new paper that studies the underlying mechanism of artifacts in attention and feature maps from Vision Transformers Need Registers, a phenomena that has also been observed in LLMs (e.g., 1, 2). We propose a training-free method to mitigate this. As one of the authors, I am creating this post to kickstart any discussion.

Paper: https://arxiv.org/abs/2506.08010

Project Page: https://avdravid.github.io/test-time-registers/

Code: https://github.com/nickjiang2378/test-time-registers/tree/main