#representation-learning

• 29 Sept 2023 · Paper summary

  Paper summary: Vision Transformers Need Registers

  ViTs repurpose some background patch tokens as internal scratch space, which produces high-norm artifacts; adding a few extra "register" tokens fixes this and smooths attention maps.