Recurrent Looped Transformer

The Recurrent Looped Transformer (RLT) processes tokens by passing the decoder's final hidden state to the next token, combined with its causal encoder representation. The decoder utilizes encoder-derived global KV memory and maintains a sliding-window attention (SWA) cache at each layer. This consistent update mechanism is applied across both prompt and response tokens. Further details are available at https://github.com/yifanzhang-pro/recurrent-looped-tranformer.