ArXiv

T^2MLR: Transformer with Temporal Middle-Layer Recurrence

2026-07-16 · 16:33 UTC ·Ziyang Cai, Xingyu Zhu, Yihe Dong... ·1 min read
Authors
Ziyang Cai, Xingyu Zhu, Yihe Dong...
Categories
cs.CL, cs.AI
arXiv
https://arxiv.org/abs/2607.15178v1
PDF
https://arxiv.org/pdf/2607.15178v1

Brief

T^2MLR (Transformer with Temporal Middle-Layer Recurrence) addresses autoregressive decoding’s loss of intermediate hidden computation by caching a middle-layer representation from the previous token and injecting it into an earlier layer of the current token. With little inference overhead, targeted middle-layer recurrence (sometimes only 20% of layers) outperforms matched baselines across pretraining and multi-hop reasoning; retrofitting a 1.7B pretrained model improves math reasoning.

Why it matters

Paper (Ziyang Cai et al., published on arXiv 2026-07-16) introduces T^2MLR (Transformer with Temporal Middle-Layer Recurrence) that fuses a cached middle-layer representation from the previous token into an earlier layer of the current token to preserve intermediate reasoning states.

Key details

  • T^2MLR consistently outperforms data- and parameter-matched Transformer baselines across natural-language pretraining and multi-hop reasoning finetuning; applying recurrence to only a localized middle-layer block — as little as 20% of the network — often beats full-layer recurrence.
  • The method has low inference overhead and can be retrofitted into an existing pretrained 1.7B-parameter Transformer with brief finetuning, yielding substantial improvements on math reasoning tasks.
Source evidence

Abstract

Transformer reasoning is limited by autoregressive decoding, which repeat edly compresses rich hidden computation through token space and makes it difficult for intermediate reasoning states to persist across time. We in troduce Transformers with Temporal Middle-Layer Recurrence (T2MLR), a transformers-based latent reasoning architecture that fuses a cached middle layer representation from the previous token directly into an earlier layer of the current token position, enabling abstract intermediate computation to persist across decoding steps with little inference overhead. Across natural-language pretraining and multi-hop reasoning finetuning, T2MLR consistently outperforms data- and parameter-matched Transformer base lines. Moreover, applying recurrence to only a localized middle-layer block (as little as 20% of the network) often outperforms full-layer recurrence. Im portantly, T2MLR does not require pretraining from scratch: retrofitting the recurrent pathway into an existing pretrained 1.7B Transformer and briefly finetuning substantially improves math reasoning, lowering the barrier to practical adoption. These results suggest that effective latent reasoning in Transformers does not require looping over all layers as in previous works, but can instead emerge more strongly from targeted middle-layer recurrence.