循环 Transformer(Looped Transformers)通过在递归深度上复用同一 Transformer 模块,构建了一条对训练与推理均有效的计算轴,在参数量固定的前提下提升了有效深度。然而,该共享模块必须调控整个隐藏状态轨迹——该轨迹在训练及外推深度下均呈现动态变化。此外,在加性注入式循环 Transformer 中,输入条件信号在每一递归步均被重新引入,因此即使在输入条件参考点处应用共享状态转移,仍可能导致隐藏状态发生偏移。本文提出源中心状态演化(Source-Centered State Evolution, SCSE),旨在协调输入条件性与参考点保持型共享递归。具体而言,SCSE 通过其可学习锚点(anchor)与初始偏差(deviation)维持输入依赖性;允许非零偏差驱动递归计算,同时将零偏差映射为零;并通过零偏差掩码(zero-deviation mask)确保锚点的精确不变性。由此,指定锚点在构造上即为一步不动点(one-step fixed point)。零偏差强制偏置(zero-deviation forcing bias)指从锚点自身生成的下一偏差,在 SCSE 中该偏置消失,而非零偏差则保持活跃并支持状态依赖的递归计算。理论分析表明,零偏差强制偏置是一种设计自由度,其任务效应可能有害、中性或有益;SCSE 通过将其设为零,以实现锚点的精确不变性,从而消解该设计选择问题。在 WikiText-2、WikiText-103、直接网络语料预训练、保留网络文本迁移任务以及 LAMBADA 完形填空任务上,SCSE 均提升了受控递归质量前沿。消融实验表明,可学习锚点与锚点坐标系下的偏差递归是性能增益的主要来源;一项针对训练模型的案例研究进一步验证了上述结论。
Looped Transformers create a useful train- and test-time compute axis by reusing the same Transformer block over recurrent depth, increasing effective depth at a fixed parameter count. However, that shared block must then govern an entire trajectory of varying hidden states over trained and extrapolated depths. Furthermore, in additive-injection looped Transformers, an input-conditioned signal is reintroduced at every recurrent step, so applying the shared transition at an input-conditioned reference can still move the hidden state. In this paper, we propose Source-Centered State Evolution (SCSE), which is designed to reconcile input conditioning with reference-preserving shared recurrence. Specifically, SCSE retains input dependence through its learned anchor and initial deviation, allows nonzero deviations to drive recurrent computation while mapping zero deviation to zero, and guarantees exact anchor invariance through its zero-deviation mask. The designated anchor is thereby a one-step fixed point by construction. The zero-deviation forcing bias is the next deviation produced from the anchor itself and vanishes in SCSE, while nonzero deviations remain active and support state-dependent recurrent computation. Our theory shows that the zero-deviation forcing bias is a design degree of freedom whose task effect can be harmful, neutral, or beneficial; SCSE resolves this choice in favor of exact anchor invariance by setting the bias to zero. Across WikiText-2, WikiText-103, direct web-corpus pretraining, held-out web-text transfer, and LAMBADA completion, SCSE improves the controlled recurrent quality frontier. Ablation studies identify the learned anchor and the anchor-coordinate deviation recurrence as the primary contributors to the gain, and a trained-model case study grounds the anchor-response diagnostic in observed recurrent motion.