OpenAI's upcoming Astra reportedly adopts a "recurrent depth" architecture, allowing the same set of Transformer layers to recompute hidden states
OpenAI's upcoming Astra reportedly adopts a "recurrent depth" architecture, allowing the same set of Transformer layers to recompute hidden states, thereby increasing the model's internal reasoning without continuously generating longer text chains.
Traditional Transformers typically allow each input representation to pass through fixed, distinct network layers sequentially; recurrent depth, on the other hand, repeatedly runs shared core layers for multiple rounds, feeding the previous round's hidden state back into the same computation module before decoding the output. This increases the internal computational depth during a single inference rather than the length of text the model must output.
This approach can exchange more computational testing for reasoning capability with a similar number of parameters. Research on Gated Recurrent Transformers shows that under similar training and inference computational budgets, a 3-layer recurrent model can match the performance of a 12-layer GPT-2 Small baseline; in larger scale experiments, the recurrent architecture achieves about 63% fewer parameters and 59% less peak decoding memory, at the cost of about a 10% increase in generation latency.
ByteDance's Seed team previously released the Ouro series Looped Language Model. Ouro is pre-trained on 77 trillion tokens, and the paper and technical interpretations claim it achieves about 2 to 3 times the parameter efficiency of standard feedforward architectures on several tasks by processing hidden representations through shared Transformer blocks in a loop; its focus is not on expanding knowledge storage but on enhancing the model's ability to combine existing knowledge and perform multi-step operations.
The safety issue with recurrent depth lies in the fact that additional reasoning occurs within hidden vectors rather than forming complete thought chains in readable text tokens. Existing Chain-of-Thought monitoring can scan the intermediate text outputs of the model to identify jailbreaks, deceptive intentions, or dangerous operational plans; however, hidden state loops make it impossible for monitoring systems to see parts of the actual reasoning process.
Reportedly, OpenAI is therefore limiting the extent of recurrent depth usage in Astra to retain readable text thought chains and plans to add additional CoT monitoring. This does not mean that monitoring can fully reconstruct hidden state reasoning; it merely covers a portion of the decision-making process that the model will still externally exhibit with visible reasoning trajectories.
In market mechanisms, buyers are AI developers, enterprises, and cloud customers who require higher reasoning quality, lower parameter costs, or shorter output lengths; sellers are cutting-edge model companies that possess model architectures, GPU computing power, training data, inference infrastructure, and safety assessment capabilities. Recurrent depth shifts some value from "stacking more parameters and outputting more tokens" to "achieving results through more efficient internal computation," with potential beneficiaries including model providers constrained by memory and deployment costs; those under pressure are model routes that purely use parameter scale or visible thought chain length as capability indicators. Safety assessment, interpretability, and hidden reasoning audits then become new scarce infrastructures.
Source: Public information
ABAB AI Insight
Recurrent depth is not a new concept. Early recurrent neural networks processed sequential information repeatedly through shared parameters, and research like Universal Transformer has explored iterative computation in Transformer layers; the current change is that cutting-edge reasoning models are beginning to use this "shared layer multi-round execution" for computational expansion during testing. The reported adoption of this method by OpenAI's Astra indicates that competition in model capabilities is no longer solely reliant on expanding parameters and data during pre-training, but also on dynamically investing more internal computation before providing answers.
In terms of capital pathways, standard large models typically exchange more parameters, more GPU memory, and longer Chain-of-Thought outputs for better performance on complex problems. Recurrent architectures attempt to reuse a set of trained weights repeatedly: companies can reduce the need for new parameters for each independent round of computation, but must pay for more inference rounds, latency, scheduling, and validation costs. For model companies, this changes the cost function—capital is no longer entirely focused on parameter expansion during the training phase, but also increasingly invested in inference clusters, dynamic stopping mechanisms, KV caching, low-latency networks, and safety monitoring. The early-exit gating proposed by Ouro essentially decides which tokens are worth looping multiple rounds, avoiding the highest computational costs for all tasks.
Historically, DeepMind's AlphaGo and AlphaZero combined neural network evaluations with tree search, significantly improving decision-making through more testing time search; OpenAI's o series and other reasoning models have externalized more computation as text thought chains. Recurrent depth represents a third route: transferring additional reasoning from tree search or text tokens to continuous latent representations. It also differs from MoE logic: MoE reduces per-token computation by activating only part of the experts, while recurrent depth enhances the processing depth of a single problem by repeatedly reusing shared layers. The industry is transitioning from "models outputting answers in a single forward pass" to "models allocating different computational budgets for difficult problems."
This represents a technological substitution. The first assumption to be replaced is the default belief that "more visible text reasoning equals more intelligence." Text thought chains are beneficial for user understanding, training, and safety auditing, but each token occupies context, introduces latency, and consumes reasoning resources; hidden loops can compress some complex computations into vector space, improving parameter utilization, but they also take away the most easily observable signals for safety teams. The key dividing line for future model capabilities will not be whether they can produce long reasoning, but whether they can efficiently reason internally while leaving sufficient auditable, verifiable, and accountable behavioral evidence.
ABAB News · Cognitive Laws
Visible reasoning does not equal all reasoning
Parameters determine capacity, recurrence determines computational depth
Capabilities are hidden in vectors, regulation loses text entry