Researchers at Nvidia have developed a new technique that flips the script on how large language models (LLMs) learn to reason.

The method, called reinforcement learning pre-training (RLP), integrates RL into the initial training phase rather than saving it for the end.

This approach encourages the model to “think for itself before predicting what comes next, thus teaching an independent thinking behavior earlier in the pretraining,” the researchers state in their paper.

By learning to reason on plain text without needing external verifiers, models trained with RLP show significant improvements in learning complex reasoning tasks downstream, hinting at a future of more capable and adaptable AI for real-world tasks.

The typical LLM training cycle

Typically, large language models are first pre-trained on vast amounts of text using a "next-token prediction" objective, where they are given a string of text and asked to continuously guess what the next word (or token) will be. In this phase, they learn grammar, facts, and basic associations.

In the later post-training phase, models usually learn complex reasoning abilities such as chain-of-thought (CoT) where a model lays out its reasoning step-by-step. This stage often involves supervised fine-tuning (SFT) or reinforcement learning from human feedback (RLHF), which require specialized, curated datasets.

The paper’s authors argue this sequential process does not match human comprehension, which is “not a linear token-by-token process, but rather a parallel integration of input with prior knowledge.” Existing pre-training methods lack this mechanism, hindering a model's ability to develop deep reasoning from the start.

How reinforcement learning pre-training works

RLP reframes this process by treating CoT generation as an action the model takes before predicting the next token. At each step, the model first generates an internal "thought" or reasoning chain. It then predicts the next word in the text, using the original context augmented with its new thought.

The model receives a reward based on how much its thought improved the accuracy of its prediction compared to a baseline that didn't generate a thought (pure next-token prediction). This reward signal is calculated automatically based on the change in probability, eliminating the need for external verifiers or human-labeled data. 

The reward is positive only when the generated thought helps the model better predict the next token. By rewarding thoughts based on their predictive benefit, RLP effectively teaches the model how to think usefully on the same massive, unstructured datasets used for standard pre-training. 

This continuous feedback loop allows the model to learn when a simple predictive guess is sufficient and when it needs to engage in deeper reasoning. As the researchers put it, “RLP is designed to shape thinking in base models by rewarding only those thoughts that measurably help next-token prediction.”

This foundational approach, however, doesn't make later fine-tuning stages obsolete. According to Bryan Catanzaro, VP of applied deep learning research at Nvidia and a co-author of the paper, RLP is designed to complement, not replace, these crucial steps. "RLP isn’t meant to replace the later post-training stages like supervised fine-tuning or reinforcement learning from human feedback," Catanzaro told VentureBeat. "Those stages remain crucial for refining model behavior... It’s really designed to amplify the effectiveness of those later phases by giving the model a head start."

RLP in action

In experiments with Qwen3-1.7B and Nemotron-Nano-12B, Nvidia’s team tested RLP across a suite of math and science reasoning benchmarks. The results show that models enhanced with RLP consistently outperformed their conventionally trained counterparts, with particularly strong gains in reasoning-heavy tasks. 

For an enterprise, this improved reasoning could translate to more reliable outputs in multi-step workflows like financial analysis or legal document summarization.

"RLP encourages the model during pretraining to think before it predicts, helping the model internalize a more coherent reasoning style," said Catanzaro. "This could help reduce subtle logical errors, especially in longer workflows.” 

While stressing that RLP-trained models will still need the usual guardrails such as verification layers, human oversight, and consistency checks, Catanzaro said that “RLP gives you a stronger baseline."

Importantly, the benefits of RLP compound instead of disappearing during subsequent fine-tuning stages (catastrophic forgetting is a common problem in LLM training, where later training stages cause the model to forget its previously learned skills and knowledge). The RLP-trained model achieved an overall score that was 7-8% higher than baselines after an identical post-training regimen. The researchers conclude that RLP “establishes robust reasoning foundations that are not washed out by downstream alignment but instead compound with post-training.”

The efficiency of the technique is a key finding. On the Qwen3-1.7B model, RLP improved performance by 17% over standard continuous pre-training and also beat a similar technique called Reinforcement Pretraining via prefix-matching rewards (RPT). This advantage held even when the baseline model was trained with 35 times more data to match the computational cost, confirming the gains come from the method itself, not just more processing.

Furthermore, RLP demonstrates impressive scalability and versatility, successfully extracting a reasoning signal from general-purpose web data—not just curated datasets. When applied to the hybrid Mamba-Transformer model Nemotron-Nano-12B, RLP achieved a 35% relative improvement over a heavily trained baseline while using just a tiny fraction of the data.

While these results point toward a more efficient path for building powerful models, Catanzaro frames the innovation as a fundamental shift in the learning process itself, rather than an immediate solution to high training costs.

"This research is exciting because it offers a shift in how models absorb information during pretraining leading to a smarter learning process," he explained. "It wouldn’t replace large-scale pretraining, but offer another creative method in building the best possible models."

A new foundation for AI training

Ultimately, RLP points toward a future where pre-training is no longer a monolithic process of next-token prediction. Instead, the next generation of models could be built on a hybrid of objectives, creating AI that learns to think more robustly from day one. Catanzaro offers a powerful analogy to frame this shift:

"Next-token prediction teaches a model what the world looks like; reinforcement-style objectives like RLP can teach it how to think about what it’s seeing," he said. "The combination of these two objectives could help models develop deeper, more structured thinking much earlier in training... Tools like RLP can build on top of that foundation, making learning more active, curious, and even more efficient."

There is still a lot to learn about the dynamics of reinforcement learning in the pre-training phase, but what seems clear is that “introducing exploration earlier in training opens a new axis for scaling — not just in size, but in how models learn to reason,” Catanzaro said.

Many of Microsoft’s US data center regions are experiencing shortages of physical space or servers.

The company said the tools will help data centers optimize AI workloads by boosting speed and efficiency while better handling intense AI traffic.

These seven MCP servers bridge the gap between AI capabilities and real-world infrastructure management, turning complex workflows into simple natural language commands.

Echelon, an artificial intelligence startup that automates enterprise software implementations, emerged from stealth mode today with $4.75 million in seed funding led by Bain Capital Ventures, targeting a fundamental shift in how companies deploy and maintain critical business systems.

The San Francisco-based company has developed AI agents specifically trained to handle end-to-end ServiceNow implementations — complex enterprise software deployments that traditionally require months of work by offshore consulting teams and cost companies millions of dollars annually.

"The biggest barrier to digital transformation isn't technology — it's the time it takes to implement it," said Rahul Kayala, Echelon's founder and CEO, who previously worked at AI-powered IT company Moveworks. "AI agents are eliminating that constraint entirely, allowing enterprises to experiment, iterate, and deploy platform changes with unprecedented speed."

The announcement signals a potential disruption to the $1.5 trillion global IT services market, where companies like Accenture, Deloitte, and Capgemini have long dominated through labor-intensive consulting models that Echelon argues are becoming obsolete in the age of artificial intelligence.

Why ServiceNow deployments take months and cost millions

ServiceNow, a cloud-based platform used by enterprises to manage IT services, human resources, and business workflows, has become critical infrastructure for large organizations. However, implementing and customizing the platform typically requires specialized expertise that most companies lack internally.

The complexity stems from ServiceNow's vast customization capabilities. Organizations often need hundreds of "catalog items" — digital forms and workflows for employee requests — each requiring specific configurations, approval processes, and integrations with existing systems. According to Echelon's research, these implementations frequently stretch far beyond planned timelines due to technical complexity and communication bottlenecks between business stakeholders and development teams.

"What starts out simple often turns into weeks of effort once the actual work begins," the company noted in its analysis of common implementation challenges. "A basic request form turns out to be five requests stuffed into one. We had catalog items with 50+ variables, 10 or more UI policies, all connected. Update one field, and something else would break."

The traditional solution involves hiring offshore development teams or expensive consultants, creating what Echelon describes as a problematic cycle: "One question here, one delay there, and suddenly you're weeks behind."

How AI agents replace expensive offshore consulting teams

Echelon's approach replaces human consultants with AI agents trained by elite ServiceNow experts from top consulting firms. These agents can analyze business requirements, ask clarifying questions in real-time, and automatically generate complete ServiceNow configurations including forms, workflows, testing scenarios, and documentation.

The technology delivers a significant advancement from general-purpose AI tools. Rather than providing generic code suggestions, Echelon's agents understand ServiceNow's specific architecture, best practices, and common integration patterns. They can identify gaps in requirements and propose solutions that align with enterprise governance standards.

"Instead of routing every piece of input through five people, the business process owner directly uploaded their requirements," Kayala explained, describing a recent customer implementation. "The AI developer analyzes it and asks follow-up questions like: 'I see a process flow with 3 branches, but only 2 triggers. Should there be a 3rd?' The kinds of things a seasoned developer would ask. With AI, these questions came instantly."

Early customers report dramatic time savings. One financial services company saw a service catalog migration project that was projected to take six months completed in six weeks using Echelon's AI agents.

What makes Echelon's AI different from coding assistants

Echelon's technology addresses several technical challenges that have prevented broader AI adoption in enterprise software implementation. The agents are trained not just on ServiceNow's technical capabilities but on the accumulated expertise of senior consultants who understand complex enterprise requirements, governance frameworks, and integration patterns.

This approach differs from general-purpose AI coding assistants like GitHub Copilot, which provide syntax suggestions but lack domain-specific expertise. Echelon's agents understand ServiceNow's data models, security frameworks, and upgrade considerations—knowledge typically acquired through years of consulting experience.

The company's training methodology involves elite ServiceNow experts from consulting firms like Accenture and specialized ServiceNow partner Thirdera. This embedded expertise enables the AI to handle complex requirements and edge cases that typically require senior consultant intervention.

The real challenge isn't teaching AI to write code — it's capturing the intuitive expertise that separates junior developers from seasoned architects. Senior ServiceNow consultants instinctively know which customizations will break during upgrades and how simple requests spiral into complex integration problems. This institutional knowledge creates a far more defensible moat than general-purpose coding assistants can offer.

The $1.5 trillion consulting market faces disruption

Echelon's emergence reflects broader trends reshaping the enterprise software market. As companies accelerate digital transformation initiatives, the traditional consulting model increasingly appears inadequate for the speed and scale required.

ServiceNow itself has grown rapidly, reporting over $10.98 billion in annual revenue in 2024, and $12.06 billion for the trailing twelve months ending June 30, 2025, as organizations continue to digitize more business processes. However, this growth has created a persistent talent shortage, with demand for skilled ServiceNow professionals — particularly those with AI expertise — significantly outpacing supply.

The startup's approach could fundamentally alter the economics of enterprise software implementation. Traditional consulting engagements often involve large teams working for months, with costs scaling linearly with project complexity. AI agents, by contrast, can handle multiple projects simultaneously and apply learned knowledge across customers.

Rak Garg, the Bain Capital Ventures partner who led Echelon's funding round, sees this as part of a larger shift toward AI-powered professional services. "We see the same trend with other BCV companies like Prophet Security, which automates security operations, and Crosby, which automates legal services for startups. AI is quickly becoming the delivery layer across multiple functions."

Scaling beyond ServiceNow while maintaining enterprise reliability

Despite early success, Echelon faces significant challenges in scaling its approach. Enterprise customers prioritize reliability above speed, and any AI-generated configurations must meet strict security and compliance requirements.

"Inertia is the biggest risk," Garg acknowledged. "IT systems shouldn't ever go down, and companies lose thousands of man-hours of productivity with every outage. Proving reliability at scale, and building on repeatable results will be critical for Echelon."

The company plans to expand beyond ServiceNow to other enterprise platforms including SAP, Salesforce, and Workday — each creating substantial additional market opportunities. However, each platform requires developing new domain expertise and training models on platform-specific best practices.

Echelon also faces potential competition from established consulting firms that are developing their own AI capabilities. However, Garg views these firms as potential partners rather than competitors, noting that many have already approached Echelon about collaboration opportunities.

"They know that AI is shifting their business model in real-time," he said. "Customers are placing immense pricing pressure on larger firms and asking hard questions, and these firms can use Echelon agents to accelerate their projects."

How AI agents could reshape all professional services

Echelon's funding and emergence from stealth marks a significant milestone in the application of AI to professional services. Unlike consumer AI applications that primarily enhance individual productivity, enterprise AI agents like Echelon's directly replace skilled labor at scale.

The company's approach — training AI systems on expert knowledge rather than just technical documentation — could serve as a model for automating other complex professional services. Legal research, financial analysis, and technical consulting all involve similar patterns of applying specialized expertise to unique customer requirements.

For enterprise customers, the promise extends beyond cost savings to strategic agility. Organizations that can rapidly implement and modify business processes gain competitive advantages in markets where customer expectations and regulatory requirements change frequently.

As Kayala noted, "This unlocks a completely different approach to business agility and competitive advantage."

The implications extend far beyond ServiceNow implementations. If AI agents can master the intricacies of enterprise software deployment—one of the most complex and relationship-dependent areas of professional services — few knowledge work domains may remain immune to automation.

The question isn't whether AI will transform professional services, but how quickly human expertise can be converted into autonomous digital workers that never sleep, never leave for competitors, and get smarter with every project they complete.

OpenAI’s annual developer conference on Monday was a spectacle of ambitious AI product launches, from an app store for ChatGPT to a stunning video-generation API that brought creative concepts to life. But for the enterprises and technical leaders watching closely, the most consequential announcement was the quiet general availability of Codex, the company's AI software engineer. This release signals a profound shift in how software—and by extension, modern business—is built.

While other announcements captured the public’s imagination, the production-ready release of Codex, supercharged by a new specialized model and a suite of enterprise-grade tools, is the engine behind OpenAI’s entire vision. It is the tool that builds the tools, the proven agent in a world buzzing with agentic potential, and the clearest articulation of the company's strategy to win the enterprise.

The general availability of Codex moves it from a "research preview" to a fully supported product, complete with a new software development kit (SDK), a Slack integration, and administrative controls for security and monitoring.This transition declares that Codex is ready for mission-critical work inside the world’s largest companies.

"We think this is the best time in history to be a builder; it has never been faster to go from idea to product," said OpenAI CEO Sam Altman during the opening keynote presentation. "Software used to take months or years to build. You saw that it can take minutes now to build with AI." 

That acceleration is not theoretical. It's a reality born from OpenAI’s own internal use — a massive "dogfooding" effort that serves as the ultimate case study for enterprise customers.

Inside GPT-5-Codex: The AI model that codes autonomously for hours and drives 70% productivity gains

At the heart of the Codex upgrade is GPT-5-Codex, a version of OpenAI's latest flagship model that has been "purposely trained for Codex and agentic coding." The new model is designed to function as an autonomous teammate, moving far beyond simple code autocompletion.

"I personally like to think about it as a little bit like a human teammate," explained Tibo Sottiaux, an OpenAI engineer, during a technical session on Codex. "You can pair a program with it on your computer, you can delegate to it, or as you'll see, you can give it a job without explicit prompting."

This new model enables "adaptive thinking," allowing it to dynamically adjust the time and computational effort spent on a task based on its complexity.For simple requests, it's fast and efficient, but for complex refactoring projects, it can work for hours.

One engineer during the technical session noted, "I've seen the GPT-5-Codex model work for over seven hours productively... on a marathon session." This capability to handle long-running, complex tasks is a significant leap beyond the simple, single-shot interactions that define most AI coding assistants.

The results inside OpenAI have been dramatic. The company reported that 92% of its technical staff now uses Codex daily, and those engineers complete 70% more pull requests (a measure of code contribution) each week. Usage has surged tenfold since August. 

"When we as a team see the stats, it feels great," Sottiaux shared. "But even better is being at lunch with someone who then goes 'Hey I use Codex all the time. Here's a cool thing that I do with it. Do you want to hear about it?'" 

How OpenAI uses Codex to build its own AI products and catch hundreds of bugs daily

Perhaps the most compelling argument for Codex’s importance is that it is the foundational layer upon which OpenAI’s other flashy announcements were built. During the DevDay event, the company showcased custom-built arcade games and a dynamic, AI-powered website for the conference itself, all developed using Codex.

In one session, engineers demonstrated how they built "Storyboard," a custom creative tool for the film industry, in just 48 hours during an internal hackathon. "We decided to test Codex, our coding agent... we would send tasks to Codex in between meetings. We really easily reviewed and merged PRs into production, which Codex even allowed us to do from our phones," said Allison August, a solutions engineering leader at OpenAI. 

This reveals a critical insight: the rapid innovation showcased at DevDay is a direct result of the productivity flywheel created by Codex. The AI is a core part of the manufacturing process for all other AI products.

A key enterprise-focused feature is the new, more robust code review capability. OpenAI said it "purposely trained GPT-5-Codex to be great at ultra thorough code review," enabling it to explore dependencies and validate a programmer's intent against the actual implementation to find high-quality bugs.Internally, nearly every pull request at OpenAI is now reviewed by Codex, catching hundreds of issues daily before they reach a human reviewer.

"It saves you time, you ship with more confidence," Sottiaux said. "There's nothing worse than finding a bug after we actually ship the feature." 

Why enterprise software teams are choosing Codex over GitHub Copilot for mission-critical development

The maturation of Codex is central to OpenAI’s broader strategy to conquer the enterprise market, a move essential to justifying its massive valuation and unprecedented compute expenditures. During a press conference, CEO Sam Altman confirmed the strategic shift.

"The models are there now, and you should expect a huge focus from us on really winning enterprises with amazing products, starting here," Altman said during a private press conference. 

OpenAI President and Co-founder Greg Brockman immediately added, "And you can see it already with Codex, which I think has been just an incredible success and has really grown super fast." 

For technical decision-makers, the message is clear. While consumer-facing agents that book dinner reservations are still finding their footing, Codex is a proven enterprise agent delivering substantial ROI today. Companies like Cisco have already rolled out Codex to their engineering organizations, cutting code review times by 50% and reducing project timelines from weeks to days.

With the new Codex SDK, companies can now embed this agentic power directly into their own custom workflows, such as automating fixes in a CI/CD pipeline or even creating self-evolving applications. During a live demo, an engineer showcased a mobile app that updated its own user interface in real-time based on a natural language prompt, all powered by the embedded Codex SDK. 

While the launch of an app ecosystem in ChatGPT and the breathtaking visuals of the Sora 2 API rightfully generated headlines, the general availability of Codex marks a more fundamental and immediate transformation. It is the quiet but powerful engine driving the next era of software development, turning the abstract promise of AI-driven productivity into a tangible, deployable reality for businesses today.

Presented by Zendesk

---

Zendesk powers nearly 5 billion resolutions every year for over 100,000 customers around the world, with about 20,000 of its customers (and growing) using its AI services. Zendesk is poised to generate about $200 million in AI-related revenue this year, double than some of its largest competitors, while investing $400 million dollars in R&D. Much of that research is focused on upgrading the Zendesk Resolution Platform, a complete AI-first solution for customer service, employee service, and contact center teams, announced at Relate this past March.

During AI Summit, Chief Executive Officer Tom Eggemeier, along with members of the Zendesk team, took to the stage to announce several major advancements, including voice AI agents, video calling, and screen sharing for Zendesk Contact Center, and improved IT asset management, as well as the introduction of next-generation analytics, in the wake of its acquisition of HyperArc.

"We have built the only platform that is purpose-built for service and purpose-built for AI," Eggemeier said. "That focus is why we lead in AI for all types of service. And it is why we can deliver what no one else can for every service need you have in your organization."

New capabilities across use cases and companies

At its core, the Resolution Platform powers autonomous AI agents that solve complex issues in real time, leveraging leading LLMs like GPT-5, developed in collaboration with OpenAI, and supporting Model Context Protocol (MCP) to instantly access data, which streamlines workflows and improves autonomous problem-solving.

"Since our launch in March, we’ve been building fast, focused on making AI agents smarter, more flexible, and ready for even more channels," said Shashi Upadhyay, president of product, engineering, and AI at Zendesk. "And now, these AI agents are getting even better. They work across messaging, email, and now voice. They are getting smarter; able to handle multiple intents in a single message, detecting, remembering, and resolving many issues at once."

The only platform with native built-in QA, resolutions are automatically scored down to the conversation level, so teams can track resolution quality at scale. For startups, these insights are critical. They not only show what worked, but what needs fixing before it costs them time, reputation, or growth, and importantly, fit within a startup budget. That’s because Zendesk is the only company that charges only for successful resolutions, which are verified through the industry’s longest validation window, with two layers of quality checks.

Making the product CX admin a hero

Zendesk demonstrated the platform’s new features by highlighting a hypothetical wearable device company’s product launch. Service leaders at every stop along the product launch journey — from design to manufacturing — manage emerging issues with the support of the upgraded Resolution Platform.

For a global manufacturer that builds complex, state-of-the-art wearable tech, the pressure starts the moment a new product hits the market, tickets start pouring in, and a red-flagged backlog piles up.

"It is not a product issue, it is a resolution bottleneck," Upadhyay said. But, he added, "What once took days can now be resolved instantly."

The new Zendesk Admin Copilot is designed specifically to assist human agents, helping them spot what is not working, what to do next, and carry out changes quickly. It flags operational issues, like missing intent tags, broken internal processes, or routing conflicts that delay resolution. Copilot explains what is happening in plain language, recommends specific fixes, and with the admin’s approval, can make the changes itself. It's grounded in live Zendesk data, like tickets, triggers, and knowledge, so every recommendation is specific, current, and based on how the service operation actually runs.

Once the admin identifies the issue and implements a fix, the next step is ensuring everyone has access to the right knowledge to support it. For many organizations, that information lives outside of Zendesk. The newly launched Knowledge Connectors allows admins to pull in relevant content, like configuration guides or policy details, without needing to migrate anything so both human and AI agents have access to real-time instructions tied to the exact product version.

The admin also creates a smarter feedback loop with the new Action Builder, which automatically tags, summarizes, and sends notifications to the product team through Microsoft Teams.

And finally, Zendesk HyperArc will bring customers insights that combine AI and human analysis in a clear, narrative-driven view of what is happening and why, instead of siloed dashboards or static reports.

"With these innovations in place, change at the manufacturing plant cascades quickly, tickets are routed cleanly, support agents know what to say, engineering sees real signals instead of scattered anecdotes, and customers who just want the product to work get fast, reliable resolutions," Upadhyay said. "The CX Admin becomes the quiet hero of the manufacturer’s story."

Solutions for the retail CX leader

As a CX or contact center leader for a retail company, when a must-have wearable drops, how do you deliver service for your new hit product that feels personal and consistent when your team is stretched across multiple countries, channels, and customer expectations at once?

"Intelligent automation doesn’t just streamline operations — it enhances the customer experience across borders and channels," said Lisa Kant, senior vice president of marketing at Zendesk.

Zendesk’s Voice AI Agents are fully autonomous AI agents designed to understand natural speech, take action, and resolve issues without needing to escalate. They can verify identity, track orders, update deliveries, and answer setup questions in multiple languages, while keeping the brand experience consistent. Meanwhile, Video Calling lets a live agent spin up a video session, confirm the device is working, and walk the customer through setup or troubleshooting.

And because a help center is a critical part of delivering great service, especially when scaling fast across multiple countries and languages, Zendesk built Knowledge Builder, an AI-powered tool that helps teams build and maintain their help center content automatically. It analyzes real customer conversations and turns them into localized help articles for trending issues.

Giving IT leaders a strong edge

When a company adopts that new product, it becomes critical to resolve issues fast, to ensure employee productivity stays strong. Available with early access in November, Zendesk's new employee service offering, IT Asset Management (ITAM), natively integrates service and asset data together into the Zendesk service desk to help IT move from reactive troubleshooting to proactive service.

Now, when a vague “tablet not working” ticket comes in, Zendesk ITAM surfaces the device details right inside the ticket, so IT knows exactly what they are dealing with. Zendesk Copilot uses that same asset data to recommend model-specific troubleshooting steps. And with Knowledge Connectors, those steps can be pulled directly from SharePoint or Confluence without migration. If the fix does not work, the IT specialist confirms in seconds that the device is under warranty and issues a replacement without any back-and-forth.

With real-time visibility across every hardware asset, the IT leader can spot patterns before they become a flood of tickets, or failures at the point of care, so IT resolves issues faster and prevents problems before they happen.

"With Zendesk, IT is not just reacting to issues — it is setting the standard for how proactive employee service is delivered," Upadhyay said.

For more on the latest Zendesk updates and improvements, and to watch a conversation with Zendesk's special guest, co-founder of LinkedIn, Reid Hoffman, and more, watch the full videos here. And for the latest updates, detailed information, and product availability, visit Zendesk’s official announcements page.

---

Sponsored articles are content produced by a company that is either paying for the post or has a business relationship with VentureBeat, and they’re always clearly marked. For more information, contact sales@venturebeat.com.

Gulf nations have been subject to US restrictions on advanced AI chip shipments since 2023 due to concerns about technology diversion to China.

We look at some of the latest data center developments announced over the past month.

Researchers at the University of Illinois Urbana-Champaign and Google Cloud AI Research have developed a framework that enables large language model (LLM) agents to organize their experiences into a memory bank, helping them get better at complex tasks over time.

The framework, called ReasoningBank, distills “generalizable reasoning strategies” from an agent’s successful and failed attempts to solve problems. The agent then uses this memory during inference to avoid repeating past mistakes and make better decisions as it faces new problems. The researchers show that when combined with test-time scaling techniques, where an agent makes multiple attempts at a problem, ReasoningBank significantly improves the performance and efficiency of LLM agents.

Their findings show that ReasoningBank consistently outperforms classic memory mechanisms across web browsing and software engineering benchmarks, offering a practical path toward building more adaptive and reliable AI agents for enterprise applications.

The challenge of LLM agent memory

As LLM agents are deployed in applications that run for long periods, they encounter a continuous stream of tasks. One of the key limitations of current LLM agents is their failure to learn from this accumulated experience. By approaching each task in isolation, they inevitably repeat past mistakes, discard valuable insights from related problems, and fail to develop skills that would make them more capable over time.

The solution to this limitation is to give agents some kind of memory. Previous efforts to give agents memory have focused on storing past interactions for reuse by organizing information in various forms from plain text to structured graphs. However, these approaches often fall short. Many use raw interaction logs or only store successful task examples. This means they can't distill higher-level, transferable reasoning patterns and, crucially, they don’t extract and use the valuable information from the agent’s failures. As the researchers note in their paper, “existing memory designs often remain limited to passive record-keeping rather than providing actionable, generalizable guidance for future decisions.”

How ReasoningBank works

ReasoningBank is a memory framework designed to overcome these limitations. Its central idea is to distill useful strategies and reasoning hints from past experiences into structured memory items that can be stored and reused.

According to Jun Yan, a Research Scientist at Google and co-author of the paper, this marks a fundamental shift in how agents operate. "Traditional agents operate statically—each task is processed in isolation," Yan explained. "ReasoningBank changes this by turning every task experience (successful or failed) into structured, reusable reasoning memory. As a result, the agent doesn’t start from scratch with each customer; it recalls and adapts proven strategies from similar past cases."

The framework processes both successful and failed experiences and turns them into a collection of useful strategies and preventive lessons. The agent judges success and failure through LLM-as-a-judge schemes to obviate the need for human labeling.

Yan provides a practical example of this process in action. An agent tasked with finding Sony headphones might fail because its broad search query returns over 4,000 irrelevant products. "ReasoningBank will first try to figure out why this approach failed," Yan said. "It will then distill strategies such as ‘optimize search query’ and ‘confine products with category filtering.’ Those strategies will be extremely useful to get future similar tasks successfully done."

The process operates in a closed loop. When an agent faces a new task, it uses an embedding-based search to retrieve relevant memories from ReasoningBank to guide its actions. These memories are inserted into the agent’s system prompt, providing context for its decision-making. Once the task is completed, the framework creates new memory items to extract insights from successes and failures. This new knowledge is then analyzed, distilled, and merged into the ReasoningBank, allowing the agent to continuously evolve and improve its capabilities.

Supercharging memory with scaling

The researchers found a powerful synergy between memory and test-time scaling. Classic test-time scaling involves generating multiple independent answers to the same question, but the researchers argue that this “vanilla form is suboptimal because it does not leverage inherent contrastive signal that arises from redundant exploration on the same problem.”

To address this, they propose Memory-aware Test-Time Scaling (MaTTS), which integrates scaling with ReasoningBank. MaTTS comes in two forms. In “parallel scaling,” the system generates multiple trajectories for the same query, then compares and contrasts them to identify consistent reasoning patterns. In sequential scaling, the agent iteratively refines its reasoning within a single attempt, with the intermediate notes and corrections also serving as valuable memory signals.

This creates a virtuous cycle: the existing memory in ReasoningBank steers the agent toward more promising solutions, while the diverse experiences generated through scaling enable the agent to create higher-quality memories to store in ReasoningBank. 

“This positive feedback loop positions memory-driven experience scaling as a new scaling dimension for agents,” the researchers write.

ReasoningBank in action

The researchers tested their framework on WebArena (web browsing) and SWE-Bench-Verified (software engineering) benchmarks, using models like Google’s Gemini 2.5 Pro and Anthropic’s Claude 3.7 Sonnet. They compared ReasoningBank against baselines including memory-free agents and agents using trajectory-based or workflow-based memory frameworks.

The results show that ReasoningBank consistently outperforms these baselines across all datasets and LLM backbones. On WebArena, it improved the overall success rate by up to 8.3 percentage points compared to a memory-free agent. It also generalized better on more difficult, cross-domain tasks, while reducing the number of interaction steps needed to complete tasks. When combined with MaTTS, both parallel and sequential scaling further boosted performance, consistently outperforming standard test-time scaling.

This efficiency gain has a direct impact on operational costs. Yan points to a case where a memory-free agent took eight trial-and-error steps just to find the right product filter on a website. "Those trial and error costs could be avoided by leveraging relevant insights from ReasoningBank," he noted. "In this case, we save almost twice the operational costs," which also improves the user experience by resolving issues faster.

For enterprises, ReasoningBank can help develop cost-effective agents that can learn from experience and adapt over time in complex workflows and areas like software development, customer support, and data analysis. As the paper concludes, “Our findings suggest a practical pathway toward building adaptive and lifelong-learning agents.”

Yan confirmed that their findings point toward a future of truly compositional intelligence. For example, a coding agent could learn discrete skills like API integration and database management from separate tasks. "Over time, these modular skills... become building blocks the agent can flexibly recombine to solve more complex tasks," he said, suggesting a future where agents can autonomously assemble their knowledge to manage entire workflows with minimal human oversight.