Understanding Tool-Integrated Reasoning

Tool-Integrated Reasoning (TIR) fundamentally expands the capabilities of Large Language Models (LLMs) by integrating external tools like Python interpreters. Unlike pure-text models limited by their token-based reasoning, TIR breaks these barriers by enabling deterministic state transitions.

The authors prove that TIR strictly enlarges both the empirical and feasible support of LLMs. This means TIR models can generate correct reasoning trajectories impossible for pure-text models, especially for complex problems requiring algorithmic strategies that are token-inefficient to simulate in natural language.

Token efficiency is key: programmatic solutions represent algorithms compactly (O(1) tokens), while natural language simulations scale linearly or worse with problem size, quickly exceeding feasible token budgets. For example, iterative tasks or solving large linear systems are succinct in code but verbose in text.

Under any fixed token budget, TIR models have strictly larger feasible support than pure-text models. This means many algorithmic strategies are practically unreachable for pure-text models but accessible via TIR, making tool integration not just convenient but necessary for complex reasoning.

To encourage earlier and more interactive tool use, the authors propose Advantage Shaping Policy Optimization (ASPO), a novel RL algorithm that modifies the advantage function directly. ASPO overcomes instability issues of naive reward shaping and guides models to invoke tools earlier without sacrificing accuracy.

Experiments on challenging math benchmarks (AIME24, AIME25, Omni-MATH-512) with Qwen3-8B show TIR models decisively outperform pure-text baselines across all pass@k metrics, breaking the capability ceiling and expanding problem-solving coverage by over 15%.

TIR benefits extend beyond computationally intensive problems. Using an “algorithmic friendliness” rubric, the authors show TIR models outperform pure-text even on highly abstract problems requiring deep insight, indicating TIR enables novel reasoning strategies beyond mere calculation.

Qualitative analysis reveals three emergent cognitive patterns in TIR models: 1) transforming abstract insight into programmatic computation, 2) iterative exploration and verification via code, and 3) offloading complex calculations to minimize errors. These patterns represent fundamentally new problem-solving classes.

ASPO training leads to earlier and more frequent code invocation, transforming tool use from a late-stage calculator to an interactive partner throughout reasoning. This shift doubles code usage rounds and increases code invocation from 1,000 to 4,000 tokens earlier, without reward hacking or performance loss.

The authors conclude that TIR shifts the paradigm: LLMs become reasoning engines that delegate complex tasks to specialized tools. Their theoretical framework and ASPO algorithm provide principled explanations and practical methods to harness tool integration for more powerful, stable, and controllable AI reasoning.