TMax v1.1

TMax v1.0 introduced a simple recipe for training terminal agents with RL: synthetic environments sampled along nine structured task axes, trained with outcome-only DPPO.Hamish Ivison*, Junjie Oscar Yin*, Rulin Shao, Teng Xiao, Nathan Lambert, and Hannaneh Hajishirzi. “TMax: A simple recipe for terminal agents.” arXiv preprint arXiv:2606.23321, 2026. *Equal contribution.

In v1.1, we keep the recipe and improve the data it trains on and the infrastructure it runs on.

What’s new in v1.1

1. Auditing and repairing every environment

v1.0 deliberately skipped validating its environments. The pipeline only checked that each environment builds, and relied on RL to drop tasks the model can never solve (a group where every rollout scores zero carries no gradient). That handles tasks that are too hard. It does not handle tasks that are broken the other way: a verifier that a lazy solution can pass. RL is very good at finding those shortcuts.

So v1.1 adds a one-shot audit-and-repair pass after generation, inspired by River.Yao et al., “River,” 2026. River uses rubric-based repair and filtering on existing datasets; we apply the same idea to our own generated data.

• Auditor. GPT-5.6 Sol reads each task’s instruction, environment setup, and verifier, and assigns one of eight labels, such as verifier too weak, answer leak, instruction–verifier mismatch, or clean.
• Repairer. For the three most common fixable labels, GPT-5.6 Sol gets the original task plus the auditor’s diagnosis, and writes a hardened verifier or a cleaned image specification.
• Rebuild and spot-check. We rebuild the repaired images, check that they still build with the dependencies they need, and drop everything else that was flagged.

The audit found far more problems than we expected:

Weak verifiers dominate. A typical one checks that an output file exists and matches one fixed expected value, but never runs the program the task asked for, so an agent can hardcode the answer. Below are real verdicts from the auditor, lightly trimmed.

A leaked answer has to leave the image. Our first leak repairs added a check that the agent’s output is not a byte-for-byte copy of the leaked reference file. That doesn’t work: the agent can copy the reference, append a comment, and pass both the inequality check and the behavioral tests. Of the first 18 leak repairs we accepted, 11 relied on exactly that check. The final repairer instead rebuilds the image without the leaked file. If the verifier needs the reference, it ships with the tests, which are only copied into the sandbox at verification time, after the agent has finished.

As in v1.0, we still do not filter out tasks a strong model can’t solve; RL’s zero-variance filtering takes care of those.

So does repair help? Holding everything else fixed, training on the repaired data improves Terminal-Bench Lite by 5.8 points and Terminal-Bench 2.1 by 2.3 points over the unrepaired v1.0 data. Our interpretation: weak verifiers reward shortcuts, so a policy trained on them partly learns to take shortcuts instead of doing the task.

2. Results

TMax-v1.1-13k against 12 other datasets

Since v1.0 we have doubled the set of baselines, adding six recent datasets: LiteCoder, SETA, CalibForge, Terminal-Lego, RST, and Facet. As before, we hold everything fixed (Qwen3.5-9B, our recipe, our harness) and change only the RL data. For several of these datasets, this is the first time they have been used for RL training in the open.

TMax-v1.1-13k has the highest mean on both benchmarks: 65.6 on TB Lite and 29.6 on TB 2.1. Its lead over the strongest baselines (RST on TB Lite, Facet on TB 2.1) is within the error bars, so the fair summary is that TMax-v1.1-13k matches or beats every dataset we tried. Notice where the unrepaired v1.0 data lands on TB Lite: mid-pack. Repair is what moves it to the top.

Every model size improves

We apply the recipe without modification to the other Qwen3.5 sizes and to Qwen3.6-27B.

On Terminal-Bench 2.0, v1.1 beats v1.0 at every size: TMax-2B goes from 2.9 to 6.4, TMax-4B from 18.9 to 25.1, TMax-9B from 27.2 to 33.0, and TMax-27B from 42.7 to 43.8.We re-ran the base models for v1.1 (three trials per task instead of five), so their scores differ slightly from the v1.0 post. For example, Qwen3.6-27B moved from 39.6% to 42.0%.

TMax-9B now beats every prior terminal agent up to 30B parameters in our comparison, including CalibForge-30B-A3B (32.6%), as well as Claude Haiku 4.5 and GPT-5-mini. The 27B gain is small, since Qwen3.6-27B is already strong (it even dips slightly on TB Lite), but TMax-27B still edges past the 1T-parameter Kimi-K2.5 (43.2%).

The v1.0 generalization results also hold for the new TMax-9B. It improves SWE-bench Verified from 46.2 to 55.3 and AIME’24/25 (solved with the terminal) from 72.2 to 93.3, and it gains 8 to 14 points on TB Lite under every other harness we tried (OpenHands, mini-SWE-agent, Terminus-2).

3. Models can generate their own training data

v1.0 used Gemini-3-Pro to write every environment, and v1.1 adds GPT-5.6 Sol for auditing and repair. That raises a fair question: does the recipe only work because these teachers are much stronger than the 9B policy?

To test this, we reran the entire pipeline (generation, auditing, and repair) with Qwen3.5-9B, the same model we then train. This produced 3.1k environments.

The self-generated data improves Qwen3.5-9B by 12.6 points on TB Lite (49.4 to 62.0) and 6.8 points on TB 2.1 (20.5 to 27.3). That beats more than half of the prior datasets, despite being smaller than most of them, and lands within a few points of TMax-v1.1-13k. The pipeline does not need a strong teacher: a model of similar strength to the policy is enough.

Why would this work? We think it comes down to an information asymmetry: for many tasks, building the environment is easier than solving it. Writing a sudoku puzzle together with a checker is easy; solving it through a terminal is harder. The generator only has to do the easy side.

Self-generation does cost more in yield. In our first self-generation run, only about 13–17% of the 9B model’s environments built and passed their initial-state check, so it took roughly six times as many task templates per kept environment as with Gemini.

4. Stable RL: it was the infrastructure

In v1.0, long training runs were often unstable, and some collapsed after a few hundred steps. The standard fixes, an FP32 LM head to reduce the mismatch between trainer and inference engine and larger groups, helped but did not remove the instability. The v1.0 paper also named a suspect: to save cost, our Podman sandboxes ran on the same nodes as the inference engines and competed with them for CPU and memory.

That suspect turned out to be the main cause. Once we reduced infrastructure noise, training ran smoothly for 1,000 steps.

Why sandbox noise breaks RL

Every rollout is a long agent episode, around 25 commands, inside its own sandbox, and every layer underneath can fail. An image fails to pull, a node is preempted, a command hangs because the node is overloaded, or the rollout spends its time budget waiting for a free sandbox. In v1.0, all of these looked exactly like the model failing: reward 0.

That is worse than random noise. Group-based RL scores each rollout against its siblings, so a fake zero actively pushes the policy away from whatever that rollout was doing, even if it was on track.

Failures are also correlated. When a registry or a node goes down, it takes many rollouts with it at once, so the update is not just noisy but large and wrong. In one incident during development, an image registry outage turned over 4,000 rollouts into zeros. The measured training solve rate fell from 0.70 to 0.14 even though the model had not changed, and the policy kept training on that reward for 24 steps.

A dedicated sandbox fleet

We fixed this on two fronts: give sandboxes their own machines, and make the trainer tell “the model failed” apart from “the infrastructure failed”.

• Sandboxes get their own machines. Sandboxes run on CPU nodes that the controller requests from Slurm on demand and releases when idle, so they no longer compete with inference for CPU, memory, and disk. Each rollout holds one 2-CPU sandbox lease. Leases expire unless renewed, and lost or orphaned sandboxes are cleaned up, so a multi-day run does not slowly leak capacity.
• Every task has a prebuilt image. We build and check one image per task before training starts, so nothing is pulled from a registry mid-run. A missing image is an error, never a silent fallback.
• Infrastructure failures are masked, not scored. Lost sandboxes, preempted workers, and setup timeouts raise typed errors. The trainer drops those rollouts and computes advantages from the valid siblings only. Failures the policy causes, like filling the disk or exhausting memory, still score 0. If more than half of recent rollouts are masked, the run stops rather than train on a broken fleet.
• The clock is fair. A rollout’s time budget starts once it holds a sandbox, not while it waits in the queue. A command that times out comes back to the agent as an observation instead of ending the episode.
• Same task, same behavior, wherever it lands. Sandboxes start without the host’s environment variables, run without root, have capped memory, disk, and output, and see thread counts that match their two CPUs. That last one mattered more than we expected: Go programs sized their thread pools to every CPU on the host and crashed under the sandbox’s memory cap. In a replay of 10 Go tasks, 1 passed with all host CPUs visible and all 10 passed with the thread count pinned.

With essentially the same recipe and data as v1.0, training now runs to 1,000 steps without collapsing. The same fleet also runs our TB Lite and TB 2.1 evaluations, at a fraction of the cost of a hosted sandbox service.

5. Smaller changes

• New evaluation setup. TB Lite and TB 2.1 now run on our own sandbox fleet over a fixed task subset (96 of 100 TB Lite tasks, 88 of 89 TB 2.1 tasks). We exclude, for every model, the few tasks that rootless Apptainer cannot run reliably. These scores are not comparable to the TB Lite and TB 2.1 tables in the v1.0 post: the infrastructure change alone moves the base Qwen3.5-9B from 41.9 to 49.4 on TB Lite. Headline TB 2.0 numbers still use Daytona on all tasks.
• Reward hacking. A light manual check of TMax-9B’s TB 2.0 runs found two clear hacking attempts, both scored zero. In one, the model notes that reading the cipher’s internal key is “cheating”, then submits it anyway.
• SFT. Our v1.0 finding still holds: SFT warm-starts help Qwen3-8B but hurt Qwen3.5, so we go straight to RL for Qwen3.5 and 3.6.

Citation

Ivison, Hamish and Yin, Junjie Oscar and Shao, Rulin and Xiao, Teng and
Lambert, Nathan and Hajishirzi, Hannaneh, "Tmax: A simple recipe for terminal
agents", arXiv preprint arXiv:2606.23321, 2026.

@misc{ivison2026tmaxsimplerecipeterminal,
      title={Tmax: A simple recipe for terminal agents},
      author={Hamish Ivison and Junjie Oscar Yin and Rulin Shao and Teng Xiao and Nathan Lambert and Hannaneh Hajishirzi},
      year={2026},
      eprint={2606.23321},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2606.23321},
}

Suffix Cache Reuse Explained

Resources: 📄 Paper · 👨‍💻 GitHub · 🐦 Tweet

We recently introduced Context Language Models (CLMs),Rulin Shao, Shannon Zejiang Shen, Junjie Oscar Yin, Yuetai Li, Minheng Wang, Hamish Ivison, Radha Poovendran, Nathan Lambert, Teng Xiao, Mike Lewis, Wen-tau Yih, Luke Zettlemoyer, and Pang Wei Koh. “Context Language Models.” arXiv preprint arXiv:2609.37725, 2026.

which treat context as a file and can perform arbitrary manipulations on it. We showed that CLMs outperform state-of-the-art, human-designed context-management harnesses at lower cost. In this blog, we dive deeper into the efficiency side of CLMs: what metric do we use to capture the realistic serving cost while being cache aware, and how could we further improve the cache hit rate by designing serving systems for agents?

Prefix-Reuse FLOPs

Background: existing cache reuse often assumes append-only context. Serving engines such as SGLang and vLLM cache KV states, for example in a radix tree, and reuse the longest matching prefix of a new request. Tokens after the first mismatch must be re-prefilled. This works naturally for append-only histories, but after an in-the-middle edit, even unchanged suffix tokens are recomputed.

Below is a simplified example: the context A B C is edited into A B′ C, with the colors matching the token strip. The prefix A still matches, so its cache is reused; the match breaks at B′, so B′ and the unchanged C after it are prefilled together.

When measuring CLM efficiency, we account for prefix-cache reuse, i.e., which tokens can reuse cached KV states. We capture this with a theoretical inference-cost metric we call Prefix-Reuse FLOPs:

\mathrm{FLOPs}_{\text{Prefix Reuse}} = \underbrace{\mathrm{FLOPs}_{\text{prefill}}\big(\text{unmatched suffix}\big)}_{\text{from the first prefix mismatch onward}} + \underbrace{\mathrm{FLOPs}_{\text{decode}}\big(\text{generated tokens}\big)}_{\text{new output tokens}}

By default, we use this metric to measure CLM efficiency under standard serving. Thus, when we say CLMs are cheaper than the baselines, we’ve accounted for the lower cache hit rate caused by context edits.For example, in these performance-efficiency Pareto plots, we used prefix-reuse FLOPs with standard serving.

The visualization below shows how an edit affects the Prefix-Reuse FLOPs of a single turn.

The hit rate does drop with in-the-middle edits and incurs re-prefilling of the unchanged C.This also happens beyond CLM editing: some chat endpoints remove the thinking tokens of earlier turns, so the tokens after them are re-prefilled in the next turn.

In our experiments with a Qwen3.6-27B CLM on BrowseComp-Plus, standard SGLang serves 72.9% of all prompt tokens from its prefix cache, but only 24.2% on the turns right after a context edit. The rest of an edited turn is prefilled again, including the large part of the context that the edit left unchanged.

Suffix Cache Reuse

Prefix cache reuse has been the tradition in serving engines because context has always been append-only. But we ask: can we adapt serving engines for AI’s convenience? To that end, we propose a simple yet effective method, Suffix Cache Reuse, to make CLM serving even more efficient on the system side.

Say the context is A B C, and an edit replaces B with B′. Standard serving reuses the cache for A and then stops, because a prefix cache only matches up to the first changed token: B′ and all of C are prefilled again, even though C did not change. Suffix Cache Reuse (SCR) keeps the cached states for C instead of throwing them away. When the next prompt arrives, SCR compares it with the previous prompt of the same session to find the spans that survived the edit, shifts their rotary position encodings to their new positions, and splices them in after B′. Only B′ and newly appended tokens are prefilled. Below is an interactive visualization of SCR serving compared with prefix-reuse-only serving.

Technical details. For full-attention layers, SCR reuses the cached keys and values of C as they are, except for position: after the edit, C sits earlier by the length difference between B and B′, so SCR re-rotates the rotary position encodings of its cached keys by that offset. Because rotary encodings depend only on position, this rotation is exact and far cheaper than recomputing the entries. SCR is an approximation: C’s cached states were computed under the old context, before the edit. To bound how much approximation one edit can introduce, SCR relocates at most six surviving spans per edit, the longest first, and re-prefills the rest.More details, such as how the cap on relocated spans is set and what effect it has, are in Appendix B of the paper; the main text uses k = 6.

Qwen3.6-27B is a hybrid model: 48 of its 64 layers use linear attention, which keeps a fixed-size recurrent state rather than a per-token cache, so there are no per-token entries to move. For those layers, SCR continues from a snapshot of the recurrent state taken before the edit. The edit itself is seen by the 16 full-attention layers, and through their outputs it still reaches the later linear-attention layers.

Results on BrowseComp-Plus

We perform an end-to-end evaluation of SCR on BrowseComp-Plus with the CLM harness, by simply switching the Qwen3.6-27B endpoint from standard SGLang to our patched SGLang with SCR (Section 5.3 of the paper).Our implementation of Suffix Cache Reuse is available at facebookresearch/context-language-models/suffix_cache_reuse.

Averaged over three seeds, accuracy is on par (59.0% with standard serving, 60.6% with SCR), while compute drops from 11.00 to 7.21 PFLOPs per question. On the turns right after an edit, SCR serves an extra 28.1% of the prompt from relocated cache that standard serving would have recomputed. The savings show up in wall-clock time too: each question finishes 10% faster (874 s to 786 s). CLMs were already cheaper than the baselines under standard serving, through better context management alone; SCR brings their serving cost down to 65% of that.

Bonus: reasoning-token stripping is a context edit too. Other cache misses arise from reasoning-token stripping in some chat-template serving. For models such as Qwen3.6, earlier reasoning blocks are removed from subsequent prompts, forcing the unchanged suffix to be prefilled again. SCR reuses this suffix cache as well. In fact, most of SCR’s extra reuse comes from stripped reasoning rather than CLM edits. The figure below shows a decomposition of the tokens hit by Suffix Cache Reuse.

Improvement space for cache misses. We also looked at the unchanged tokens that are still prefilled with SCR. SCR removes most re-prefilling of unchanged suffixes after an edit. Much of the remaining overhead comes from unchanged prefixes that standard SGLang fails to reuse efficiently for hybrid models, because linear-attention states are cached only at request boundaries. This leaves substantial room for improvement: finer-grained recurrent-state checkpoints could increase cache hit rates for both standard prefix caching and SCR. More broadly, serving models with editable context opens many new systems research directions.

Suffix Cache Reuse in vLLM. We also ported SCR to vLLM as a plugin, without modifying vLLM itself, and repeated the BrowseComp-Plus comparison on both engines: Qwen3.6-27B, all 830 questions, three seeds, with standard serving and SCR running side by side on the same node. On both engines, SCR lowers compute and wall-clock time per question with accuracy on par.

Time per question covers the whole trajectory, from the first request to the final answer, and is compared within each engine, since the two engines run with different server settings.

SCR also speeds up serving itself. On a multi-turn editing workload, where each conversation starts from a document of about 11K tokens and 60% of later turns delete or rewrite an earlier section, SCR on vLLM raises request throughput by 1.4 to 1.6 times and cuts the median time to first token by 32% to 54% (one H200 per server).

Related work

With standard prefix caching, changing an early part of a prompt forces the serving system to recompute the KV states of everything that follows, even when the later text is unchanged. Prior work relaxes this requirement in different settings. Prompt Cache precomputes attention states for predefined prompt modules, allowing a module to be reused in prompts that do not share the same preceding text. In retrieval-augmented generation, the same document may appear after different documents or instructions. CacheBlend and EPIC reuse cached document chunks in these new contexts, recomputing selected tokens to account for the changed surroundings.

PIE studies cache reuse when a user modifies previously processed code and requests a new completion. It retains cached states for unchanged text after an edit and corrects their rotary positions, avoiding suffix recomputation. Memento evicts each completed reasoning block from the KV cache but keeps the cached states of its summary, which were computed while the block was still in context, and finds that these states retain useful information from the evicted block. Concurrently, KV-streams, released in the last few days, keeps the KV cache across agentic compaction during reinforcement learning instead of flushing it, and reports a 2× speed-up on SWE tasks with any compaction method. Suffix Cache Reuse applies the same reuse principle to an agent’s live context: when the agent replaces a span, the unchanged suffix retains its cached states rather than being prefilled again. We integrate this mechanism into SGLang for agent-driven context editing and further extend it to hybrid architectures that combine full-attention layers with linear-attention layers.

We believe cache space can enable more context management opportunities than pure token space. We are excited to see more work along this line.

Limitations

Suffix Cache Reuse makes more approximations than prefix cache reuse. A reused prefix is exact, since its states depend only on tokens that have not changed. A relocated suffix is not. Its cached states were computed while the old span was still in context, so they may still reflect text the edit removed and miss text it added. The approximation is larger for hybrid models, whose linear-attention layers resume from a recurrent state taken before the edit and see the new span only through the full-attention layers. We cap the number of relocated spans per edit to limit this, and on BrowseComp-Plus SCR matches standard serving in accuracy. We have not yet characterized where these approximations break down. More work is needed to profile their failure cases and to make reuse adaptive, deciding for each edit what to reuse and what to recompute.

References

Citation

@article{shao2026context,
  title   = {Context Language Models},
  author  = {Shao, Rulin and Shen, Shannon Zejiang and Yin, Junjie Oscar and
             Li, Yuetai and Wang, Minheng and Ivison, Hamish and
             Poovendran, Radha and Lambert, Nathan and Xiao, Teng and
             Lewis, Mike and Yih, Wen-tau and Zettlemoyer, Luke and
             Koh, Pang Wei},
  journal = {arXiv preprint arXiv:2609.37725},
  year    = {2026}
}

TMax: A Simple Recipe for Terminal Agents

1. Terminal agents, and why they’re hard to train

Terminal-using agents have quickly become the most popular way people put language models to work. A terminal agent is a language model that interacts with a terminal environment: by issuing commands in a shell, calling tools, reading the output, and iterating. It might resolve a failing build, profile a slow service, recover a corrupted archive, or wire up a multi-service stack. The model plans, runs a command, looks at stdout/stderr, and tries again. This is now the dominant interface for agentic coding products,For example, Anthropic’s Claude Code and other terminal-native coding products.

and the tasks these agents are asked to solve keep getting more complex and long-horizon.

Despite this popularity, there is surprisingly little open academic work on training terminal agents with RL. We think this comes down to three gaps:

• Hard benchmarks. Evaluations like Terminal BenchMerrill et al., “Terminal Bench,” 2026.
  
  capture genuinely long-horizon, multi-step terminal work, but they are hard, and much prior work sidesteps them in favor of simpler bug-fixing or natural-language-to-bash setups.
• A lack of data. Real-world terminal traces are scarce or proprietary, and the synthetic datasets that do exist tend to be small, narrow (mostly file manipulation or one dominant domain), or never release their actual RL environments.
• A lack of a simple baseline recipe. Most data-generation papers stop at supervised fine-tuning; the ones that try RL often report ~1-point gains, leaving little signal for the community to build on.

TMax closes the data and recipe gaps. We provide a large, difficulty-controlled dataset of complex terminal tasks and a simple RL recipe that produces large, reliable improvements of over 5 points on Terminal Bench, with gains that transfer across tasks and harnesses. Our contributions are:

1. TMax-15k, a dataset of 14,600 RL environments, over 2.5× larger than prior open terminal datasets and notably harder.
2. Open-weights terminal agents at 2B/4B/9B/27B scale, trained on this data. Our best 9B model reaches 27.2% on Terminal Bench 2.0, the strongest open model under 10B we are aware of under official settings.
3. A simple, reproducible RL recipe, with all components released for training the models.
4. Evidence that terminal RL generalizes across harnesses and across tasks, suggesting the model learns real new skills rather than overfitting a single setup.

2. Generating terminal environments at scale

Training a terminal agent with RL needs a lot of executable environments: each task has to ship with the files, dependencies, and an automated check that verifies whether the agent succeeded. Existing pipelines tend to fall short in three ways. They lean on complex, multi-stage generation that is hard to scale; they produce homogeneous task suites dominated by one or two domains; and they offer little control over difficulty, yielding tasks that are either trivially solved or entirely out of reach.

We propose a simple terminal environment generation pipeline that is scalable, diverse, and difficulty-aware. We use Gemini-3-Pro as the generator,We chose Gemini-3-Pro for its strong performance on Terminal Bench.

and the core idea is simple: compose each task by hierarchically sampling from a set of structured axes.

2.1 Generation pipeline

A task is composed by drawing one value from each of nine structured axes. We follow Nemotron-Terminal to seed the first two axesPi et al., “Nemotron-Terminal,” 2026. We seed our domain and skill axes from their taxonomy.

(domain and skills), and introduce six orthogonal axes targeting diversity and difficulty as shown in the figure above.

Because each task is sampled across these axes, the sampler yields combinatorially many distinct task signatures, while still letting us calibrate each axis on its own. The sampling is hierarchical because a few axes, such as personas and skills, are conditioned on the sampled domain to keep combination compatible. We intentionally make three design choices for the synthetic terminal data generation.

Scalability via soft filtering. Prior pipelines validate every task by having a strong model attempt it for multiple rollouts, which is expensive. We skip that step entirely. Our RL training already does soft filtering for free: it drops any task where all rollouts get the same reward, since those contribute no gradient. In practice the all-zero rate on our data is low (fewer than 8 samples filtered per batch). All we need to guarantee at inference time is that the environment successfully builds, which we do with one Docker image per task (sharing a pre-built base image per domain). Turning two build steps into one allows synthetic environments to scale cheaply.

Diversity via hierarchical sampling. The hierarchical sampler is itself the main diversity mechanism, and we add two more. First, we condition generation on domain-specific personasPersona-conditioned synthesis follows a line of prior data work, e.g. Ge et al., “Scaling Synthetic Data Creation with Personas,” 2025.

(6–18 per domain), so that a “red-team operator crafting an evasion payload” naturally shapes a plausible security task. Second, we incorporate multi-modal fixtures as part of task files: PNG, audio file, video, stripped binary, vendored package, etc. The policy itself stays a text-only model. It inspects and interacts these artifacts through ordinary terminal tooling (OCR, audio transcription, ffmpeg).

Difficulty via explicit calibration. To avoid the usual bimodal “trivial or impossible” task split, we calibrate difficulty along two new complexity axes (task complexity, from a handful of commands to intricate 30–60-command workflows, and command complexity, from bash-only to bash + code + system services) and sample uniformly across buckets. We also go beyond exact string-equality checks with graded verifiers: metric thresholds (e.g. accuracy ≥ 0.95), adversarial corpora (accept clean, reject malicious), fuzz equivalence against an oracle, or multi-protocol service checks. Thresholds give a continuous signal for terminal agents to hillclimb, while also increasing turn length.

We release the full set of RL environments as TMax-15K.

2.2 Comparing terminal datasets

We annotated our data and six prior datasets with Gemini-3-Pro and measured difficulty with Gemini-3-Flash (250-task subsample per dataset, 8 rollouts each).

Difficulty and balance statistics (Gemini-3-Flash, pass@k as mean over tasks). Balance is a “fraction-of-uniform” diversity score in [0, 1].

Two things worth to note. First, balance. Prior datasets pile 34–95% of their mass onto a single domain (software engineering alone is 95% of SWE-smith), while ours spreads roughly uniformly across all nine. That gives us the highest domain balance (0.998) and skill-type balance (0.732) of any dataset.

Second, difficulty. Our data is among the hardest of the bunch. Pass@1 is 42% (vs. 41–92% for prior datasets), and it has the lowest pass@8 (53%). The difficulty gap persists even as we draw more rollouts, so the tasks are genuinely hard rather than just high-variance. We also checked for contamination with a 13-gram sliding window against the Terminal Bench tasks and found 0% overlap.

2.3 SFT data and a simple harness

A light SFT warm-up can help stability and performance before RL,Shown e.g. in DeepSeek-R1 and Olmo 3.

so for our weaker-model experiments on the older Qwen3-8B, we also generate a small SFT dataset. Reusing the same pipeline, we make 2.2k more environments and sample 8 trajectories each from Qwen3.6-27B, for 16.5K trajectories (8K of them successful). We use this SFT data only for the Qwen3-8B runs, and release it as TMax-SFT-16.5K.

Both data generation and RL rollouts run through a simple harness based on mini-swe-agent with a persistent shell. We found the default Terminus-2 harness more brittle with small models, since it expects the agent to send raw keystrokes.

3. Training TMax with RL

With the data in hand, the question is whether a simple recipe can turn it into a strong agent. We deliberately keep the algorithm plain. The goal is a clean, reproducible testbed with obvious room for future algorithmic improvement.

3.1 Training recipe

We train with a variant of GRPOShao et al., “DeepSeekMath,” 2024.

, outcome-only with no learned reward model, plus a handful of changes for stability in the long-horizon agentic setting:

• DPPO instead of vanilla GRPO.Qi et al., “Rethinking the Trust Region for LLM RL” (DPPO), 2026.
  
  DPPO masks tokens where the inference (vLLM) and training logprobs disagree, using a binary approximation of the total-variation divergence. This small change meaningfully tames training collapse.
• A token-level loss,Token-level loss as in Yu et al., “DAPO,” 2025.
  
  fully asynchronous training, filtering of zero-standard-deviation groups, and active sampling to keep batches full.Following Olmo 3.
  
  
• An FP32 LM head, keeping the language-model head in full precision to minimize the train/inference mismatch. This mattered most for the hybrid Qwen3.5 models.

Infrastructure-wise, we extend open-instruct, use vLLM for rollouts, and run sandboxes via Podman or Apptainer. A typical run uses 2 nodes for training and 6 for inference on H100s, and takes 2–3 days. The key hyperparameters are:

We keep thinking on intermediate turns (“interleaved thinking”), and evaluate on Terminal Bench 2.1 and Terminal Bench Lite, averaging over 3 runs.

3.2 TMax-15K outperforms other terminal data

We first hold the model fixed (Qwen3.5-9B) and vary only the RL dataset. Training on TMax-15k gives the strongest Terminal Bench performance of any dataset we tried. We release the resulting model as TMax-9B.

RL on Qwen3.5-9B across datasets; mean over 3 runs. Our data improve Terminal Bench 2.1 by nearly 13 points over the base model.

We find that TMax-15K stays difficult throughout training. If we plot the average number of steps the model takes during training, TMax-15k consistently drives more steps per episode than other datasets. The tasks keep demanding real, multi-step work rather than being solved in a couple of commands.

We also see the model learn to think more over the course of training: the number of tokens it spends per assistant turn, for both reasoning and tool-calling, climbs steadily. This is the agentic analogue of inference-time reasoning scaling, and it tracks the model’s improving performance.

3.3 TMax-9B outperforms prior small terminal agents

At 27.2% on Terminal Bench 2.0, TMax-9B is the best open-weights model under 10B we compare against under official Terminal Bench settings,Terminal Bench scores can be sensitive to harness and timeout settings. With non-standard settings (for example, overriding the default timeouts) the base Qwen3.5-9B can score noticeably higher, so we report all numbers under official Terminal Bench settings for a fair comparison.

beating 32B terminal agents from several prior works and approaching closed offerings like Claude Haiku 4.5 (29.8%).

The same recipe scales across the whole Qwen3.5 family. TMax-2B and TMax-4B reach 2.9% and 18.9% (from 2.3% and 16.6%), and TMax-27B (built on Qwen3.6-27B) reaches 42.7% (from 39.6%). At that size, TMax-27B approaches models that are a few order of magnitude larger, like the 1T-parameter Kimi K2.5 (43.2%) and the 230B MiniMax M2.7 (45.1%).

3.4 Generalization across tasks, harnesses, and models

Finally, we show our training and data recipe generalizes well across three important axes: tasks (i.e., what it is asked to do), harnesses (i.e., what tools and prompts it is provided when performing tasks), and model families (i.e., the starting point model).

Across tasks. Evaluating TMax-9B beyond Terminal Bench, performance improves across the board, including on a non-agentic math benchmark:

Generalization to other tasks; mean over 3 runs. RL on terminal data improves an agentic SWE benchmark by ~9 points and math by ~18.

Across harnesses. When we swap out the harness for different prompts and tools than the one used during RL, TMax-9B improves by roughly 9 to 15 points in every harness we tried, even ones it never saw in training.

Terminal Bench Lite across evaluation harnesses; mean over 3 runs.

Across models. We also apply it to the older Qwen3-8B, with a short SFT warm-start and shorter context. It improves substantially on Terminal Bench Lite (7.3 → 17.7), though gains on the harder Terminal Bench 2.1 are smaller, since at this scale the benchmark’s difficulty makes improvements hard to see.

Qwen3-8B on Terminal Bench Lite and 2.1 after SFT and then RL; mean over 3 runs.

Taken together, these results strongly suggest that TMax RL training teaches the model new, transferable terminal skills, not mere harness-fitting or overfitting to Terminal Bench-style tasks.

4. Analysis & discussion

Two findings shaped the TMax recipe, and we think both are useful for the community training terminal agent in this setting.

4.1 Strong models don’t always want your SFT data

Common post-training pipelines advise warm-start RL with SFT for training stability. However, we find that existing datasets degrades Qwen 3.5’s performance, likedy due to the model undergoing heavy post-training already. This holds even for our own mixture, which we generated from a strong Qwen3.6-27B teacher. The older Qwen3-8B, by contrast, clearly benefits from the same SFT.

SFT before RL helps the older model but hurts the stronger one.

We suspect the larger mixtures lean on relatively weak teacher models from prior work, and that a heavily post-trained model has less to gain and more to lose from imitation. We leave further exploration of good SFT mixtures for Qwen 3.5 as future work.

4.2 Terminal-agent RL is hard to stabilize

We found training was frequently unstable and often collapsed past 200–300 steps. The main culprit is numeric mismatch between training and inference, which the hybrid Qwen3.5 architecture makes worse. Keeping the LM head in FP32 removes the worst logprob spikes (below), and moving from GRPO to DPPO, together with a larger group size, further limits collapse. Long horizons (often 20+ steps) and infrastructure load from running many sandboxes make things harder still. We give the full analysis, including the GRPO-vs-DPPO and group-size comparisons, in the paper.

We saw the same instabilities on Qwen3-8B, so this is not a quirk of one model family. We believe more stable, longer training would yield substantially better models.

5. Conclusion

TMax is a simple recipe for training strong terminal agents at small scale. Its two pieces are TMax-15k, a difficulty- and diversity-controlled dataset of 14,600 RL environments, and a plain DPPO training recipe. Together they are enough to train TMax-9B and TMax-27B, which is among the strongest open-weights models under 10B and 30B at the time of writing, respectively. The gains transfer to SWE-Bench, to AIME, and across harnesses, which is evidence that the model has genuinely improved at using a terminal rather than memorizing one setup. We release the data, models, and code as a starting point for others.

Where we’d go next. Follow from our results, three directions naturally falls out. Better training stability, more complex harnesses, and more complex data.

We hope TMax serves as a strong baseline and a useful testbed for the community to improve the stability, performance, and efficiency of terminal agents.

Acknowledgements

TMax is a collaboration between the University of Washington and the Allen Institute for AI (Ai2). We thank members of UW NLP and the Open Ecosystem team at Ai2 for feedback and discussion throughout this project, Michael Noukhovitch for useful discussions on RL stability, and the Ai2 Beaker team for help with infrastructure.

Citation

Ivison, Hamish and Yin, Junjie Oscar and Shao, Rulin and Xiao, Teng and
Lambert, Nathan and Hajishirzi, Hannaneh, "Tmax: A simple recipe for terminal
agents", arXiv preprint arXiv:2606.23321, 2026.

@misc{ivison2026tmaxsimplerecipeterminal,
      title={Tmax: A simple recipe for terminal agents},
      author={Hamish Ivison and Junjie Oscar Yin and Rulin Shao and Teng Xiao and Nathan Lambert and Hannaneh Hajishirzi},
      year={2026},
      eprint={2606.23321},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2606.23321},
}