TMax v1.1

TMax v1.0 introduced a simple recipe for training terminal agents with RL: synthetic environments sampled along nine structured task axes, trained with outcome-only DPPO.Hamish Ivison*, Junjie Oscar Yin*, Rulin Shao, Teng Xiao, Nathan Lambert, and Hannaneh Hajishirzi. “TMax: A simple recipe for terminal agents.” arXiv preprint arXiv:2606.23321, 2026. *Equal contribution.

In v1.1, we keep the recipe and improve the data it trains on and the infrastructure it runs on.

What’s new in v1.1

1. Auditing and repairing every environment

v1.0 deliberately skipped validating its environments. The pipeline only checked that each environment builds, and relied on RL to drop tasks the model can never solve (a group where every rollout scores zero carries no gradient). That handles tasks that are too hard. It does not handle tasks that are broken the other way: a verifier that a lazy solution can pass. RL is very good at finding those shortcuts.

So v1.1 adds a one-shot audit-and-repair pass after generation, inspired by River.Yao et al., “River,” 2026. River uses rubric-based repair and filtering on existing datasets; we apply the same idea to our own generated data.

• Auditor. GPT-5.6 Sol reads each task’s instruction, environment setup, and verifier, and assigns one of eight labels, such as verifier too weak, answer leak, instruction–verifier mismatch, or clean.
• Repairer. For the three most common fixable labels, GPT-5.6 Sol gets the original task plus the auditor’s diagnosis, and writes a hardened verifier or a cleaned image specification.
• Rebuild and spot-check. We rebuild the repaired images, check that they still build with the dependencies they need, and drop everything else that was flagged.

The audit found far more problems than we expected:

Weak verifiers dominate. A typical one checks that an output file exists and matches one fixed expected value, but never runs the program the task asked for, so an agent can hardcode the answer. Below are real verdicts from the auditor, lightly trimmed.

A leaked answer has to leave the image. Our first leak repairs added a check that the agent’s output is not a byte-for-byte copy of the leaked reference file. That doesn’t work: the agent can copy the reference, append a comment, and pass both the inequality check and the behavioral tests. Of the first 18 leak repairs we accepted, 11 relied on exactly that check. The final repairer instead rebuilds the image without the leaked file. If the verifier needs the reference, it ships with the tests, which are only copied into the sandbox at verification time, after the agent has finished.

As in v1.0, we still do not filter out tasks a strong model can’t solve; RL’s zero-variance filtering takes care of those.

So does repair help? Holding everything else fixed, training on the repaired data improves Terminal-Bench Lite by 5.8 points and Terminal-Bench 2.1 by 2.3 points over the unrepaired v1.0 data. Our interpretation: weak verifiers reward shortcuts, so a policy trained on them partly learns to take shortcuts instead of doing the task.

2. Results

TMax-v1.1-13k against 12 other datasets

Since v1.0 we have doubled the set of baselines, adding six recent datasets: LiteCoder, SETA, CalibForge, Terminal-Lego, RST, and Facet. As before, we hold everything fixed (Qwen3.5-9B, our recipe, our harness) and change only the RL data. For several of these datasets, this is the first time they have been used for RL training in the open.

TMax-v1.1-13k has the highest mean on both benchmarks: 65.6 on TB Lite and 29.6 on TB 2.1. Its lead over the strongest baselines (RST on TB Lite, Facet on TB 2.1) is within the error bars, so the fair summary is that TMax-v1.1-13k matches or beats every dataset we tried. Notice where the unrepaired v1.0 data lands on TB Lite: mid-pack. Repair is what moves it to the top.

Every model size improves

We apply the recipe without modification to the other Qwen3.5 sizes and to Qwen3.6-27B.

On Terminal-Bench 2.0, v1.1 beats v1.0 at every size: TMax-2B goes from 2.9 to 6.4, TMax-4B from 18.9 to 25.1, TMax-9B from 27.2 to 33.0, and TMax-27B from 42.7 to 43.8.We re-ran the base models for v1.1 (three trials per task instead of five), so their scores differ slightly from the v1.0 post. For example, Qwen3.6-27B moved from 39.6% to 42.0%.

TMax-9B now beats every prior terminal agent up to 30B parameters in our comparison, including CalibForge-30B-A3B (32.6%), as well as Claude Haiku 4.5 and GPT-5-mini. The 27B gain is small, since Qwen3.6-27B is already strong (it even dips slightly on TB Lite), but TMax-27B still edges past the 1T-parameter Kimi-K2.5 (43.2%).

The v1.0 generalization results also hold for the new TMax-9B. It improves SWE-bench Verified from 46.2 to 55.3 and AIME’24/25 (solved with the terminal) from 72.2 to 93.3, and it gains 8 to 14 points on TB Lite under every other harness we tried (OpenHands, mini-SWE-agent, Terminus-2).

3. Models can generate their own training data

v1.0 used Gemini-3-Pro to write every environment, and v1.1 adds GPT-5.6 Sol for auditing and repair. That raises a fair question: does the recipe only work because these teachers are much stronger than the 9B policy?

To test this, we reran the entire pipeline (generation, auditing, and repair) with Qwen3.5-9B, the same model we then train. This produced 3.1k environments.

The self-generated data improves Qwen3.5-9B by 12.6 points on TB Lite (49.4 to 62.0) and 6.8 points on TB 2.1 (20.5 to 27.3). That beats more than half of the prior datasets, despite being smaller than most of them, and lands within a few points of TMax-v1.1-13k. The pipeline does not need a strong teacher: a model of similar strength to the policy is enough.

Why would this work? We think it comes down to an information asymmetry: for many tasks, building the environment is easier than solving it. Writing a sudoku puzzle together with a checker is easy; solving it through a terminal is harder. The generator only has to do the easy side.

Self-generation does cost more in yield. In our first self-generation run, only about 13–17% of the 9B model’s environments built and passed their initial-state check, so it took roughly six times as many task templates per kept environment as with Gemini.

4. Stable RL: it was the infrastructure

In v1.0, long training runs were often unstable, and some collapsed after a few hundred steps. The standard fixes, an FP32 LM head to reduce the mismatch between trainer and inference engine and larger groups, helped but did not remove the instability. The v1.0 paper also named a suspect: to save cost, our Podman sandboxes ran on the same nodes as the inference engines and competed with them for CPU and memory.

That suspect turned out to be the main cause. Once we reduced infrastructure noise, training ran smoothly for 1,000 steps.

Why sandbox noise breaks RL

Every rollout is a long agent episode, around 25 commands, inside its own sandbox, and every layer underneath can fail. An image fails to pull, a node is preempted, a command hangs because the node is overloaded, or the rollout spends its time budget waiting for a free sandbox. In v1.0, all of these looked exactly like the model failing: reward 0.

That is worse than random noise. Group-based RL scores each rollout against its siblings, so a fake zero actively pushes the policy away from whatever that rollout was doing, even if it was on track.

Failures are also correlated. When a registry or a node goes down, it takes many rollouts with it at once, so the update is not just noisy but large and wrong. In one incident during development, an image registry outage turned over 4,000 rollouts into zeros. The measured training solve rate fell from 0.70 to 0.14 even though the model had not changed, and the policy kept training on that reward for 24 steps.

A dedicated sandbox fleet

We fixed this on two fronts: give sandboxes their own machines, and make the trainer tell “the model failed” apart from “the infrastructure failed”.

• Sandboxes get their own machines. Sandboxes run on CPU nodes that the controller requests from Slurm on demand and releases when idle, so they no longer compete with inference for CPU, memory, and disk. Each rollout holds one 2-CPU sandbox lease. Leases expire unless renewed, and lost or orphaned sandboxes are cleaned up, so a multi-day run does not slowly leak capacity.
• Every task has a prebuilt image. We build and check one image per task before training starts, so nothing is pulled from a registry mid-run. A missing image is an error, never a silent fallback.
• Infrastructure failures are masked, not scored. Lost sandboxes, preempted workers, and setup timeouts raise typed errors. The trainer drops those rollouts and computes advantages from the valid siblings only. Failures the policy causes, like filling the disk or exhausting memory, still score 0. If more than half of recent rollouts are masked, the run stops rather than train on a broken fleet.
• The clock is fair. A rollout’s time budget starts once it holds a sandbox, not while it waits in the queue. A command that times out comes back to the agent as an observation instead of ending the episode.
• Same task, same behavior, wherever it lands. Sandboxes start without the host’s environment variables, run without root, have capped memory, disk, and output, and see thread counts that match their two CPUs. That last one mattered more than we expected: Go programs sized their thread pools to every CPU on the host and crashed under the sandbox’s memory cap. In a replay of 10 Go tasks, 1 passed with all host CPUs visible and all 10 passed with the thread count pinned.

With essentially the same recipe and data as v1.0, training now runs to 1,000 steps without collapsing. The same fleet also runs our TB Lite and TB 2.1 evaluations, at a fraction of the cost of a hosted sandbox service.

5. Smaller changes

• New evaluation setup. TB Lite and TB 2.1 now run on our own sandbox fleet over a fixed task subset (96 of 100 TB Lite tasks, 88 of 89 TB 2.1 tasks). We exclude, for every model, the few tasks that rootless Apptainer cannot run reliably. These scores are not comparable to the TB Lite and TB 2.1 tables in the v1.0 post: the infrastructure change alone moves the base Qwen3.5-9B from 41.9 to 49.4 on TB Lite. Headline TB 2.0 numbers still use Daytona on all tasks.
• Reward hacking. A light manual check of TMax-9B’s TB 2.0 runs found two clear hacking attempts, both scored zero. In one, the model notes that reading the cipher’s internal key is “cheating”, then submits it anyway.
• SFT. Our v1.0 finding still holds: SFT warm-starts help Qwen3-8B but hurt Qwen3.5, so we go straight to RL for Qwen3.5 and 3.6.

Citation

Ivison, Hamish and Yin, Junjie Oscar and Shao, Rulin and Xiao, Teng and
Lambert, Nathan and Hajishirzi, Hannaneh, "Tmax: A simple recipe for terminal
agents", arXiv preprint arXiv:2606.23321, 2026.

@misc{ivison2026tmaxsimplerecipeterminal,
      title={Tmax: A simple recipe for terminal agents},
      author={Hamish Ivison and Junjie Oscar Yin and Rulin Shao and Teng Xiao and Nathan Lambert and Hannaneh Hajishirzi},
      year={2026},
      eprint={2606.23321},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2606.23321},
}

← All posts