Experience, not imitation
Fine-tuning optimises a model to match examples. An environment optimises for the outcome: did the agent actually solve the task, choosing its own steps as it does in production?
Why not fine-tuningApplied AI research
RL environments for AI agents.
An agent that investigates, decides and acts does not get better by copying examples of the work. We build the environment it practises in — the tasks, the tools it acts through, and a verifier that scores what it actually produced — and we measure the gain on work it has never seen.
Why environments
Agents get better by doing the work, not by reading it.
A prompt library and a set of examples tell an agent what good work looks like. An environment makes it do the work: attempt the real task, act through real tools, and be scored on what it actually produced. What improves is the behaviour your users see, because it is the behaviour that was rewarded.
Draw out what is already there
The model behind your agent can usually do the task some of the time. Reward steers it toward the version of itself that does it reliably, instead of overwriting it with a narrow set of demonstrations.
The bet
Verified, not assumed
Every environment ends in a certificate: the task improved on held-out instances, and nothing the use case depends on quietly got worse. We publish what that discipline catches, including our own mistakes.
Earlier researchHow it works
An environment built around the job your agent does.
We start from what success looks like for your task, and what it costs to get it wrong. Around that we build the tasks, the tools the agent acts through, and a verifier that checks what it actually produced. The agent trains against that environment — by updating weights, or by keeping a readable playbook with the weights frozen — and the result is certified: gain on held-out tasks, and no quiet regression in what your use case depends on.
See the process
Earlier research
Why we moved to environments.
Our first research program repaired trained models with small supervised patches, and measured carefully where that approach breaks. Patches that looked perfect by training loss leaked into neighbouring capabilities. Patches merged together destroyed each other. Imitation signals overstated capability exactly where models were weakest. Those findings are the case for training on outcomes instead.
When does an LLM help a GNN on relational data?
A paired-baseline study of graph-conditioned LLMs: they tie but don’t beat a well-tuned GNN, while a structured GNN encoding beats flat-text RAG by ~50× on token cost. The GNN is the predictor; the LLM is the interface.
Read the paper
You cannot trust a patch without a certificate
The unit of trust in model repair is the certificate, not the patch: a patch that looks perfect by training loss can leak, merging two good patches destroys both, and the safe install is to route, not merge — verified across two model scales and a second domain.
Read the paperNotes
Recent writing
Short-form write-ups of what we find, published as the work concludes.
- 2026-09-17
Point it at a warehouse, get an analyst trained on the answer
Root-cause analysis has no training signal: a warehouse records what happened, never why. So we generate a simulated twin of the warehouse where the answer is known exactly, train an investigator there, and send it back to the real data read-only. One measured run: 27 minutes from an unseen database to a trained investigator, 55k tokens per investigation after that.
Read the note - 2026-09-16
A swarm of agents, trained on reward: better answers, five times the cost
We replaced the single investigating agent with a team - a lead that plans and assigns, three specialists working in parallel - on the same 20 held-out investigations. Training the team gave the clearest improvement we have measured (60% to 80% fully correct, p = 0.023), but the trained team still did not beat one trained agent on the scoring rubric: more accurate, at 4.9 times the tokens.
Read the note - 2026-09-15
Pilot: an agent that learned from reward investigated as well, at 40% of the cost
A pilot in a simulated company: an investigation agent that learned a playbook from environment reward matched the best untrained agent (70% fully correct) with 60% fewer tokens, was best on cause types it never trained on, and learned one rule that went too far. Too small to be significant; the full test plan is below, and results will follow.
Read the note
Work with us
Bring us a use case.
Tell us the job you want an agent to do well, and what doing it well means. We will tell you plainly whether an RL environment is the right tool, and what it would take to build one.
Start a conversation