Everything in an RL project except one thing is rentable. The GPUs are rentable, the trainer is an open-source library, the base model will be replaced within two quarters, and the cluster goes back when you are done. What is left afterwards is the environment — the task distribution, the tool surface the agent acts through, and the verifier that decides whether a run succeeded — and that artifact is also, exactly, the eval you needed before you ever thought about training. These four platforms take different positions on where it lives and who writes the reward, and one of them is quietly closing to new users, which is the strongest argument in the comparison for keeping the artifact in a form you can carry out the door.
At a glance
Four ways to get from "my agent is unreliable at this task" to a trained policy, and they are not four versions of the same product — one is a package registry with a trainer attached, one turns your running software into an environment, one lives inside your application code, and one is a managed service in wind-down.
| Platform | Where the environment lives | Who writes the reward | Status |
|---|---|---|---|
| Prime Intellect | An installable Python package — verifiers module with its own pyproject.toml, published to the Environments Hub | You, as a rubric in the environment | Open library, hosted training and GPUs; thousands of community environments |
| HUD | An MCP server wrapping real software, run through a hosted gateway | You, as a task-level reward on the wrapped system | Open SDK plus cloud; maintains OSWorld-Verified and SheetBench |
| OpenPipe ART | Your own application code — the rollout is a function you already wrote | A judge model ranking trajectories (RULER), or your own | Open library; the company was acquired by CoreWeave, training available serverless |
| OpenAI RFT | Nowhere — you supply graders, not an environment | The vendor's grader schema, filled in by you | Part of the fine-tuning platform, closed to new users and winding down |
One thing to hold on to before the details: none of these vendors will make your agent better. They make it cheaper to run the experiment that tells you whether a trained policy would be better, and three of the four also make it cheaper to measure the agent you already have. That second use is the one most teams should buy first.
What you are actually choosing
Look at the loop and the asset question answers itself. The inference pool is vLLM or SGLang on rented hardware. The trainer is GRPO and its descendants, published in a dozen open libraries. The buffer between them exists because agentic rollouts are heavy-tailed and one slow episode would otherwise stall a synchronous batch — real engineering, and not yours. The environment is the only box encoding anything specific to your business: which tasks matter, what the agent may touch, and what counts as done.
That asymmetry should drive the choice, and it usually does not, because the visible differences between these vendors are GPU pricing and trainer features. The questions that will matter in eighteen months are narrower. Can you take the environment to a different trainer without rewriting it? Is the reward a function you control or a schema you filled in? And if this vendor stopped taking new customers tomorrow, what would you still have? The last one is not hypothetical, as the fourth column of the table shows.
The engineering side of building the thing — rollout latency distributions, verifier precision, reset determinism, why timeouts must be scored rather than dropped — is the subject of environment engineering for agentic RL. This post is about who to buy it from.
Prime Intellect and HUD are opposite bets on where environments come from
Both are open at the core and both let you keep the artifact, but they answer "where does an environment come from" in almost exactly opposite ways, and the answer determines which one fits your problem.
Prime Intellect treats environments as a distributable package ecosystem. Its verifiers library defines an environment as a Python module with a dataset, a rollout function and a rubric, declaring dependencies in its own pyproject.toml and shipping as a wheel; the Environments Hub is, structurally, a package registry with thousands of community entries, and prime-rl installs them for training without modification. This is the most portable formulation in the group — an environment you write against the spec runs under your own trainer, on your own hardware, forever. The cost is that the abstraction is a general one, so wrapping messy real software means writing the adapter yourself, and the library's own churn is real: it moved from a v0 API to a rewritten v1 namespace during 2026 and not every community environment followed. Pin your versions.
HUD treats your running software as the environment. The premise is that you should not build a simulation of your product to train against; you should wrap the actual product — the real APIs, the real spreadsheet, the real desktop — as an MCP server with tasks and rewards attached, and run the agent against that. It maintains OSWorld-Verified and SheetBench as public benchmarks, which is a useful signal about where its attention is: computer-use and browser agents, where a faithful simulation is effectively impossible and the only honest environment is the software itself. The trade is direction of portability. Your environment is now an MCP server, which is an unusually good thing to own — it is the same interface your production agent speaks, so the wrapper is reusable outside training — but the scale-out path runs through HUD's hosted gateway rather than any trainer you choose.
The practical split: if your task is API-shaped and you can describe success programmatically, Prime Intellect's formulation gives you the more durable artifact. If your task is software-shaped — a legacy internal tool, a spreadsheet workflow, a browser flow with no API behind it — HUD is doing the hard part you would otherwise spend a quarter on, and the MCP framing means the work is not wasted even if you never train. Both are compatible with the sequencing this wiki keeps arguing for: build the environment as an eval first, and let training be a thing you can do afterwards rather than the reason you started.
ART's bet: the reward is the part you should not write
OpenPipe's Agent Reinforcement Trainer starts from a different premise: that the environment already exists, because it is your application. You do not port your task into a framework; you wrap the agent loop you already run, and ART trains against it. That makes it the fastest of the four to a first run when the task lives inside a working product, and the weakest on portability, because "the rollout" is a function in your codebase rather than an artifact you can hand to another trainer.
Its more interesting contribution is RULER — Relative Universal LLM-Elicited Rewards. Rather than scoring each trajectory against a rubric, it asks a judge model to rank a group of trajectories against each other, which works because GRPO normalises within the group and only the ordering survives into the gradient. OpenPipe reports it matching or beating hand-crafted reward functions on most of its evaluation tasks while removing the labelled data and the reward engineering. Relative judgements are also genuinely easier for a judge than absolute ones, so the claim is mechanically plausible rather than just marketing.
What it costs is worth stating plainly, because it is easy to read RULER as free. It moves the definition of correctness from a program you can read into a model you have to calibrate, and every failure mode of an LLM judge comes with it — position bias, verbosity preference, drift when the judge model is upgraded underneath you. On a task where a programmatic check exists, write the check. RULER earns its place where no check exists: open-ended research, drafting, multi-tool work whose success is a judgement call. And if you use it, calibrate it like any other judge, with the machinery in judge calibration and meta-evaluation.
One structural note: OpenPipe was acquired by CoreWeave, and ART's serverless training path runs through that lineage. The library is open and the acquisition has not changed that, but a hosted training lane owned by a GPU provider has an obvious commercial gravity, and the portability question in this section is the one to ask about it.
The managed lane is closing, which is the whole argument
The fourth option is the one most teams assume exists: hand the task to the model vendor, supply graders, get a better model back. OpenAI's reinforcement fine-tuning does work like that — graders in its own schema, o-series reasoning models only, priced around $100 per hour of core training loop with model-graded tokens billed separately. It is by far the least work to start.
It is also going away. OpenAI notified developers in May 2026 that it is winding down the fine-tuning platform: organisations that had not previously run fine-tuning could no longer create training jobs from 7 May, the restriction tightened on 2 July, and existing customers lose the ability to create new jobs from 6 January 2027. Already-tuned models keep serving until their base models are deprecated, and no new models or platform features will be added during the wind-down.
Take the specific product out of it and the general lesson is the one worth carrying. A managed pipeline that owns the environment, the trainer and the weights leaves you with nothing transferable when it stops — not the environment, because you never built one, only graders in a proprietary schema; not the weights, because they are tied to a base model on someone else's deprecation calendar; and not the measurement, because grading outputs against a schema is not the same artifact as a task distribution with a verifier. Teams who took the other three routes still hold a runnable environment today. The comparison did not need to be won on features.
This is not an argument against managed services generally, and it is not a prediction that the others will last. It is an argument for a specific test: whatever you pick, check that the environment leaves the building with you. If the answer is no, price that in as lock-in the way build vs buy prices any other layer.
When to pick which
| Situation | Pick | Because |
|---|---|---|
| Task is API-shaped with a programmatic success check | Prime Intellect | The environment ships as an installable package, so it runs under your own trainer and outlives the vendor relationship |
| Task lives in real software with no clean API — browser flows, spreadsheets, an internal legacy tool | HUD | Wrapping the running system as an MCP server is the hard part, and the wrapper is reusable in production, not just in training |
| The agent already works inside a shipped product and you want a first RL run this month | OpenPipe ART | The rollout is code you already have; nothing gets ported. Accept that the environment is not a portable artifact |
| Success is a judgement call with no programmatic check | ART with RULER | Relative ranking is a workable reward where a rubric is not — provided you calibrate the judge like any other judge |
| You want a better model without owning any of this | None of them, yet | The managed lane is closing to new users; a stronger prompt, better tools and a model upgrade beat an RL project you cannot staff |
| You cannot state your agent's success rate on thirty real tasks | Build the eval first | No verifier means no reward. Whichever platform you choose, that number is the prerequisite, and building it is most of the work anyway |
What should not decide it: GPU price per hour, which matters less than your environment's throughput; and community environment counts, which measure a registry rather than the environment that represents your product.
FAQ
Do I need RL at all, or is this a prompt problem?
Most of the time it is a prompt, tool-design or context problem, and the ordering in prompt, fine-tune or RL is the right one: exhaust the cheap levers first. The useful thing about environment work is that it is not wasted either way — you cannot tell whether the cheap levers worked without the same task distribution and verifier that an RL run would need.
Why is the environment the asset rather than the trained weights?
Because weights are tied to a base model that will be superseded, usually within a couple of quarters, and re-running the training against a new base is cheap if you still have the environment. The environment encodes what your business considers correct, which does not change when the model does.
Can I use a community environment from the Environments Hub for my own task?
For infrastructure shakedown and for tasks nobody's revenue depends on, yes, and it is much faster than writing one. Two caveats: the verifier encodes someone else's definition of correct, which you inherit unread unless you read it; and a public environment is a plausible pre-training contaminant, so a strong score on it may be exposure rather than capability.
What does a training run actually cost?
Less than teams expect on GPUs and more than they expect on wall-clock, because the accelerators idle while environments execute. A single straggler episode stalls a synchronous batch, which is why the ecosystem converged on disaggregated inference and training pools with a buffer between them. Profile your environment's p99 episode duration before reserving anything; that number, not the GPU rate, sets the bill.
Is RULER good enough to replace a hand-written verifier?
Not where a programmatic check exists — a check that reads the artifact is sharper and cannot be talked into anything. RULER is for tasks where no check exists, and there it is a real option rather than a compromise. Either way, hand-score fifty trajectories and compare before you trust the reward, because a verifier that accepts wrong trajectories does not degrade training gently; it teaches the shortcut.
Further reading
On this wiki:
- Environment engineering for agentic RL — the build side: rollout latency, verifier precision, reset determinism.
- RLVR & GRPO for agents — the training recipe these platforms implement.
- RL for tool use — sparse rewards and credit assignment across steps.
- Reward design & hacking — why verifier precision beats recall.
- Judge calibration — the prerequisite if your reward is a model.
- The cost of evaluation — what running all of this repeatedly does to a budget.
Project sources:
- Prime Intellect Environments Hub — and the verifiers library.
- HUD — RL environments and evals, MCP-based.
- OpenPipe ART — and RULER.
- OpenAI reinforcement fine-tuning — and the platform deprecation notice.