All four are Apache-2.0 and all four ship PPO and GRPO, so neither the licence nor the algorithm list decides anything. What decides it is whether your environment is a separately scheduled participant in the rollout or a callback inside the generator — because every fix for a slow tool buys throughput by training on stale data. Pick on which staleness knob you get, not on whose speedup number is biggest.
All four frameworks orchestrate LLM calls into datasets at scale, and on that axis the differences are ergonomic. The axis that decides your outcome is whether the tool can execute a verifier inside the loop — because a judge from the generator's own family filters half your rows and adds no information. Only one of the four treats programmatic validation as a first-class stage.
Trainers and GPUs are rentable and the base model changes every quarter, so the only durable thing an RL project produces is the environment — the task distribution, the tool surface and the verifier that scores a run. These four platforms disagree about where that artifact lives and who writes the reward, and the one that offered to own the whole pipeline is closing to new users. Pick on portability of the environment and ownership of the verifier; the trainer comparison is the easy part.
These four are not four alternatives at one layer — TRL is the trainer API, Axolotl and LlamaFactory wrap it, and Unsloth rewrites its source at import time. That single fact predicts the thing you will actually feel: TRL shipped 1.9.2 in July while two of the others still pin the 0.x line. The famous speed table nobody can source is the wrong axis entirely.
AI in trading is not one bot; it is a four-layer stack — signal, sizing, execution, risk — and each layer runs a different model with different failure modes. Map the layers and any "AI hedge fund" headline becomes legible in thirty seconds.
Four RL-for-trading projects, four near-identical feature lists — Gymnasium env, OHLCV ingest, PPO/SAC/A2C/DQN, backtest evaluation. The thing that actually decides which survives a serious research-or-prod loop is invisible there: who controls the simulation contract.