Why AutoInfra
Shared infrastructure for parallel research.
We built SFR-RL as an integrated framework for efficient reinforcement learning. As TrainForge began running many experiments in parallel, it became important to decouple resource management from the research loop so each experiment could scale independently. Inspired by Tinker, we built SFR-AutoInfra: a shared training API that abstracts away infrastructure and lets TrainForge and researchers launch and scale runs, compare results, and rapidly explore new data, learning algorithms, curricula, and training schedules.
A complete training loop through one API
Create a client → generate completions → compute loss → apply an optimizer step → save the updated model
Client setupCreate a training or inference client.
AutoInfra assigns GPU capacity for the selected model and workload.
Elastic capacityGPU worker
AllocatingClient capacitySaved modelckpt-1
.create_client()Scaling and parallelism
Many experiments, one responsive service.
AutoInfra keeps request handling separate from GPU execution. As users submit different workloads, workers are created, assigned, reused, and released without slowing the entry path.
Fast coordination, distributed data movement
Lightweight commands travel through a fast, high-concurrency lane, while distributed workers process larger payloads through the database.
User 1gpt-oss finetune
ActiveUser 2Inkling finetune
WaitingUser 3Inkling inference
WaitingAssigned capacityWorker A
ActiveUser 1 · 4 GPUs
Elastic capacityWorker B
Waiting8 GPUs
Resilience and scale
Persistent requests and responsive capacity.
The database keeps requests, progress, and results persistent, so work survives gateway interruptions and can resume without losing data. The gateway records incoming work quickly, while the orchestrator assigns GPU workers and redirects capacity from idle connections to users with pending requests. This design supports hundreds of thousands of concurrent .forward_backward and .generate requests, including large payloads submitted by many users at once.
Unlike Tinker, AutoInfra users can request scale-out configurations tailored to their workloads—from an inference fleet spanning thousands of GPUs for high-concurrency RL to parallelism profiles optimized for speed or memory efficiency. When configurations are compatible, the orchestrator can share workers or reassign them within a few seconds, keeping capacity available for active experiments.
Self-improving models
Improving a self-improving model.
AutoInfra can train models for any task that can be exercised and evaluated in a sandbox. It also closes the loop for SFR's own systems: the models that power TrainForge and AutoR&D-Engineer are trained through AutoInfra. Their discoveries—new data, strategies, and learning methods—feed the next model version trained by AutoInfra.
Training the systems that improve the model
TrainForge, AutoR&D-Engineer, and generic harnesses can run in isolated, scalable sandboxes that supply experience and rewards for reinforcement learning.
SandboxWork in context
Attempt runningTrainForgeOrganizes each attempt
AutoR&D-EngineerPlans, acts, and uses tools
Generic harnessCodex / Claude Code / Pi
✓Verified outcomeTask reward
External connectionCompute
External connectionInternet
Environment separation
Independent environments protect training and evaluation.
To scale RSI learning cycles efficiently while protecting system integrity and guarding against reward hacking, AutoInfra separates the algorithm, training service, and sandbox down to their Python and Conda environments and package installations. Even when these components share the same physical machines, AutoInfra retains its own protected installation and package management. Researchers and agents work in separate algorithm environments, while model rollouts run in isolated sandboxes hosting TrainForge, AutoR&D-Engineer, or general-purpose harnesses such as Codex, Claude Code, OpenCode, and Pi.
This separation also supports recursive development. Our Auto Engineer can clone AutoInfra into an isolated environment and optimize the service, while TrainForge can evaluate a newly discovered LLM architecture and promote a successful design into AutoInfra.
Evaluation · Scalability · Efficiency · Cost
Frontier efficiency
AutoInfra supports full-parameter and LoRA training across a range of model architectures and training configurations. These evaluations summarize long-context throughput, demonstrated model scale, and the cost of a representative LoRA training run.
Training performance and capacity
Full-parameter results report per-GPU throughput at one-million-token context. LoRA profiles show the minimum H200 capacity used in this evaluation. K3 results compare training and inference consistency.
Full-parameter evaluation
Per-GPU throughput at 1M-token context
tokens/s/GPU
Higher is faster.AutoInfra completes every one-million-token context configuration shown. The best tuned FSDP2 variants ran out of memory at these settings.
LoRA capabilities
Minimum GPUs demonstrated
LoRA updates a compact set of trainable parameters while retaining the base model, reducing the memory needed for each training run.
GPUs
Lower is better.Bars show the minimum H200 GPU count used to run each model in this evaluation.
Cost effectiveness
Cost-effective via Compute Efficiency
A GPT-OSS-120B LoRA run on one AWS 8×H200 node processes 276.63 million training tokens per hour. The comparison applies each price to the same training-token volume.
Cost per million training tokens
AutoInfra on AWS$52.92Tinker listed price$644.55
AutoInfra on AWS$192,531Tinker listed price$2.33M
Price comparison for equal token volume, not training speed. AutoInfra estimates use measured throughput and the quoted AWS 8×H200 node price; Tinker estimates use its listed GPT-OSS-120B training price per token. Prices may change; storage and data-transfer charges are excluded.
Training–inference consistency
Minimal training–inference mismatch
We strive to minimize training–inference mismatch while maintaining speed and efficiency. We do this without turning on batch-invariant kernels, which slow inference. Instead, we conduct per-model analysis and optimization so inference speed is not compromised, even when training and inference use different or inconsistent parallelism layouts.
| Model | AutoInfra | VERL | Tinker |
|---|---|---|---|
| 8B Dense | 2.3e−15 | 6.4e−4 | — |
| 30B MoE | 7.0e−4 | 1.5e−3 | 2e−2 |
— indicates that no result was reported.
SFR research
Scalability and efficiency research
AutoInfra builds on SFR research into efficient large-model training and reinforcement learning.
Mixture-of-Parallelisms
The study reports 4.7–8.2× higher per-GPU throughput than tuned FSDP2 across evaluated configurations, with long-context results for 120B, 600B, and 1T models.
Read the paperEfficient RL Training for the Agentic Era
SFR-RL coordinates generation, tool use, and model updates across a GPU fleet. The study reports full-parameter GPT-OSS-120B training at 1M-token context on 16 H200 GPUs.
Read the articleTrainForge + AutoInfra
More experiments reach the model, faster.
TrainForge decides what to try; AutoInfra carries each experiment from request to result. Together, they turn parallel research into a repeatable learning loop.
01ChooseTrainForge proposes and compares experiments.
02RunAutoInfra places the model and manages its work.
03LearnSaved results guide the next training decision.