In AI agent training, you can pay for powerful GPUs and still spend much of your time waiting. Agents need to attempt tasks and produce results before the model can learn from them. A few slow attempts can hold up the next training update, making the systems that run those tasks a major factor in training speed and cost.

Di Jin, VP of AI Training at Amsterdam-headquartered Nebius, says teams coming from pretraining or supervised fine-tuning can underestimate the work these systems do.

“If you staff and design it as a training project, inference becomes the bottleneck nobody owns.”

Rollout generation often takes most of a training step’s wall-clock time, making batching, cache management and scheduling central to how fast the model can learn. “If you staff and design it as a training project, inference becomes the bottleneck nobody owns,” Jin says.

He adds that engineers should measure time to a trainable batch alongside throughput. Average tokens per second can hide the long tail: in a synchronous batch, the step cannot finish until the final trajectory is completed. 

Agent trajectories do not all have the same length. Many finish quickly, while others take longer, leaving training GPUs waiting on a handful of stragglers. For this reason, Jin advises engineers to treat the inference engine as part of the post-training reinforcement learning optimizer.

Learning from data generated by an older model version involves a tradeoff. Some policy lag can improve GPU efficiency by letting generation and training overlap, Jin says, but its safety depends on how much the policy has changed and how the training algorithm handles the difference.

Source link