A Guth Labs publication ↗
Back to News

MLPerf adds agentic post-training to its AI hardware scorecard

Illustrated AI accelerator and stopwatch beneath MLPerf Training v6.1 and Guth News lettering.
Guth News / AI-generated illustration

MLPerf adds agentic post-training to its AI hardware scorecard

By guthlabs.newsroom

MLCommons released MLPerf Training v6.1 on November 16, adding a large-model reinforcement-learning workload to the benchmark suite as AI infrastructure expands beyond conventional pre-training.

The new Qwen3.5-397B test uses Group Relative Policy Optimization, or GRPO, to measure post-training for software-engineering and tool-use tasks. A second addition, DLRMv4-HSTU, replaces the earlier recommendation workload with an attention-based architecture trained on the Yambda dataset. MLCommons announcement

The round includes 250 test results from 153 submitted systems and 24 organizations. MLCommons says 57 systems were cloud-based, representing 37% of submissions, versus 29% in the preceding round. NVIDIA Rubin-generation GPUs and Google's seventh-generation Ironwood TPUs appear for the first time. MLCommons announcement

Google separately reported that 640 Ironwood chips reached the agentic benchmark's target in 18.02 minutes. Its 8,192-chip DeepSeek-V3 submission recorded 1.70 minutes. These are benchmark times from a prescribed reference checkpoint to a target quality threshold, not the time or cost of training a complete frontier model from scratch. Google explicitly states that qualification alongside its result IDs. Google's results and methodology notes

Guth News analysis: The important change is what infrastructure buyers can compare. An agent-training pipeline must generate behavior, evaluate it and update the model, rather than simply consume a fixed text corpus. A standardized post-training workload adds useful evidence, but it does not establish a universal winner for every agent or production setup. Comparisons still need matching workloads, accelerator counts and system configurations.

The official MLPerf results explorer provides system-level records for closer inspection.

Sources and citations

Revision history

Published .