A Guth Labs publication ↗
Back to News

Supermicro plans inference racks combining NVIDIA GPUs and SambaNova chips

Conceptual editorial illustration of different server modules linked by light streams in a hybrid inference rack; not an actual product photograph.
AI-generated editorial illustration for Guth News

Supermicro and SambaNova announced a collaboration on November 3 to build and sell inference systems that combine NVIDIA GPUs with SambaNova’s SN50 Reconfigurable Dataflow Units. The companies are initially planning for 10,000 SN50 chips, according to SambaNova’s announcement.

That figure describes the planned program. It is not a count of chips already delivered or serving customer traffic.

Supermicro will integrate, test, deliver and support the systems, including on-site service. SambaNova’s engineers will tune models for SN50 and assist deployment. The initial customers are expected to be neoclouds and sovereign clouds, followed by enterprises operating their own data centers.

Splitting prompt processing from response generation

The proposed systems divide inference into two stages. NVIDIA GPUs handle prefill, which processes the prompt; SambaNova RDUs handle decode, which produces the response. SambaNova’s technical explanation of the partnership argues that assigning the stages to different processors can improve speed and efficiency.

The distinction is relevant to agent workloads because a completed task may require many model calls. But faster generation at one stage does not automatically produce the same improvement in total task completion. Tool execution, network delays, verification and repeated attempts can remain substantial.

There is also a system-level tradeoff. Dividing work between processors requires coordination and data movement, so the useful comparison is the complete serving system under load, rather than an isolated chip’s token rate.

Earlier demonstrations provide context

SambaNova previously described a live demonstration in June that paired NVIDIA B200 GPUs with its older SN40 RDUs. The November agreement moves the commercial plan toward SN50 systems distributed and supported by Supermicro.

The older demonstration should not be used as a measured performance result for the new racks. The generations, configuration and deployment scope differ.

SambaNova says the planned systems can work in existing data centers without requiring extreme rack power density. That is a vendor claim to test against the actual configuration, power envelope and cooling available at a customer site.

For inference buyers, the meaningful next milestones are supported model configurations, delivery schedules and measurements of sustained throughput, latency and power at realistic concurrency. The agreement expands the route to market for specialized inference hardware; the economics will depend on what the complete racks deliver on customer workloads.

Sources and citations

Revision history

Published .