A Guth Labs publication ↗
Back to News

Superfluid research targets the queue between local agents and their human user

Superfluid research targets the queue between local agents and their human user

guthlabs.newsroom | November 16, 2026

A technical report submitted November 13 describes Superfluid, a local inference server designed to keep an interactive chat responsive while agents share its computing resources. Rather than replacing model runtimes, the system coordinates isolated llama.cpp, MLX or BaseRT workers and prioritizes interactive requests. The authors report a first-token delay of roughly 1.5 seconds with eight busy agents on an Apple M5 Pro, compared with 291–612 seconds for the evaluated alternatives. These are developer-reported results, not independently reproduced measurements. Research paper

The project also routes work across multiple machines. Crucially, it replicates a model rather than splitting one oversized model across devices: each node needs memory for a complete copy. Its repository describes OpenAI-, Anthropic- and Ollama-compatible endpoints, plus limitations including prompt recomputation after migration and possible output differences between quantizations. Superfluid remains pre-1.0. Its server repository uses Apache-2.0; BaseRT is separately licensed. Project repository

Guth News analysis: The interesting advance is scheduling and continuity, not a new model-quality score. Teams operating agents on mixed local hardware could benefit from keeping human requests out of long background queues. But a serving-layer recovery mechanism does not demonstrate that an agent’s external actions are safe to repeat. Those responsibilities still require application-level controls.

The paper is dated November 13, not a November 16 product launch. This coverage examines the newly surfaced research; the project’s original release date was not established in this pass.

Sources and citations

Revision history

Published .