A Guth Labs publication ↗
Back to News

TRUSTA upgrades AI Scaler Toolkit with SSD-backed context caching

Metallic chip, green glass memory slab and storage block connected by gold light.
AI-generated editorial illustration

ADATA’s TRUSTA storage brand announced an upgraded AI Scaler Toolkit on November 18 that moves cached AI context between GPU memory, system memory and enterprise SSDs.

The SC26 announcement, published at 12:46 a.m. Eastern, describes integration with vLLM’s KV-cache offloading. TRUSTA says storing that cache on SSD allows previously processed context to be reloaded rather than recomputed, including after server restarts.

The toolkit also supports llama.cpp and Strata, with controls for allocating GPU, DRAM and SSD resources. Its Strata integration uses Intel AMX acceleration for prompt processing on Xeon processors.

TRUSTA reports approximately 25% faster prefill in one multi-turn test, but the conditions matter: it used a Xeon w5-3535X, one RTX 6000 Ada GPU and a mixture-of-experts model, comparing its AMX-enabled version with upstream Strata 0.2.11. The same internal tests found about 11% improvement at an 8,000-token input and no significant difference above roughly 12,000 tokens. These are vendor measurements, not independent end-to-end benchmarks.

The upgrade targets an increasingly practical inference constraint: long conversations and documents consume memory even when a model itself fits on the available hardware.

Sources and citations

Revision history

Published .