llama.cpp adds FP8 cache support and fixes Muse Spark structured replies in November 11 prereleases
The llama.cpp project has published prerelease builds that add scaled FP8 key-value cache support and repair a structured-output problem affecting Muse Spark. GitHub lists build b12274 at 02:52 on November 11 and b12282 at 03:31. Both are explicitly marked prereleases, rather than stable releases. Build b12282
The cache change supports OCP-FP8 storage with optional static per-tensor scaling. Its merged pull request describes graph-level scaling shared across flash-attention and standard attention paths, and adds end-to-end checks for conversion, loading, context shifts and cache correctness. The author identifies NVIDIA’s Mistral-Medium-3.5-128B-NVFP4 checkpoint as an example using a non-unit-scaled FP8 cache. Implementation details
There are boundaries to that support. The b12274 notes exclude models that consume the cache directly and record exclusions for OpenVINO tests and certain multi-GPU configurations. Those test exclusions should not be interpreted as proof that every excluded configuration fails, but neither do they establish universal compatibility. The notes also include handling for BF16 scale values and Gemma 4 Assistant. Build b12274
A separate change targets applications expecting a JSON-schema-constrained answer from Muse Spark. According to the contributor, its parser previously built a grammar only when tools were supplied, allowing the requested response format to go unapplied or encounter a server grammar error. The fix applies a grammar to the final response while leaving reasoning unrestricted. Two chat tests passed after the change, but the contributor explicitly says it was not tested on the real model. Raw user-supplied grammars are unchanged. Muse Spark fix
For self-hosting teams, the practical distinction is between adding a supported execution path and proving a production benefit. Neither change, by itself, establishes a measured speedup, a specific reduction in total GPU memory or complete schema reliability on an actual deployment. A comparison should hold the model and workload constant, check output validity and task quality, and record the hardware and cache configuration. These prereleases provide concrete changes to evaluate, with clearly documented reasons to avoid treating the release titles as blanket performance guarantees.
Sources and citations
Revision history
Published .