Engine comparison — Flash-Next on llama.cpp vs Strata
Measured 2026-10-31 on the bench machine. Same model file, same prompts, same instrument, one run each. This page holds the numbers; strata_trial_plan holds the trial process and the verdict.
Why this was measured
Flash-Next (IQ3_XXS, 75.8 GB) is a 125 B-parameter mixture-of-experts model that does not fit in 24 GB of VRAM, so most of its experts live in system RAM. Under llama.cpp that made it very slow to read a prompt, and prompt reading is what an agent loop does most of the time: every turn carries the MCP tool schemas.
The question was whether Strata — a llama.cpp fork built for this one architecture — actually removes that cost, or whether the claim was marketing.
The machine
OS / kernel Arch Linux, 7.2.11-zen1
CPU AMD Ryzen 5 5500, 6 cores / 12 threads
GPU 2 × RTX 3060 12 GB (24 GB total), driver 615.78.08
RAM 62 GB, 3 GB swap
Models /llm on NVMe
The instrument
tools/engine_bench.py in this repository. It sends streaming /v1/chat/completions requests at three prompt sizes and records time to first token, prefill rate and decode rate.
The three prompts are built from the same source paragraph, so a naive benchmark shares a long prefix between them and any engine with prefix caching silently skips part of the work.
That happened on the first run — llama.cpp logged 5,828 tokens evaluated for an 8,424-token prompt, i.e. 2,596 tokens reused from the previous request, and reported a TTFT no user would see on a cold prompt.
The harness now puts a random identifier on the first line of every prompt, which defeats prefix reuse, and the numbers below are all from that corrected run.
Every figure was cross-checked against the engine’s own log, not just the harness. Both agreed to within 1–2%.
Results (Strata is 3.6× faster)
Prompt sizes are the token counts each engine actually reported. They agree across engines to within a few tokens, which confirms the inputs were identical.
llama.cpp build-cuda-new: 16.7 tokens/s output
Strata 0.2.8, engine 0.1.54 (cuda13): 59.8 tokens/s output
