# Engine comparison — Flash-Next on llama.cpp vs Strata ... a HUGE speed increase!

**URL:** <https://discourse.homelabbrisbane.com.au/t/engine-comparison-flash-next-on-llama-cpp-vs-strata-a-huge-speed-increase/570>\
**Category:** General\
**Created:** [October 31, 2026, 4:27am UTC](https://discourse.homelabbrisbane.com.au/t/engine-comparison-flash-next-on-llama-cpp-vs-strata-a-huge-speed-increase/570 "2026-10-31T04:27:00Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![techman](https://discourse.homelabbrisbane.com.au/user_avatar/discourse.homelabbrisbane.com.au/techman/32/45_2.png) [@techman](https://discourse.homelabbrisbane.com.au/u/techman)\
**Post date:** [October 31, 2026, 4:27am UTC](https://discourse.homelabbrisbane.com.au/t/engine-comparison-flash-next-on-llama-cpp-vs-strata-a-huge-speed-increase/570/1 "2026-10-31T04:27:00Z")

</div>

#### Engine comparison — Flash-Next on llama.cpp vs Strata

Measured 2026-10-31 on the bench machine. Same model file, same prompts, same instrument, one run each. This page holds the numbers; strata\_trial\_plan holds the trial process and the verdict.

#### Why this was measured

Flash-Next (IQ3\_XXS, 75.8 GB) is a 125 B-parameter mixture-of-experts model that does not fit in 24 GB of VRAM, so most of its experts live in system RAM. Under llama.cpp that made it very slow to read a prompt, and prompt reading is what an agent loop does most of the time: every turn carries the MCP tool schemas.

The question was whether Strata — a llama.cpp fork built for this one architecture — actually removes that cost, or whether the claim was marketing.

#### The machine

OS / kernel Arch Linux, 7.2.11-zen1  
CPU AMD Ryzen 5 5500, 6 cores / 12 threads  
GPU 2 × RTX 3060 12 GB (24 GB total), driver 615.78.08  
RAM 62 GB, 3 GB swap  
Models /llm on NVMe

#### The instrument

tools/engine\_bench.py in this repository. It sends streaming /v1/chat/completions requests at three prompt sizes and records time to first token, prefill rate and decode rate.

The three prompts are built from the same source paragraph, so a naive benchmark shares a long prefix between them and any engine with prefix caching silently skips part of the work.

That happened on the first run — llama.cpp logged 5,828 tokens evaluated for an 8,424-token prompt, i.e. 2,596 tokens reused from the previous request, and reported a TTFT no user would see on a cold prompt.

The harness now puts a random identifier on the first line of every prompt, which defeats prefix reuse, and the numbers below are all from that corrected run.

Every figure was cross-checked against the engine’s own log, not just the harness. Both agreed to within 1–2%.

#### Results (Strata is 3.6× faster)

Prompt sizes are the token counts each engine actually reported. They agree across engines to within a few tokens, which confirms the inputs were identical.

llama.cpp build-cuda-new: **16.7 tokens/s output**  
Strata 0.2.8, engine 0.1.54 (cuda13): **59.8 tokens/s output**

---

<div class="post-metadata">

**Author:** ![techman](https://discourse.homelabbrisbane.com.au/user_avatar/discourse.homelabbrisbane.com.au/techman/32/45_2.png) [@techman](https://discourse.homelabbrisbane.com.au/u/techman)\
**Post date:** [October 31, 2026, 7:08am UTC](https://discourse.homelabbrisbane.com.au/t/engine-comparison-flash-next-on-llama-cpp-vs-strata-a-huge-speed-increase/570/2 "2026-10-31T07:08:12Z")

</div>

My Strata web info page:

 ![strata-20261031-180626](https://discourse.homelabbrisbane.com.au/uploads/default/original/1X/3f58baf1f3158a761266c80c44f3796f0316f630.png)