A3B cross-machine — Predator side
Predator (GTX 1070) running the same A3B IQ2-XXS at ~3.9 tok/s mean, p50 102s — the larger half of the pair.
What Janie says
Predator is the newer of the two consumer-GPU machines tested — an Acer Predator gaming laptop with a six-gigabyte GTX 1060 and twenty-eight gigabytes of system RAM. On the same Qwen3-30B-A3B model, the same llama.cpp engine, the same prompts as the Pavilion run sitting next to this one in the catalogue, it ran half as fast: four warm tokens per second on the hard prompt against Pavilion's eight. Better hardware, same model, worse number.
The cause is almost certainly an offload-config mistake on our end. --n-gpu-layers 99 lets llama.cpp distribute layers awkwardly across a six-gigabyte VRAM budget; an explicit partition (--n-gpu-layers 32 or whichever count fills VRAM without spilling) should let Predator pull ahead, as llama-bench's prompt-processing number predicts: 40 tok/s on Predator versus 6 on Pavilion is a 6× compute-throughput advantage on the GPU side. The retune is the obvious next bench. We're publishing the un-retuned numbers now, with the methodology footnote naming the gap, because the un-retuned number is what most readers will see when they first try this hardware.
Methodology
See A3B_AND_CPU_OVERNIGHT_2026-05-05
for the full procedure.
Reproducible at git SHA ddbaaf46.
Results
| Cell | tok/s mean | tok/s p50 | tok/s p95 | duration p50 | calls |
|---|---|---|---|---|---|
| qwen3:30b-a3b-iq2m-think500 | 3.9 | 4.0 | 4.1 | 1m41s | 14 |
tokens per second — mean · p50 · p95
Cold start vs warm
Cold-start measurements are the first call into a model after it loads from disk; warm calls are everything after. The ratio shows how much of the deployment’s wall-time cost is one-time vs steady-state.
| Cell | cold n | cold tok/s | cold p50 | warm n | warm tok/s | warm p50 | warm/cold |
|---|---|---|---|---|---|---|---|
| predator:llamacpp:qwen3:30b-a3b-i… | 3 | 3.5 | 1m37s | 9 | 4.0 | 1m46s | 1.16× |
By prompt difficulty
Tokens per second by prompt class. hello is a trivial
one-line prompt; P-MEDIUM and P-HARD are the
deeper questions in the suite. The shape of the gap tells you whether
the model is bottlenecked on parsing or on generation.
| Cell | hello | P-MEDIUM | P-HARD |
|---|---|---|---|
| predator:llamacpp:qwen3:30b-a3b-i… | 3.6 tok/s 16.1s · n=4 | 4.0 tok/s 1m41s · n=4 | 4.0 tok/s 3m16s · n=4 |
Reasoning vs answer
Thinking models split their output into a hidden reasoning trace and a visible answer. The ratio shows how much of the budget the model spent thinking vs answering.
| Cell | reasoning chars | answer chars | reasoning / answer |
|---|---|---|---|
| predator:llamacpp:qwen3:30b-a3b-i… | 1081 | 835 | 1.29× |
Per-call timeline
Every call placed during this run, in order, colored by phase. Width is proportional to the call’s share of the cell’s wall-time. Hover any segment for the prompt id and tok/s.
Raw data
Every run gets its JSONL, log, summary, and metadata published. Clone the archive; re-run it; tell us where we got it wrong.
Cite
Margetic, S. et al. (2026). benchmarks.weeyuga.com/benchmarks/5fb2913d.html Public benchmarks of the Weeyuga cluster. Run id: 5fb2913d-6500-4ecf-9e97-d43f7dd61145. SHA ddbaaf46.