Hello check
Complete
Exact prompt succeeded and returned a clean greeting.
CPU-only lane covering the Pavilion hello check plus the five-question, twenty-question, and ten-question suites. GTX 1050 offload stayed disabled for this pass.
All finished artifacts are in this report. The page calls out exactly which planned phases completed and which ones still did not produce any files.
Exact prompt succeeded and returned a clean greeting.
All five tasks finished and produced scored artifacts.
All twenty primary and follow-up prompts finished.
All ten real-context tasks finished and were pulled into the report.
No output files were produced for the long-context append lane.
These cards surface the highest-signal latency, throughput, and formatting stats for this run so you can compare qwen2.5-coder:14b against other hosts.
Exact prompt: hi can you help me?
90% marker coverage
3.11 tok/s
3.05 tok/s
Every primary prompt returned a non-empty answer.
Follow-ups were much more obedient than primaries.
2.88 tok/s
Primary prompts that returned a usable answer.
This is the exact smoke-test prompt that kicked off the benchmark lane.
These are the short shell, ops, planning, and debugging tasks. Lower is better.
Primary-prompt durations for the full Python suite, sorted from slowest to fastest.
Higher is better. This shows how much of each prompt's requested structure {model_display} actually hit on the first try.
Quick read: great on planning and SSH triage, shakier on exact shell formatting and the IPv4 implementation details.
| Task | Primary | Follow-up | Primary tok/s | Marker hit | Format OK | Usable |
|---|---|---|---|---|---|---|
| Disk Guard Scriptshell | 33.9s | n/a | 3.35 tok/s | 75% | No | Yes |
| IPv4 Validatorpython | 66.9s | n/a | 2.99 tok/s | 100% | Yes | Yes |
| Nginx Safe Reloadops | 14.9s | n/a | 3.26 tok/s | 75% | Yes | Yes |
| YAML Validator Planplanning | 217.5s | n/a | 2.93 tok/s | 100% | Yes | Yes |
| SSH Lockout Triagedebugging | 67.5s | n/a | 2.97 tok/s | 100% | Yes | Yes |
This is the practical coding workload, so each row shows how the model handled the primary and follow-up formats.
| Task | Primary | Follow-up | Primary tok/s | Marker hit | Format OK | Usable |
|---|---|---|---|---|---|---|
| CSV Parserparsing | 20.0s | 53.0s | 3.23 tok/s | 75% | No | Yes |
| File Scannerfile_io | 32.7s | 41.3s | 3.09 tok/s | 100% | No | Yes |
| CLI Argumentscli | 36.2s | 48.9s | 3.12 tok/s | 100% | No | Yes |
| Typed Dataclasstyping | 18.9s | 29.9s | 3.14 tok/s | 100% | No | Yes |
| Pytest Fixturetests | 28.2s | 38.5s | 3.06 tok/s | 100% | No | Yes |
| Async Fetchasync | 34.1s | 28.4s | 3.13 tok/s | 100% | No | Yes |
| HTTP Retryhttp | 52.6s | 44.6s | 3.05 tok/s | 75% | No | Yes |
| JSON Validationvalidation | 51.2s | 46.7s | 3.10 tok/s | 100% | No | Yes |
| SQLite Storesqlite | 68.2s | 54.6s | 3.04 tok/s | 100% | No | Yes |
| FastAPI Handlerweb | 18.8s | 25.4s | 3.15 tok/s | 100% | No | Yes |
| Config Dataclassconfig | 44.5s | 32.8s | 2.97 tok/s | 75% | No | Yes |
| Logging Setuplogging | 49.3s | 56.9s | 3.06 tok/s | 75% | No | Yes |
| Thread Poolconcurrency | 24.6s | 26.9s | 3.14 tok/s | 100% | No | Yes |
| Package Layoutpackage | 28.1s | 40.5s | 3.12 tok/s | 100% | No | Yes |
| Debug Stacktracedebugging | 14.0s | 21.2s | 3.19 tok/s | 50% | No | Yes |
| Refactor Splitrefactor | 24.6s | 45.8s | 3.06 tok/s | 50% | No | Yes |
| CSV Summaryanalysis | 43.7s | 36.9s | 3.11 tok/s | 75% | No | Yes |
| Pathlib Cleanerfilesystem | 25.9s | 55.4s | 3.12 tok/s | 100% | No | Yes |
| Pydantic Modelvalidation | 16.0s | 26.1s | 3.18 tok/s | 75% | No | Yes |
| Regex Log Parserparsing | 32.5s | 51.3s | 3.12 tok/s | 75% | No | Yes |
This groups the finished 20-question suite by task family so you can see where the model stayed sharp and where it softened.
| Category | Tasks | Avg primary | Avg primary tok/s | Avg primary hit | Avg follow-up hit |
|---|---|---|---|---|---|
| analysis | 1/1 | 43.7s | 3.11 tok/s | 75% | 0% |
| async | 1/1 | 34.1s | 3.13 tok/s | 100% | 0% |
| cli | 1/1 | 36.2s | 3.12 tok/s | 100% | 0% |
| concurrency | 1/1 | 24.6s | 3.14 tok/s | 100% | 100% |
| config | 1/1 | 44.5s | 2.97 tok/s | 75% | 100% |
| debugging | 1/1 | 14.0s | 3.19 tok/s | 50% | 100% |
| file_io | 1/1 | 32.7s | 3.09 tok/s | 100% | 100% |
| filesystem | 1/1 | 25.9s | 3.12 tok/s | 100% | 100% |
| http | 1/1 | 52.6s | 3.05 tok/s | 75% | 0% |
| logging | 1/1 | 49.3s | 3.06 tok/s | 75% | 0% |
| package | 1/1 | 28.1s | 3.12 tok/s | 100% | 100% |
| parsing | 2/2 | 26.3s | 3.17 tok/s | 75% | 100% |
| refactor | 1/1 | 24.6s | 3.06 tok/s | 50% | 100% |
| sqlite | 1/1 | 68.2s | 3.04 tok/s | 100% | 0% |
| tests | 1/1 | 28.2s | 3.06 tok/s | 100% | 100% |
| typing | 1/1 | 18.9s | 3.14 tok/s | 100% | 100% |
| validation | 2/2 | 33.6s | 3.14 tok/s | 88% | 100% |
| web | 1/1 | 18.8s | 3.15 tok/s | 100% | 100% |
These are the heavier repository-shaped prompts. Lower is better.
Higher is better. This shows how much of the requested triage, test, planning, and review structure landed on the first response.
This is the more realistic repo-style suite. It is the best signal for whether the model stays reliable once prompts stop looking like toy exercises.
| Task | Primary | Follow-up | Primary tok/s | Marker hit | Format OK | Usable |
|---|---|---|---|---|---|---|
| 88.6s | 37.9s | 2.95 tok/s | 88% | No | Yes | |
| Board Snapshot Regression Testtests | 111.5s | 41.1s | 2.89 tok/s | 86% | No | Yes |
| Lane Config Patch Planplanning | 160.5s | 52.8s | 2.86 tok/s | 83% | No | Yes |
| API Token Audit Regression Testtests | 194.6s | 39.8s | 2.85 tok/s | 86% | No | Yes |
| Announcements State Sync Reviewreview | 179.8s | 34.3s | 2.86 tok/s | 83% | No | Yes |
| Feature Flag Lifecycle Testtests | 182.1s | 50.3s | 2.87 tok/s | 86% | No | Yes |
| Task Bulk Job Debug Packetdebugging | 106.2s | 51.3s | 2.88 tok/s | 67% | No | Yes |
| User Preferences Contract Testtests | 199.2s | 34.0s | 2.85 tok/s | 86% | No | Yes |
| Orchestration Timeline Forensicsforensics | 89.8s | 43.9s | 2.87 tok/s | 100% | No | Yes |
| 98.2s | 37.8s | 2.89 tok/s | 83% | No | Yes |
qwen2.5-coder:14b is not meant to replace the largest workers, but the telemetry shows it handled this stack end to end. It averaged 33.2s per primary prompt, pushed about 3.11 tok/s, and landed 20/20 usable primaries. Follow-up formatting remained the safety net: 0/20 primary format passes versus 13/20 on the follow-ups.