Python Model Report
TL;DR
This saved report records Qwen2.5 Coder 0.5B on 20 primary Python prompts. The recorded average first-answer latency is 9.4 s. Marker coverage counts expected text in the response; it is not a code execution or correctness score. First-versus-later prompt differences also do not establish a controlled cold-start effect.
This report records response times, output speed and expected text markers for the listed Python prompts. First and later requests are shown separately; request order alone does not prove a cold-start effect.
Measurements in this report
These cards summarize only the models and prompts present in this saved report.
Qwen2.5 Coder 0.5B
9.4 s
Large model with the quickest average first-answer pass.
Qwen2.5 Coder 0.5B
3.6 s
Quickest summary-request average once the model already had context.
Qwen2.5 Coder 0.5B
81%
Average expected text-marker coverage for the primary responses.
Qwen2.5 Coder 0.5B
60%
Expected text-marker coverage in the follow-up responses; this does not measure code execution.
Qwen2.5 Coder 0.5B
-3.5 s
Latency difference between the first primary request and the later requests in this saved sequence.
Qwen2.5 Coder 0.5B
-0.15 tok/s
Output-speed difference between the first primary request and later requests in this saved sequence.
Primary Prompt Latency
Average time to answer the main Python task prompt. Lower is better.
Qwen2.5 Coder 0.5B
Large
9.4 s
Follow-up Prompt Latency
Average time to answer the follow-up summary request. Lower is better.
Qwen2.5 Coder 0.5B
Large
3.6 s
Primary Prompt Throughput
Average tokens per second while answering the main Python tasks. Higher is better.
Qwen2.5 Coder 0.5B
Large
20.88 tok/s
Follow-up Prompt Throughput
Average tokens per second on the summary requests. Higher is better.
Qwen2.5 Coder 0.5B
Large
22.08 tok/s
Primary Marker Coverage
Average marker coverage on the full Python task prompts. Higher is better.
Qwen2.5 Coder 0.5B
Large
81%
Follow-up Marker Coverage
Average marker coverage on the follow-up summary prompts. Higher is better.
Qwen2.5 Coder 0.5B
Large
60%
First vs Later Latency Difference
Positive values mean the first primary prompt was slower than the later-request average. Negative values mean the first prompt was faster. Prompt length and task difficulty may also differ.
Later prompts got faster
Later prompts got slower
Qwen2.5 Coder 0.5B
Large
-3.5 s
Later prompts got slower
First vs Later Throughput Difference
Positive values mean token generation sped up after the first primary prompt. Negative values mean throughput dropped on later prompts.
Later prompts got faster
Later prompts got slower
Qwen2.5 Coder 0.5B
Large
-0.15 tok/s
Later prompts got slower
Model Overview
This table separates speed, output rate, primary marker coverage, usable-response checks and first-versus-later latency for each model.
| Model | Primary Avg | Follow-up Avg | Primary tok/s | Follow-up tok/s | Primary Marker Coverage | Usable | First/Later Difference |
|---|---|---|---|---|---|---|---|
| Qwen2.5 Coder 0.5B Large | 9.4 s | 3.6 s | 20.88 tok/s | 22.08 tok/s | 81% | 20/20 | -3.5 s |
Python Task-Type Heatmap
Values show primary-prompt marker coverage by task type. Green means the model hit more of the requested Python requirements for that category.
| Model | analysis | async | cli | concurrency | config | debugging | file_io | filesystem | http | logging | package | parsing | refactor | sqlite | tests | typing | validation | web |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Qwen2.5 Coder 0.5B Large |
75%
|
100%
|
100%
|
100%
|
25%
|
75%
|
75%
|
50%
|
100%
|
75%
|
75%
|
100%
|
75%
|
75%
|
100%
|
75%
|
75%
|
100%
|