Python Model Report
TL;DR
This saved report records the listed models on 20 primary Python prompts. The recorded average first-answer latency is n/a. Marker coverage counts expected text in the response; it is not a code execution or correctness score. First-versus-later prompt differences also do not establish a controlled cold-start effect.
This report records response times, output speed and expected text markers for the listed Python prompts. First and later requests are shown separately; request order alone does not prove a cold-start effect.
Measurements in this report
These cards summarize only the models and prompts present in this saved report.
Primary Prompt Latency
Average time to answer the main Python task prompt. Lower is better.
No rows were available for this chart.
Follow-up Prompt Latency
Average time to answer the follow-up summary request. Lower is better.
No rows were available for this chart.
Primary Prompt Throughput
Average tokens per second while answering the main Python tasks. Higher is better.
No rows were available for this chart.
Follow-up Prompt Throughput
Average tokens per second on the summary requests. Higher is better.
No rows were available for this chart.
Primary Marker Coverage
Average marker coverage on the full Python task prompts. Higher is better.
No rows were available for this chart.
Follow-up Marker Coverage
Average marker coverage on the follow-up summary prompts. Higher is better.
No rows were available for this chart.
First vs Later Latency Difference
Positive values mean the first primary prompt was slower than the later-request average. Negative values mean the first prompt was faster. Prompt length and task difficulty may also differ.
No delta rows were available for this chart.
First vs Later Throughput Difference
Positive values mean token generation sped up after the first primary prompt. Negative values mean throughput dropped on later prompts.
No delta rows were available for this chart.