{"schemaVersion":2,"suite":"cheap-model-tests","evidenceEndpoint":{"readOnly":true,"machineReadable":true},"runAt":"2026-06-16T20:31:30.728Z","repositorySource":"frontend/src/lib/seoTests.ts","test":{"slug":"code-snippet-fibonacci","title":"Writing a Code Snippet","shortTitle":"Code Snippet","category":"Code Generation","icon":"{ }","summary":"A focused coding task: produce a correct, efficient, 0-indexed iterative Fibonacci function in Python — and nothing but the code.","criteria":["Algorithm is correct (fib(0)=0, fib(1)=1, fib(n)=fib(n-1)+fib(n-2)).","Uses an efficient O(n) iterative approach (no naive recursion).","Handles the n=0 edge case correctly.","Returns only the code, with no surrounding explanation or stray reasoning."],"metaTitle":"Best Cheap AI for Writing Code Snippets (Under $3/M Tokens) — Tested","metaDescription":"We sent the same Python coding task to every budget LLM under $3 per million tokens and measured real speed, cost, and accuracy. See which cheap model writes the cleanest code.","keywords":["cheap AI for coding","best budget LLM for code","AI code generation benchmark","GPT-5.4 Nano vs Llama coding","cheapest coding AI 2026","LLM code snippet comparison"],"prompt":"Write a Python function called `fib(n)` that returns the n-th Fibonacci number (0-indexed, so fib(0)=0 and fib(1)=1). Use an efficient iterative approach and handle n=0 correctly. Return only the code inside a single code block, with no explanation.","results":[{"modelId":"gpt-5.4-nano","modelName":"GPT-5.4 Nano","provider":"OpenAI","outputPricePerM":1.25,"output":"```python\ndef fib(n):\n    if n < 0:\n        raise ValueError(\"n must be non-negative\")\n    if n == 0:\n        return 0\n    a, b = 0, 1\n    for _ in range(1, n):\n        a, b = b, a + b\n    return b\n```","status":"completed","finishReason":"","latencyMs":1742,"tokensIn":66,"tokensOut":77,"tokensPerSecond":44.2,"costUsd":0.00010944999999999999,"accuracy":100,"accuracyNote":"Correct iterative algorithm; fib(0)=0 and fib(1)=1 verified. Clean, code-only, and adds a sensible negative-input guard.","receiptVersion":1,"promptHash":"d7291bb1d553582240d939b4f9968735f29004f81c9c6136d1f60876605339c1","rawOutputHash":"a57327357e981f19bce34cfb0c23297cc50ded8b330f716c24f57ccfa80bd655","graderVersion":"claude-agent-grader-v1","solverVersion":null,"harnessVersion":null,"validationState":"recorded","validationFlags":["prompt-hash=sha256","raw-output-hash=sha256","status=completed","accuracy=range-valid","latency=recorded","tokens=recorded","cost=recorded","grader-score=recorded","reasoning-leak=absent","code-fence=present"],"correctionVersion":"correction-v1"},{"modelId":"gemini-3.1-flash-lite","modelName":"Gemini 3.1 Flash Lite","provider":"Google","outputPricePerM":1.5,"output":"```python\ndef fib(n):\n    if n <= 0:\n        return 0\n    elif n == 1:\n        return 1\n    a, b = 0, 1\n    for _ in range(2, n + 1):\n        a, b = b, a + b\n    return b\n```","status":"completed","finishReason":"","latencyMs":995,"tokensIn":63,"tokensOut":76,"tokensPerSecond":76.4,"costUsd":0.00012975,"accuracy":97,"accuracyNote":"Correct and code-only. Returns 0 for negative n silently rather than raising, otherwise spot-on.","receiptVersion":1,"promptHash":"d7291bb1d553582240d939b4f9968735f29004f81c9c6136d1f60876605339c1","rawOutputHash":"412402373211028756fd80bb8d4dcb36eccff6f30f720459c6d8b76ce0baf11d","graderVersion":"claude-agent-grader-v1","solverVersion":null,"harnessVersion":null,"validationState":"recorded","validationFlags":["prompt-hash=sha256","raw-output-hash=sha256","status=completed","accuracy=range-valid","latency=recorded","tokens=recorded","cost=recorded","grader-score=recorded","reasoning-leak=absent","code-fence=present"],"correctionVersion":"correction-v1"},{"modelId":"grok-4.3","modelName":"Grok 4.3","provider":"xAI","outputPricePerM":2.5,"output":"<think>\nThe task is: Write a Python function called `fib(n)` that returns the n-th Fibonacci number (0-indexed, so fib(0)=0 and fib(1)=1). Use an efficient iterative approach and handle n=0 correctly. Return ...\n</think>\n```python\ndef fib(n):\n    a, b = 0, 1\n    for _ in range(n):\n        a, b = b, a + b\n    return a\n```","status":"completed","finishReason":"","latencyMs":3305,"tokensIn":63,"tokensOut":95,"tokensPerSecond":28.7,"costUsd":0.00031625,"accuracy":80,"accuracyNote":"The canonical solution is correct, but a (truncated) <think> reasoning block leaked into the output despite the \"only the code\" instruction.","receiptVersion":1,"promptHash":"d7291bb1d553582240d939b4f9968735f29004f81c9c6136d1f60876605339c1","rawOutputHash":"cb0caeb9645e1bae32af74ffcd103c8181c0a774deae8d40a2b64051c67ea769","graderVersion":"claude-agent-grader-v1","solverVersion":null,"harnessVersion":null,"validationState":"recorded","validationFlags":["prompt-hash=sha256","raw-output-hash=sha256","status=completed","accuracy=range-valid","latency=recorded","tokens=recorded","cost=recorded","grader-score=recorded","reasoning-leak=detected","code-fence=present"],"correctionVersion":"correction-v1"},{"modelId":"llama-4-scout","modelName":"Llama 4 Scout","provider":"Groq","outputPricePerM":0.34,"output":"```python\ndef fib(n):\n    if n <= 0:\n        return 0\n    elif n == 1:\n        return 1\n    else:\n        a, b = 0, 1\n        for _ in range(2, n + 1):\n            a, b = b, a + b\n        return b\n```","status":"completed","finishReason":"","latencyMs":344,"tokensIn":70,"tokensOut":75,"tokensPerSecond":218,"costUsd":0.0000332,"accuracy":98,"accuracyNote":"Correct with explicit base cases, code-only. No issues.","receiptVersion":1,"promptHash":"d7291bb1d553582240d939b4f9968735f29004f81c9c6136d1f60876605339c1","rawOutputHash":"4f153c75da4a20c233cecae0c093c4443c29afbcddbfea4cfbff72aa26a10986","graderVersion":"claude-agent-grader-v1","solverVersion":null,"harnessVersion":null,"validationState":"recorded","validationFlags":["prompt-hash=sha256","raw-output-hash=sha256","status=completed","accuracy=range-valid","latency=recorded","tokens=recorded","cost=recorded","grader-score=recorded","reasoning-leak=absent","code-fence=present"],"correctionVersion":"correction-v1"},{"modelId":"gpt-oss-120b","modelName":"GPT-OSS 120B","provider":"Groq","outputPricePerM":0.6,"output":"<think>\nThe user wants a Python function fib(n) that returns n-th Fibonacci number using an efficient iterative approach, handling n=0 correctly. Provide only code inside a single code block, no explanation.\n\nWe must comply. Provide code.\n</think>\n```python\ndef fib(n):\n    if n < 0:\n        raise ValueError(\"n must be a non-negative integer\")\n    a, b = 0, 1\n    for _ in range(n):\n        a, b = b, a + b\n    return a\n```","status":"completed","finishReason":"","latencyMs":394,"tokensIn":131,"tokensOut":117,"tokensPerSecond":297,"costUsd":0.00008985,"accuracy":82,"accuracyNote":"Correct canonical solution with a negative guard, but a <think> block leaked in ahead of the code block.","receiptVersion":1,"promptHash":"d7291bb1d553582240d939b4f9968735f29004f81c9c6136d1f60876605339c1","rawOutputHash":"9bfe8306c29b98bdce17c9ed9ba9c4cf1049a28bab74a0b5100ff111d73bd7f3","graderVersion":"claude-agent-grader-v1","solverVersion":null,"harnessVersion":null,"validationState":"recorded","validationFlags":["prompt-hash=sha256","raw-output-hash=sha256","status=completed","accuracy=range-valid","latency=recorded","tokens=recorded","cost=recorded","grader-score=recorded","reasoning-leak=detected","code-fence=present"],"correctionVersion":"correction-v1"},{"modelId":"gpt-oss-20b","modelName":"GPT-OSS 20B","provider":"Groq","outputPricePerM":0.3,"output":"<think>\nUser wants a Python function fib(n) that returns nth Fibonacci number 0-indexed. Use efficient iterative approach, handle n=0 correctly. Return only code inside a single code block, no explanation.\n\nWe need to output just the code inside a code block. So something like:\n\n```\ndef fib(n):\n    a, b = 0, 1\n    for _ in range(n):\n        a, b = b, a + b\n    return a\n```\n\nEdge cases: fib(0)=0; works: loop 0 times, a=0. fib(1)=1: loop once, a,b -> a=1,b=1, return a=1. Works. Also negative? Not required.\n\nSo output code.\n</think>\n```python\ndef fib(n):\n    a, b = 0, 1\n    for _ in range(n):\n        a, b = b, a + b\n    return a\n```","status":"completed","finishReason":"","latencyMs":576,"tokensIn":131,"tokensOut":205,"tokensPerSecond":355.9,"costUsd":0.00007132499999999999,"accuracy":80,"accuracyNote":"Final code is correct, but the model exposed its <think> reasoning before the code block.","receiptVersion":1,"promptHash":"d7291bb1d553582240d939b4f9968735f29004f81c9c6136d1f60876605339c1","rawOutputHash":"0677c7b2d9fb1de2c1a8337499c71c8635ae1c8b03730c41a502f4e6d50f3b9c","graderVersion":"claude-agent-grader-v1","solverVersion":null,"harnessVersion":null,"validationState":"recorded","validationFlags":["prompt-hash=sha256","raw-output-hash=sha256","status=completed","accuracy=range-valid","latency=recorded","tokens=recorded","cost=recorded","grader-score=recorded","reasoning-leak=detected","code-fence=present"],"correctionVersion":"correction-v1"},{"modelId":"llama-3.3-70b","modelName":"Llama 3.3 70B","provider":"Groq","outputPricePerM":0.79,"output":"```python\ndef fib(n):\n    if n <= 1:\n        return n\n    a, b = 0, 1\n    for _ in range(2, n + 1):\n        a, b = b, a + b\n    return b\n```","status":"completed","finishReason":"","latencyMs":298,"tokensIn":95,"tokensOut":59,"tokensPerSecond":198,"costUsd":0.00010266,"accuracy":99,"accuracyNote":"Correct and elegant (returns n for n<=1). Clean and code-only.","receiptVersion":1,"promptHash":"d7291bb1d553582240d939b4f9968735f29004f81c9c6136d1f60876605339c1","rawOutputHash":"d25a76102a8f210d19b5e4a03fd019813491443e11f3afebd03780c35e682936","graderVersion":"claude-agent-grader-v1","solverVersion":null,"harnessVersion":null,"validationState":"recorded","validationFlags":["prompt-hash=sha256","raw-output-hash=sha256","status=completed","accuracy=range-valid","latency=recorded","tokens=recorded","cost=recorded","grader-score=recorded","reasoning-leak=absent","code-fence=present"],"correctionVersion":"correction-v1"},{"modelId":"llama-3.1-8b","modelName":"Llama 3.1 8B","provider":"Groq","outputPricePerM":0.08,"output":"```python\ndef fib(n):\n    if n <= 1:\n        return n\n    a, b = 0, 1\n    for i in range(2, n + 1):\n        a, b = b, a + b\n    return b\n```","status":"completed","finishReason":"","latencyMs":144,"tokensIn":95,"tokensOut":59,"tokensPerSecond":409.7,"costUsd":0.00000947,"accuracy":99,"accuracyNote":"Correct, clean, code-only — and the fastest response in this test.","receiptVersion":1,"promptHash":"d7291bb1d553582240d939b4f9968735f29004f81c9c6136d1f60876605339c1","rawOutputHash":"116ae97fa49c8db2ecc0a76bf31769aef837266ce264e4bc80ff2a30b6543f3e","graderVersion":"claude-agent-grader-v1","solverVersion":null,"harnessVersion":null,"validationState":"recorded","validationFlags":["prompt-hash=sha256","raw-output-hash=sha256","status=completed","accuracy=range-valid","latency=recorded","tokens=recorded","cost=recorded","grader-score=recorded","reasoning-leak=absent","code-fence=present"],"correctionVersion":"correction-v1"},{"modelId":"qwen3-32b","modelName":"Qwen 3 32B","provider":"Groq","outputPricePerM":0.59,"output":"<think>\nOkay, I need to write a Python function called fib(n) that returns the n-th Fibonacci number. Let me think about how to approach this.\n\nFirst, the problem says it's 0-indexed. So fib(0) is 0, fib(1) is 1. For example, fib(2) should be 1, fib(3) is 2, and so on. Got it.\n\nThe user wants an efficient iterative approach. So recursion isn't the way to go here because for larger n, recursion would be inefficient and cause stack overflows. Iterative method is better for both time and space complexity.\n\nLet me plan the steps. For n=0, return 0. For n=1, return 1. For higher numbers, calculate iteratively.\n\nMaybe start with two variables, a and b, representing fib(0) and fib(1). Then for each step from 2 up to n, compute the next Fibonacci number by adding the previous two. Let me think:\n\nInitialize a = 0 (fib(0)), b = 1 (fib(1)). Then for i in range(2, n+1), compute c = a + b. Then update a and b: a becomes the old b, and b becomes c. Wait, but how does this work for each step?\n\nWait, let's take an example. Let's say n is 2. For i=2, we compute 0+1=1. So the result is 1. Which is correct. For n=3, the loop runs once (i=2 and i=3?), depends on the loop setup. Wait, maybe I need to loop from 2 to n, inclusive?\n\nAlternatively, maybe the loop should run n-1 times. Let me think: if n is 0, return 0. If n is 1, return 1. Else, for n >= 2, we can iterate (n-1) times. Wait, let's think of it as starting with a=0, b=1. For each step, we calculate the next number. So for n=2, we need one step: a + b = 0 + 1 = 1. For n=3, two steps: first step gives 1 (for n=2), second step gives 1 + 1 = 2 (for n=3). So if n is 4, three steps: 0, 1 -> 1, 1 -> 1, 2 -> 2, 3. The result is 3. Which is correct.\n\nSo for the general case where n >= 1, we need to perform (n-1) iterations. Because for each iteration, we're moving from the i-th to (i+1)-th Fibonacci number.\n\nLet me structure this:\n\nif n is 0: return 0\n\nelse:\n\n   a, b = 0, 1\n\n   for i in range(2, n+1):\n\n       c = a + b\n\n       a = b\n\n       b = c\n\n   return b\n\nWait, let's check for n=1. If n is 1, the loop would be range(2, 2), which doesn't execute. So returns b which is 1. Correct.\n\nn=2: loop runs once (i=2). c=0+1=1. a becomes 1, b becomes 1. Return b=1. Correct.\n\nn=3: loop runs for i=2 and 3? Wait, range(2, n+1) when n=3 is range(2,4), which is 2 and 3. Wait, but for n=3, how many steps? Let me see. Starting with a=0, b=1. Then for i in 2 to n (inclusive), which for n=3 is two iterations?\n\nWait wait, for n=3:\n\nInitially a=0, b=1 (for 0 and 1). Then for i=2, which would compute fib(2) as a + b = 1. Then a becomes 1 (previous b) and b becomes 1 (new value). Then i=3, compute a + b = 1 + 1 = 2. a becomes 1, b becomes 2. So return b is 2. Which is correct for fib(3) = 2. So the loop for n=3 runs for two steps (i=2 and i=3). So the number of iterations is (n - 1) steps. Because for n >= 1, the loop runs from 2 to n inclusive. So for n=1, no steps. For n=2, one step. For n=3, two steps. Which seems right.\n\nSo the code structure would be:\n\ndef fib(n):\n\n    if n == 0:\n\n        return 0\n\n    a, b = 0, 1\n\n    for i in range(2, n + 1):\n\n        c = a + b\n\n        a = b\n\n        b = c\n\n    return b\n\nTesting this for various values. Let's see:\n\nfib(0) → 0. Correct.\n\nfib(1) → 1. Correct.\n\nfib(2): loop runs once. a=0, b=1 → c=1 → a becomes 1, b becomes 1 → return 1.\n\nfib(3): loop runs twice. First iteration (i=2): same as before. a=1, b=1. Second iteration (i=3): c=1+1=2. a becomes 1, b becomes 2. Return 2. Correct.\n\nfib(4): returns 3. Let's check. After first iteration (i=2): a=1, b=1. i=3: a=1, b=2. i=4: c=1+2=3. a=2, b=3. return 3. Correct.\n\nThis seems to work.\n\nWhat about n=5? Let's compute. The sequence is 0,1,1,2,3,5. For n=5, should return 5. Let's simulate.\n\nLoop runs from 2 to 5: i=2,3,4,5 (4 iterations).\n\nStart with a=0, b=1.\n\ni=2:\n\nc=0+1=1 → a=1, b=1.\n\ni=3:\n\nc=1+1=2 → a=1, b=2.\n\ni=4:\n\nc=1+2=3 → a=2, b=3.\n\ni=5:\n\nc=2+3=5 → a=3, b=5. return 5. Correct.\n\nLooks good.\n\nNow, what if n is 0? The code handles that. So the code seems correct.\n\nAnother test case: n=0 returns 0. What if someone passes a negative number? The problem says to handle n=0 correctly, but perhaps the function should assume n is non-negative. Since the problem says nothing about invalid inputs, perhaps no need to handle them. So assuming that n is a non-negative integer.\n\nSo the code is correct. The code handles all cases.\n\nThe code is efficient: O(n) time, O(1) space. Which is the best possible for this problem without using closed-form formulas (which would be O(1) time but might have precision issues for large n).\n\nTherefore, the Python function as written should work.\n</think>\n\n```python\ndef fib(n):\n    if n == 0:\n        return 0\n    a, b = 0, 1\n    for _ in range(2, n + 1):\n        a, b = b, a + b\n    return b\n```","status":"completed","finishReason":"","latencyMs":4658,"tokensIn":68,"tokensOut":1645,"tokensPerSecond":353.2,"costUsd":0.0009902700000000001,"accuracy":68,"accuracyNote":"The final code is correct, but the model dumped a ~1,650-token <think> monologue into the output — a major breach of the code-only instruction and the priciest run here.","receiptVersion":1,"promptHash":"d7291bb1d553582240d939b4f9968735f29004f81c9c6136d1f60876605339c1","rawOutputHash":"dde27b8a27cec3bbf94b0170dccb6a9b6a685bf200e497a6cbbc5099627535cb","graderVersion":"claude-agent-grader-v1","solverVersion":null,"harnessVersion":null,"validationState":"recorded","validationFlags":["prompt-hash=sha256","raw-output-hash=sha256","status=completed","accuracy=range-valid","latency=recorded","tokens=recorded","cost=recorded","grader-score=recorded","reasoning-leak=detected","code-fence=present"],"correctionVersion":"correction-v1"},{"modelId":"deepseek-v4-flash","modelName":"DeepSeek V4 Flash","provider":"DeepSeek","outputPricePerM":0.28,"output":"<think>\nWe need to write a Python function fib(n) that returns the n-th Fibonacci number, 0-indexed. Efficient iterative approach, handle n=0 correctly. Return only code in a single code block, no explanation.\n\nWe'll write a simple iterative solution: start a=0, b=1, loop n times, update a,b. For n=0, return a (0). For n=1, after 1 iteration a becomes 1, b becomes 1? Actually careful: typical iterative:\n\ndef fib(n):\n    a, b = 0, 1\n    for _ in range(n):\n        a, b = b, a + b\n    return a\n\nTest: n=0: loop runs 0 times, return a=0. n=1: loop runs 1: a=1, b=1; return a=1. Good. That's efficient O(n), O(1) space. Also handle negative? Not required.\n\nWe'll output the code in a code block.\n</think>\n```python\ndef fib(n):\n    a, b = 0, 1\n    for _ in range(n):\n        a, b = b, a + b\n    return a\n```","status":"completed","finishReason":"","latencyMs":2860,"tokensIn":64,"tokensOut":254,"tokensPerSecond":88.8,"costUsd":0.00008007999999999999,"accuracy":80,"accuracyNote":"Correct canonical solution, but leaked a <think> reasoning block before the code.","receiptVersion":1,"promptHash":"d7291bb1d553582240d939b4f9968735f29004f81c9c6136d1f60876605339c1","rawOutputHash":"115b101e08808e34b5e082f1075d331d63d5f542516c47b4f7f472b7d06c37d9","graderVersion":"claude-agent-grader-v1","solverVersion":null,"harnessVersion":null,"validationState":"recorded","validationFlags":["prompt-hash=sha256","raw-output-hash=sha256","status=completed","accuracy=range-valid","latency=recorded","tokens=recorded","cost=recorded","grader-score=recorded","reasoning-leak=detected","code-fence=present"],"correctionVersion":"correction-v1"},{"modelId":"deepseek-v4-pro","modelName":"DeepSeek V4 Pro","provider":"DeepSeek","outputPricePerM":0.87,"output":"<think>\nWe need to provide a Python function `fib(n)` that returns the n-th Fibonacci number (0-indexed, fib(0)=0, fib(1)=1). Use iterative approach, efficient, handle n=0. Only code in a single code block, no explanation.\n\nSo output:\n\n```python\ndef fib(n):\n    if n == 0:\n        return 0\n    elif n == 1:\n        return 1\n    a, b = 0, 1\n    for _ in range(2, n + 1):\n        a, b = b, a + b\n    return b\n```\n\nThat's it.\n</think>\n```python\ndef fib(n):\n    if n == 0:\n        return 0\n    if n == 1:\n        return 1\n    a, b = 0, 1\n    for _ in range(2, n + 1):\n        a, b = b, a + b\n    return b\n```","status":"completed","finishReason":"","latencyMs":2336,"tokensIn":64,"tokensOut":207,"tokensPerSecond":88.6,"costUsd":0.00020793,"accuracy":82,"accuracyNote":"Correct with explicit base cases, but preceded by a leaked <think> block.","receiptVersion":1,"promptHash":"d7291bb1d553582240d939b4f9968735f29004f81c9c6136d1f60876605339c1","rawOutputHash":"c0e8b6d5774ef5d5da18b4b9c9de94b1f71289f1d9ca1cd66ecabfaae344605c","graderVersion":"claude-agent-grader-v1","solverVersion":null,"harnessVersion":null,"validationState":"recorded","validationFlags":["prompt-hash=sha256","raw-output-hash=sha256","status=completed","accuracy=range-valid","latency=recorded","tokens=recorded","cost=recorded","grader-score=recorded","reasoning-leak=detected","code-fence=present"],"correctionVersion":"correction-v1"},{"modelId":"mistral-medium","modelName":"Mistral Medium 3","provider":"Mistral","outputPricePerM":2,"output":"```python\ndef fib(n):\n    if n == 0:\n        return 0\n    a, b = 0, 1\n    for _ in range(2, n + 1):\n        a, b = b, a + b\n    return b\n```","status":"completed","finishReason":"","latencyMs":1629,"tokensIn":76,"tokensOut":60,"tokensPerSecond":36.8,"costUsd":0.0001504,"accuracy":99,"accuracyNote":"Correct, concise, and strictly code-only. No reasoning leakage.","receiptVersion":1,"promptHash":"d7291bb1d553582240d939b4f9968735f29004f81c9c6136d1f60876605339c1","rawOutputHash":"cd7a80fb1cd78f7ad1f83292b9c855a52c54fefd44aba9ffbd079cc96811eb4b","graderVersion":"claude-agent-grader-v1","solverVersion":null,"harnessVersion":null,"validationState":"recorded","validationFlags":["prompt-hash=sha256","raw-output-hash=sha256","status=completed","accuracy=range-valid","latency=recorded","tokens=recorded","cost=recorded","grader-score=recorded","reasoning-leak=absent","code-fence=present"],"correctionVersion":"correction-v1"},{"modelId":"mistral-small","modelName":"Mistral Small 3.1","provider":"Mistral","outputPricePerM":0.6,"output":"```python\ndef fib(n):\n    if n == 0:\n        return 0\n    a, b = 0, 1\n    for _ in range(2, n + 1):\n        a, b = b, a + b\n    return b\n```","status":"completed","finishReason":"","latencyMs":629,"tokensIn":76,"tokensOut":60,"tokensPerSecond":95.4,"costUsd":0.000047399999999999993,"accuracy":99,"accuracyNote":"Correct, concise, code-only — identical quality to the larger Mistral at a fraction of the cost.","receiptVersion":1,"promptHash":"d7291bb1d553582240d939b4f9968735f29004f81c9c6136d1f60876605339c1","rawOutputHash":"cd7a80fb1cd78f7ad1f83292b9c855a52c54fefd44aba9ffbd079cc96811eb4b","graderVersion":"claude-agent-grader-v1","solverVersion":null,"harnessVersion":null,"validationState":"recorded","validationFlags":["prompt-hash=sha256","raw-output-hash=sha256","status=completed","accuracy=range-valid","latency=recorded","tokens=recorded","cost=recorded","grader-score=recorded","reasoning-leak=absent","code-fence=present"],"correctionVersion":"correction-v1"},{"modelId":"ministral-8b","modelName":"Ministral 8B","provider":"Mistral","outputPricePerM":0.15,"output":"```python\ndef fib(n):\n    if n == 0:\n        return 0\n    a, b = 0, 1\n    for _ in range(2, n + 1):\n        a, b = b, a + b\n    return b\n```","status":"completed","finishReason":"","latencyMs":1368,"tokensIn":64,"tokensOut":60,"tokensPerSecond":43.9,"costUsd":0.000018599999999999998,"accuracy":99,"accuracyNote":"Correct, code-only, and one of the cheapest runs in the test.","receiptVersion":1,"promptHash":"d7291bb1d553582240d939b4f9968735f29004f81c9c6136d1f60876605339c1","rawOutputHash":"cd7a80fb1cd78f7ad1f83292b9c855a52c54fefd44aba9ffbd079cc96811eb4b","graderVersion":"claude-agent-grader-v1","solverVersion":null,"harnessVersion":null,"validationState":"recorded","validationFlags":["prompt-hash=sha256","raw-output-hash=sha256","status=completed","accuracy=range-valid","latency=recorded","tokens=recorded","cost=recorded","grader-score=recorded","reasoning-leak=absent","code-fence=present"],"correctionVersion":"correction-v1"},{"modelId":"codestral","modelName":"Codestral","provider":"Mistral","outputPricePerM":0.9,"output":"```python\ndef fib(n):\n    if n == 0:\n        return 0\n    a, b = 0, 1\n    for _ in range(2, n + 1):\n        a, b = b, a + b\n    return b\n```","status":"completed","finishReason":"","latencyMs":653,"tokensIn":64,"tokensOut":60,"tokensPerSecond":91.9,"costUsd":0.00007319999999999999,"accuracy":99,"accuracyNote":"Correct, code-only — exactly what you want from a code-specialised model.","receiptVersion":1,"promptHash":"d7291bb1d553582240d939b4f9968735f29004f81c9c6136d1f60876605339c1","rawOutputHash":"cd7a80fb1cd78f7ad1f83292b9c855a52c54fefd44aba9ffbd079cc96811eb4b","graderVersion":"claude-agent-grader-v1","solverVersion":null,"harnessVersion":null,"validationState":"recorded","validationFlags":["prompt-hash=sha256","raw-output-hash=sha256","status=completed","accuracy=range-valid","latency=recorded","tokens=recorded","cost=recorded","grader-score=recorded","reasoning-leak=absent","code-fence=present"],"correctionVersion":"correction-v1"},{"modelId":"cerebras-gpt-oss-120b","modelName":"GPT-OSS 120B (Cerebras)","provider":"Cerebras","outputPricePerM":0.75,"output":"<think>\nThe user wants a Python function called `fib(n)` that returns the n-th Fibonacci number with 0-indexed. Use efficient iterative approach and handle n=0 correctly. Return only the code inside a single code block, with no explanation. So I need to provide just a code block with the function definition. Simple. Ensure no extra text. Provide something like:\n\n```python\ndef fib(n):\n    a, b = 0, 1\n    for _ in range(n):\n        a, b = b, a + b\n    return a\n```\n\nThat handles n=0 returns 0. That's correct. Provide only code block.\n</think>\n```python\ndef fib(n):\n    a, b = 0, 1\n    for _ in range(n):\n        a, b = b, a + b\n    return a\n```","status":"completed","finishReason":"","latencyMs":213,"tokensIn":127,"tokensOut":184,"tokensPerSecond":863.8,"costUsd":0.00018245,"accuracy":80,"accuracyNote":"Correct canonical solution at blistering speed, but a <think> block leaked into the output.","receiptVersion":1,"promptHash":"d7291bb1d553582240d939b4f9968735f29004f81c9c6136d1f60876605339c1","rawOutputHash":"a1d43cb0157cd3d04bde4fb07d1e3152e86036c295e9832872c4513e58674bb1","graderVersion":"claude-agent-grader-v1","solverVersion":null,"harnessVersion":null,"validationState":"recorded","validationFlags":["prompt-hash=sha256","raw-output-hash=sha256","status=completed","accuracy=range-valid","latency=recorded","tokens=recorded","cost=recorded","grader-score=recorded","reasoning-leak=detected","code-fence=present"],"correctionVersion":"correction-v1"},{"modelId":"cerebras-glm-4.7","modelName":"GLM 4.7 (Cerebras)","provider":"Cerebras","outputPricePerM":2.75,"output":"<think>\nWe need to write a Python function called fib(n) that returns the n-th Fibonacci number (0-indexed). Use an efficient iterative approach and handle n=0 correctly. Return only the code inside a single code block, with no explanation.\n\nSo the solution: define function fib(n). Use iterative method: if n <= 1: return n. else: a, b = 0, 1; for i in range(2, n+1): a, b = b, a+b; return b.\n\nMake sure to handle n=0 correctly: returns 0. Also handle n=1 returns 1.\n\nWe need to output only the code inside a single code block. So we'll output:\n\n```python\ndef fib(n):\n    if n <= 1:\n        return n\n    a, b = 0, 1\n    for _ in range(2, n+1):\n        a, b = b, a + b\n    return b\n```\n\nThat's it. No explanation.\n</think>\n```python\ndef fib(n):\n    if n <= 1:\n        return n\n    a, b = 0, 1\n    for _ in range(2, n + 1):\n        a, b = b, a + b\n    return b\n```","status":"completed","finishReason":"","latencyMs":377,"tokensIn":65,"tokensOut":271,"tokensPerSecond":718.8,"costUsd":0.0008915,"accuracy":80,"accuracyNote":"Correct solution, but the model exposed its <think> reasoning before the final code block.","receiptVersion":1,"promptHash":"d7291bb1d553582240d939b4f9968735f29004f81c9c6136d1f60876605339c1","rawOutputHash":"5b09a6ecae0b0c4ebeaf0fed639ffa7d38aa30e8baea93c2c861070b29a3c3aa","graderVersion":"claude-agent-grader-v1","solverVersion":null,"harnessVersion":null,"validationState":"recorded","validationFlags":["prompt-hash=sha256","raw-output-hash=sha256","status=completed","accuracy=range-valid","latency=recorded","tokens=recorded","cost=recorded","grader-score=recorded","reasoning-leak=detected","code-fence=present"],"correctionVersion":"correction-v1"},{"modelId":"nova-lite","modelName":"Amazon Nova Lite","provider":"Amazon","outputPricePerM":0.24,"output":"```python\ndef fib(n):\n    if n == 0:\n        return 0\n    elif n == 1:\n        return 1\n    a, b = 0, 1\n    for _ in range(2, n + 1):\n        a, b = b, a + b\n    return b\n```","status":"completed","finishReason":"","latencyMs":562,"tokensIn":65,"tokensOut":75,"tokensPerSecond":133.5,"costUsd":0.0000219,"accuracy":98,"accuracyNote":"Correct, code-only, clean explicit base cases.","receiptVersion":1,"promptHash":"d7291bb1d553582240d939b4f9968735f29004f81c9c6136d1f60876605339c1","rawOutputHash":"3e1f8b3777e1bd772c4a101c70e97e395ed1f59eca5dfe703ebc3a2d80427572","graderVersion":"claude-agent-grader-v1","solverVersion":null,"harnessVersion":null,"validationState":"recorded","validationFlags":["prompt-hash=sha256","raw-output-hash=sha256","status=completed","accuracy=range-valid","latency=recorded","tokens=recorded","cost=recorded","grader-score=recorded","reasoning-leak=absent","code-fence=present"],"correctionVersion":"correction-v1"},{"modelId":"nova-micro","modelName":"Amazon Nova Micro","provider":"Amazon","outputPricePerM":0.14,"output":"```\ndef fib(n):\n    if n == 0:\n        return 0\n    elif n == 1:\n        return 1\n    a, b = 0, 1\n    for _ in range(2, n + 1):\n        a, b = b, a + b\n    return b\n```","status":"completed","finishReason":"","latencyMs":530,"tokensIn":65,"tokensOut":73,"tokensPerSecond":137.7,"costUsd":0.000012494999999999998,"accuracy":97,"accuracyNote":"Correct and code-only; used a bare ``` fence without a language tag — a trivial nit.","receiptVersion":1,"promptHash":"d7291bb1d553582240d939b4f9968735f29004f81c9c6136d1f60876605339c1","rawOutputHash":"1306c9cd67e804a40617e532eae72b59247ae26c25cadafdbdfbaf2e2ed6783e","graderVersion":"claude-agent-grader-v1","solverVersion":null,"harnessVersion":null,"validationState":"recorded","validationFlags":["prompt-hash=sha256","raw-output-hash=sha256","status=completed","accuracy=range-valid","latency=recorded","tokens=recorded","cost=recorded","grader-score=recorded","reasoning-leak=absent","code-fence=present"],"correctionVersion":"correction-v1"},{"modelId":"muse-spark-1.3-contributor","modelName":"Muse Spark 1.3 Contributor","provider":"Meta","outputPricePerM":0.2,"output":"```python\ndef fib(n):\n    if n <= 1:\n        return n\n    a, b = 0, 1\n    for _ in range(2, n + 1):\n        a, b = b, a + b\n    return b\n```","status":"completed","finishReason":"stop","latencyMs":3703,"tokensIn":67,"tokensOut":707,"tokensPerSecond":190.9,"costUsd":0.0001481,"accuracy":100,"accuracyNote":"Correct iterative algorithm; fib(0)=0 and fib(1)=1 verified. Clean, code-only. A heavy internal reasoner — most of the 707 output tokens were hidden reasoning, making it slower and pricier per answer than its low per-token rate suggests.","receiptVersion":1,"promptHash":"d7291bb1d553582240d939b4f9968735f29004f81c9c6136d1f60876605339c1","rawOutputHash":"d25a76102a8f210d19b5e4a03fd019813491443e11f3afebd03780c35e682936","graderVersion":"claude-agent-grader-v1","solverVersion":null,"harnessVersion":null,"validationState":"recorded","validationFlags":["prompt-hash=sha256","raw-output-hash=sha256","status=completed","accuracy=range-valid","latency=recorded","tokens=recorded","cost=recorded","grader-score=recorded","reasoning-leak=absent","code-fence=present"],"correctionVersion":"correction-v1"}],"receiptVersion":1,"promptHash":"d7291bb1d553582240d939b4f9968735f29004f81c9c6136d1f60876605339c1","graderVersion":"claude-agent-grader-v1","solverVersion":null,"harnessVersion":null,"validationState":"recorded","validationFlags":["suite-family=cheap","prompt-hash=sha256","raw-output-hashes=per-result-sha256","grader-version=declared","validation=deterministic"],"correctionVersion":"correction-v1"}}