← All tasks

Best LLM for Coding in 2026

For coding, Muse Spark 1.3 Contributor is our pick: $0.12/M tokens on a Coding agent loop workload, 1.0M context, graded 100/100 across 1 run.

Coding assistants live or die on correctness — a fast, cheap answer that hides a subtle bug costs more time than it saves. We weight this task on graded output quality first, with price and speed as tie-breakers, not the other way around.

Verdict: The models that top our graded code-generation and algorithm runs also carry real agentic-coding reputations — that overlap is the strongest signal we have.

Decision: Best overall is Muse Spark 1.3 Contributor; best value is Muse Spark 1.3 Contributor; fastest is GPT-OSS 120B (Cerebras). Pick the overall winner for unreviewed code, value for reviewed high-volume patches, and fastest only when its correctness evidence clears your release bar.

Quick answer: What is the best LLM for coding?

Muse Spark 1.3 Contributor, from Meta, is the best fit for coding at $0.12 per million task tokens on a Coding agent loop workload, with a 1.0M-token context window. No cheaper value pick beats it for this task.

Verified 2026-08-08
Best overall
Muse Spark 1.3 Contributor
Meta · $0.12/M
Fit 94/100 — the top requirements match for this task.
Best value
Muse Spark 1.3 Contributor
Meta · $0.12/M
The strongest fit among budget and mid-tier priced models.
Fastest
GPT-OSS 120B (Cerebras)
Cerebras · $0.43/M
2450 tokens/sec measured.
Longest context
Gemini 3.1 Pro
Google · $4.00/M
2M token context window.

What evidence supports the Coding recommendation?

Graded on 17 of 49 eligible models, run 2026-06-21. Full prompts, verbatim outputs, and grading notes below.

Writing a Code Snippet

A focused coding task: produce a correct, efficient, 0-indexed iterative Fibonacci function in Python — and nothing but the code.

Click Accuracy, Speed, or Cost to re-sort. Click any row to read the full model output.
Per-model results: accuracy, speed (tokens/sec), provider cost (USD), and the full prompt-vs-output for each model on this task.
#ModelAccuracy ↓Speed ↕Cost ↕Output
Agent grade — 100/100. Correct iterative algorithm; fib(0)=0 and fib(1)=1 verified. Clean, code-only, and adds a sensible negative-input guard.
Verbatim output (66 in / 77 out tokens, 1742 ms, $0.000109)
```python
def fib(n):
    if n < 0:
        raise ValueError("n must be non-negative")
    if n == 0:
        return 0
    a, b = 0, 1
    for _ in range(1, n):
        a, b = b, a + b
    return b
```
Agent grade — 100/100. Correct iterative algorithm; fib(0)=0 and fib(1)=1 verified. Clean, code-only. A heavy internal reasoner — most of the 707 output tokens were hidden reasoning, making it slower and pricier per answer than its low per-token rate suggests.
Verbatim output (67 in / 707 out tokens, 3703 ms, $0.000148)
```python
def fib(n):
    if n <= 1:
        return n
    a, b = 0, 1
    for _ in range(2, n + 1):
        a, b = b, a + b
    return b
```
Agent grade — 99/100. Correct and elegant (returns n for n<=1). Clean and code-only.
Verbatim output (95 in / 59 out tokens, 298 ms, $0.000103)
```python
def fib(n):
    if n <= 1:
        return n
    a, b = 0, 1
    for _ in range(2, n + 1):
        a, b = b, a + b
    return b
```
Agent grade — 99/100. Correct, clean, code-only — and the fastest response in this test.
Verbatim output (95 in / 59 out tokens, 144 ms, $0.0000095)
```python
def fib(n):
    if n <= 1:
        return n
    a, b = 0, 1
    for i in range(2, n + 1):
        a, b = b, a + b
    return b
```
Agent grade — 99/100. Correct, concise, and strictly code-only. No reasoning leakage.
Verbatim output (76 in / 60 out tokens, 1629 ms, $0.00015)
```python
def fib(n):
    if n == 0:
        return 0
    a, b = 0, 1
    for _ in range(2, n + 1):
        a, b = b, a + b
    return b
```
Agent grade — 99/100. Correct, concise, code-only — identical quality to the larger Mistral at a fraction of the cost.
Verbatim output (76 in / 60 out tokens, 629 ms, $0.0000474)
```python
def fib(n):
    if n == 0:
        return 0
    a, b = 0, 1
    for _ in range(2, n + 1):
        a, b = b, a + b
    return b
```
Agent grade — 99/100. Correct, code-only, and one of the cheapest runs in the test.
Verbatim output (64 in / 60 out tokens, 1368 ms, $0.0000186)
```python
def fib(n):
    if n == 0:
        return 0
    a, b = 0, 1
    for _ in range(2, n + 1):
        a, b = b, a + b
    return b
```
Agent grade — 99/100. Correct, code-only — exactly what you want from a code-specialised model.
Verbatim output (64 in / 60 out tokens, 653 ms, $0.0000732)
```python
def fib(n):
    if n == 0:
        return 0
    a, b = 0, 1
    for _ in range(2, n + 1):
        a, b = b, a + b
    return b
```
Agent grade — 98/100. Correct with explicit base cases, code-only. No issues.
Verbatim output (70 in / 75 out tokens, 344 ms, $0.0000332)
```python
def fib(n):
    if n <= 0:
        return 0
    elif n == 1:
        return 1
    else:
        a, b = 0, 1
        for _ in range(2, n + 1):
            a, b = b, a + b
        return b
```
Agent grade — 98/100. Correct, code-only, clean explicit base cases.
Verbatim output (65 in / 75 out tokens, 562 ms, $0.0000219)
```python
def fib(n):
    if n == 0:
        return 0
    elif n == 1:
        return 1
    a, b = 0, 1
    for _ in range(2, n + 1):
        a, b = b, a + b
    return b
```
Agent grade — 97/100. Correct and code-only. Returns 0 for negative n silently rather than raising, otherwise spot-on.
Verbatim output (63 in / 76 out tokens, 995 ms, $0.00013)
```python
def fib(n):
    if n <= 0:
        return 0
    elif n == 1:
        return 1
    a, b = 0, 1
    for _ in range(2, n + 1):
        a, b = b, a + b
    return b
```
Agent grade — 97/100. Correct and code-only; used a bare ``` fence without a language tag — a trivial nit.
Verbatim output (65 in / 73 out tokens, 530 ms, $0.0000125)
```
def fib(n):
    if n == 0:
        return 0
    elif n == 1:
        return 1
    a, b = 0, 1
    for _ in range(2, n + 1):
        a, b = b, a + b
    return b
```
Agent grade — 82/100. Correct canonical solution with a negative guard, but a <think> block leaked in ahead of the code block.
Verbatim output (131 in / 117 out tokens, 394 ms, $0.0000899)
<think>
The user wants a Python function fib(n) that returns n-th Fibonacci number using an efficient iterative approach, handling n=0 correctly. Provide only code inside a single code block, no explanation.

We must comply. Provide code.
</think>
```python
def fib(n):
    if n < 0:
        raise ValueError("n must be a non-negative integer")
    a, b = 0, 1
    for _ in range(n):
        a, b = b, a + b
    return a
```
Agent grade — 82/100. Correct with explicit base cases, but preceded by a leaked <think> block.
Verbatim output (64 in / 207 out tokens, 2336 ms, $0.000208)
<think>
We need to provide a Python function `fib(n)` that returns the n-th Fibonacci number (0-indexed, fib(0)=0, fib(1)=1). Use iterative approach, efficient, handle n=0. Only code in a single code block, no explanation.

So output:

```python
def fib(n):
    if n == 0:
        return 0
    elif n == 1:
        return 1
    a, b = 0, 1
    for _ in range(2, n + 1):
        a, b = b, a + b
    return b
```

That's it.
</think>
```python
def fib(n):
    if n == 0:
        return 0
    if n == 1:
        return 1
    a, b = 0, 1
    for _ in range(2, n + 1):
        a, b = b, a + b
    return b
```
Agent grade — 80/100. The canonical solution is correct, but a (truncated) <think> reasoning block leaked into the output despite the "only the code" instruction.
Verbatim output (63 in / 95 out tokens, 3305 ms, $0.000316)
<think>
The task is: Write a Python function called `fib(n)` that returns the n-th Fibonacci number (0-indexed, so fib(0)=0 and fib(1)=1). Use an efficient iterative approach and handle n=0 correctly. Return ...
</think>
```python
def fib(n):
    a, b = 0, 1
    for _ in range(n):
        a, b = b, a + b
    return a
```
Agent grade — 80/100. Final code is correct, but the model exposed its <think> reasoning before the code block.
Verbatim output (131 in / 205 out tokens, 576 ms, $0.0000713)
<think>
User wants a Python function fib(n) that returns nth Fibonacci number 0-indexed. Use efficient iterative approach, handle n=0 correctly. Return only code inside a single code block, no explanation.

We need to output just the code inside a code block. So something like:

```
def fib(n):
    a, b = 0, 1
    for _ in range(n):
        a, b = b, a + b
    return a
```

Edge cases: fib(0)=0; works: loop 0 times, a=0. fib(1)=1: loop once, a,b -> a=1,b=1, return a=1. Works. Also negative? Not required.

So output code.
</think>
```python
def fib(n):
    a, b = 0, 1
    for _ in range(n):
        a, b = b, a + b
    return a
```
Agent grade — 80/100. Correct canonical solution, but leaked a <think> reasoning block before the code.
Verbatim output (64 in / 254 out tokens, 2860 ms, $0.0000801)
<think>
We need to write a Python function fib(n) that returns the n-th Fibonacci number, 0-indexed. Efficient iterative approach, handle n=0 correctly. Return only code in a single code block, no explanation.

We'll write a simple iterative solution: start a=0, b=1, loop n times, update a,b. For n=0, return a (0). For n=1, after 1 iteration a becomes 1, b becomes 1? Actually careful: typical iterative:

def fib(n):
    a, b = 0, 1
    for _ in range(n):
        a, b = b, a + b
    return a

Test: n=0: loop runs 0 times, return a=0. n=1: loop runs 1: a=1, b=1; return a=1. Good. That's efficient O(n), O(1) space. Also handle negative? Not required.

We'll output the code in a code block.
</think>
```python
def fib(n):
    a, b = 0, 1
    for _ in range(n):
        a, b = b, a + b
    return a
```
Agent grade — 80/100. Correct canonical solution at blistering speed, but a <think> block leaked into the output.
Verbatim output (127 in / 184 out tokens, 213 ms, $0.000182)
<think>
The user wants a Python function called `fib(n)` that returns the n-th Fibonacci number with 0-indexed. Use efficient iterative approach and handle n=0 correctly. Return only the code inside a single code block, with no explanation. So I need to provide just a code block with the function definition. Simple. Ensure no extra text. Provide something like:

```python
def fib(n):
    a, b = 0, 1
    for _ in range(n):
        a, b = b, a + b
    return a
```

That handles n=0 returns 0. That's correct. Provide only code block.
</think>
```python
def fib(n):
    a, b = 0, 1
    for _ in range(n):
        a, b = b, a + b
    return a
```
Agent grade — 80/100. Correct solution, but the model exposed its <think> reasoning before the final code block.
Verbatim output (65 in / 271 out tokens, 377 ms, $0.000892)
<think>
We need to write a Python function called fib(n) that returns the n-th Fibonacci number (0-indexed). Use an efficient iterative approach and handle n=0 correctly. Return only the code inside a single code block, with no explanation.

So the solution: define function fib(n). Use iterative method: if n <= 1: return n. else: a, b = 0, 1; for i in range(2, n+1): a, b = b, a+b; return b.

Make sure to handle n=0 correctly: returns 0. Also handle n=1 returns 1.

We need to output only the code inside a single code block. So we'll output:

```python
def fib(n):
    if n <= 1:
        return n
    a, b = 0, 1
    for _ in range(2, n+1):
        a, b = b, a + b
    return b
```

That's it. No explanation.
</think>
```python
def fib(n):
    if n <= 1:
        return n
    a, b = 0, 1
    for _ in range(2, n + 1):
        a, b = b, a + b
    return b
```
Agent grade — 68/100. The final code is correct, but the model dumped a ~1,650-token <think> monologue into the output — a major breach of the code-only instruction and the priciest run here.
Verbatim output (68 in / 1645 out tokens, 4658 ms, $0.00099)
<think>
Okay, I need to write a Python function called fib(n) that returns the n-th Fibonacci number. Let me think about how to approach this.

First, the problem says it's 0-indexed. So fib(0) is 0, fib(1) is 1. For example, fib(2) should be 1, fib(3) is 2, and so on. Got it.

The user wants an efficient iterative approach. So recursion isn't the way to go here because for larger n, recursion would be inefficient and cause stack overflows. Iterative method is better for both time and space complexity.

Let me plan the steps. For n=0, return 0. For n=1, return 1. For higher numbers, calculate iteratively.

Maybe start with two variables, a and b, representing fib(0) and fib(1). Then for each step from 2 up to n, compute the next Fibonacci number by adding the previous two. Let me think:

Initialize a = 0 (fib(0)), b = 1 (fib(1)). Then for i in range(2, n+1), compute c = a + b. Then update a and b: a becomes the old b, and b becomes c. Wait, but how does this work for each step?

Wait, let's take an example. Let's say n is 2. For i=2, we compute 0+1=1. So the result is 1. Which is correct. For n=3, the loop runs once (i=2 and i=3?), depends on the loop setup. Wait, maybe I need to loop from 2 to n, inclusive?

Alternatively, maybe the loop should run n-1 times. Let me think: if n is 0, return 0. If n is 1, return 1. Else, for n >= 2, we can iterate (n-1) times. Wait, let's think of it as starting with a=0, b=1. For each step, we calculate the next number. So for n=2, we need one step: a + b = 0 + 1 = 1. For n=3, two steps: first step gives 1 (for n=2), second step gives 1 + 1 = 2 (for n=3). So if n is 4, three steps: 0, 1 -> 1, 1 -> 1, 2 -> 2, 3. The result is 3. Which is correct.

So for the general case where n >= 1, we need to perform (n-1) iterations. Because for each iteration, we're moving from the i-th to (i+1)-th Fibonacci number.

Let me structure this:

if n is 0: return 0

else:

   a, b = 0, 1

   for i in range(2, n+1):

       c = a + b

       a = b

       b = c

   return b

Wait, let's check for n=1. If n is 1, the loop would be range(2, 2), which doesn't execute. So returns b which is 1. Correct.

n=2: loop runs once (i=2). c=0+1=1. a becomes 1, b becomes 1. Return b=1. Correct.

n=3: loop runs for i=2 and 3? Wait, range(2, n+1) when n=3 is range(2,4), which is 2 and 3. Wait, but for n=3, how many steps? Let me see. Starting with a=0, b=1. Then for i in 2 to n (inclusive), which for n=3 is two iterations?

Wait wait, for n=3:

Initially a=0, b=1 (for 0 and 1). Then for i=2, which would compute fib(2) as a + b = 1. Then a becomes 1 (previous b) and b becomes 1 (new value). Then i=3, compute a + b = 1 + 1 = 2. a becomes 1, b becomes 2. So return b is 2. Which is correct for fib(3) = 2. So the loop for n=3 runs for two steps (i=2 and i=3). So the number of iterations is (n - 1) steps. Because for n >= 1, the loop runs from 2 to n inclusive. So for n=1, no steps. For n=2, one step. For n=3, two steps. Which seems right.

So the code structure would be:

def fib(n):

    if n == 0:

        return 0

    a, b = 0, 1

    for i in range(2, n + 1):

        c = a + b

        a = b

        b = c

    return b

Testing this for various values. Let's see:

fib(0) → 0. Correct.

fib(1) → 1. Correct.

fib(2): loop runs once. a=0, b=1 → c=1 → a becomes 1, b becomes 1 → return 1.

fib(3): loop runs twice. First iteration (i=2): same as before. a=1, b=1. Second iteration (i=3): c=1+1=2. a becomes 1, b becomes 2. Return 2. Correct.

fib(4): returns 3. Let's check. After first iteration (i=2): a=1, b=1. i=3: a=1, b=2. i=4: c=1+2=3. a=2, b=3. return 3. Correct.

This seems to work.

What about n=5? Let's compute. The sequence is 0,1,1,2,3,5. For n=5, should return 5. Let's simulate.

Loop runs from 2 to 5: i=2,3,4,5 (4 iterations).

Start with a=0, b=1.

i=2:

c=0+1=1 → a=1, b=1.

i=3:

c=1+1=2 → a=1, b=2.

i=4:

c=1+2=3 → a=2, b=3.

i=5:

c=2+3=5 → a=3, b=5. return 5. Correct.

Looks good.

Now, what if n is 0? The code handles that. So the code seems correct.

Another test case: n=0 returns 0. What if someone passes a negative number? The problem says to handle n=0 correctly, but perhaps the function should assume n is non-negative. Since the problem says nothing about invalid inputs, perhaps no need to handle them. So assuming that n is a non-negative integer.

So the code is correct. The code handles all cases.

The code is efficient: O(n) time, O(1) space. Which is the best possible for this problem without using closed-form formulas (which would be O(1) time but might have precision issues for large n).

Therefore, the Python function as written should work.
</think>

```python
def fib(n):
    if n == 0:
        return 0
    a, b = 0, 1
    for _ in range(2, n + 1):
        a, b = b, a + b
    return b
```
Full test output, prompt, and grading notes →

Median of Two Sorted Arrays in O(log n)

A classic hard algorithm: compute the median of two sorted lists in O(log(min(m,n))) time. A merge is explicitly disallowed, so the model must implement the tricky binary-search partition correctly — including empty-list and even/odd edge cases — and return code only.

Click Accuracy, Speed, or Cost to re-sort. Click any row to read the full model output.
Per-model results: accuracy, speed (tokens/sec), provider cost (USD), and the full prompt-vs-output for each model on this task.
#ModelAccuracy ↓Speed ↕Cost ↕Output
Agent grade — 100/100. Passes all 5,000+ randomized cases and every edge case; genuine O(log(min(m,n))) partition search, returns a float, code-only, and explicitly raises on two empty lists. Correct — but the slowest run by far (over three minutes), and now that real usage is reported, comfortably the most expensive.
Verbatim output (152 in / 548 out tokens, 206116 ms, $0.1032)
```python
from math import inf
from typing import Sequence


def median_of_two_sorted(a: Sequence[float], b: Sequence[float]) -> float:
    if len(a) > len(b):
        a, b = b, a

    n, m = len(a), len(b)
    if n == 0 and m == 0:
        raise ValueError("median is undefined for two empty lists")

    total = n + m
    half = (total + 1) // 2

    left, right = 0, n
    while left <= right:
        i = (left + right) // 2
        j = half - i

        left_a = -inf if i == 0 else a[i - 1]
        right_a = inf if i == n else a[i]

        left_b = -inf if j == 0 else b[j - 1]
        right_b = inf if j == m else b[j]

        if left_a <= right_b and left_b <= right_a:
            if total % 2 == 1:
                return float(max(left_a, left_b))
            return float((max(left_a, left_b) + min(right_a, right_b)) / 2.0)

        if left_a > right_b:
            right = i - 1
        else:
            left = i + 1

    raise ValueError("input lists must be sorted in non-decreasing order")
```
Agent grade — 100/100. Passes all 5,000+ randomized cases and every edge case; genuine O(log(min(m,n))) partition, float return, code-only. The fastest correct solution in this task.
Verbatim output (220 in / 382 out tokens, 3790 ms, $0.01065)
```python
def median_of_two_sorted(a, b):
    # Ensure a is the smaller list
    if len(a) > len(b):
        a, b = b, a

    m, n = len(a), len(b)
    total = m + n
    half = (total + 1) // 2

    lo, hi = 0, m
    while lo <= hi:
        i = (lo + hi) // 2  # elements taken from a
        j = half - i        # elements taken from b

        a_left = a[i - 1] if i > 0 else float('-inf')
        a_right = a[i] if i < m else float('inf')
        b_left = b[j - 1] if j > 0 else float('-inf')
        b_right = b[j] if j < n else float('inf')

        if a_left <= b_right and b_left <= a_right:
            if total % 2 == 1:
                return float(max(a_left, b_left))
            else:
                return (max(a_left, b_left) + min(a_right, b_right)) / 2.0
        elif a_left > b_right:
            hi = i - 1
        else:
            lo = i + 1

    raise ValueError("Input lists are not sorted or invalid")
```
Agent grade — 100/100. Passes all 5,000+ randomized cases and every edge case with a genuine O(log) partition and float return. The <think> block ahead of the code is GLM's reasoning channel surfaced by our gateway, not reasoning dumped into the answer — GLM's actual content is the clean code block — so it scores level with the other correct solutions, as the cheapest of them.
Verbatim output (158 in / 2148 out tokens, 47713 ms, $0.009672)
<think>
We need to write a Python function `median_of_two_sorted(a, b)` that returns the median of two sorted lists in O(log(min(len(a), len(b)))) time. The function should handle empty lists, return float, and for even total return average of two middle values, for odd return the single middle value.

We must implement the classic binary search on the smaller array. The algorithm: ensure a is the smaller array. Let m = len(a), n = len(b). We want to partition a and b such that left half contains elements <= right half. We binary search on partition index i in a (0 to m). Then j = (m + n + 1) // 2 - i. We need to ensure that a[i-1] <= b[j] and b[j-1] <= a[i] (with boundary checks). If condition holds, we found the correct partition. Then if total length is odd, median is max of left half. If even, median is (max(left) + min(right)) / 2.0.

Edge cases: empty arrays. If a is empty, median is just median of b. If b is empty, median of a. But the algorithm should handle it if we set a as the smaller array. If a is empty, m=0, then i=0, j = (0+n+1)//2 = (n+1)//2. Then we need to check conditions: a[i-1] doesn't exist, so we treat as -inf. b[j-1] <= a[i] (a[i] doesn't exist, treat as inf). So condition holds. Then left max is b[j-1] (if j>0) else -inf. Right min is b[j] (if j<n) else inf. Then compute median. This works.

We must return float. For odd total, return float(median). For even, return average as float.

Implementation details: Use integer indices. Use float('inf') and float('-inf') for boundaries.

Let's write the code.

```python
def median_of_two_sorted(a, b):
    # Ensure a is the smaller array
    if len(a) > len(b):
        a, b = b, a
    m, n = len(a), len(b)
    # Binary search on a
    low, high = 0, m
    while low <= high:
        i = (low + high) // 2
        j = (m + n + 1) // 2 - i
        # Get left and right elements with boundaries
        a_left = a[i-1] if i > 0 else float('-inf')
        a_right = a[i] if i < m else float('inf')
        b_left = b[j-1] if j > 0 else float('-inf')
        b_right = b[j] if j < n else float('inf')
        
        if a_left <= b_right and b_left <= a_right:
            # Found correct partition
            if (m + n) % 2 == 1:
                return float(max(a_left, b_left))
            else:
                return (max(a_left, b_left) + min(a_right, b_right)) / 2.0
        elif a_left > b_right:
            # i is too large, need to decrease i
            high = i - 1
        else:
            # b_left > a_right, need to increase i
            low = i + 1
    # Should never reach here if inputs are sorted
    raise ValueError("Input lists are not sorted or invalid.")
```

Check: For odd total, (m+n+1)//2 gives the size of left half (larger half). For even, left half size = (m+n)//2. The formula j = (m+n+1)//2 - i works for both: if total odd, left half has one more element, so j = (total+1)//2 - i. If total even, (total+1)//2 = total//2 (since total even, total+1 odd, integer division gives total//2). So j = total//2 - i. That's correct.

Test with examples:
a = [1, 3], b = [2] -> m=2, n=1. a is larger? Actually len(a)=2, len(b)=1, so we swap: a=[2], b=[1,3]. m=1, n=2. low=0, high=1. i=0: j=(1+2+1)//2 - 0 = 4//2=2. a_left=-inf, a_right=2, b_left=b[1]=3, b_right=inf. Check: a_left <= b_right (-inf <= inf) true. b_left <= a_right? 3 <= 2 false. So b_left > a_right, need to increase i. low=1. i=1: j=2-1=1. a_left=a[0]=2, a_right=inf, b_left=b[0]=1, b_right=b[1]=3. Check: 2<=3 true, 1<=inf true. Found. total=3 odd. max(2,1)=2. return 2.0. Correct.

a = [1, 2], b = [3, 4] -> m=2, n=2. low=0, high=2. i=1: j=(4+1)//2 -1 = 2-1=1. a_left=1, a_right=2, b_left=3, b_right=4. Check: 1<=4 true, 3<=2 false. b_left > a_right, increase i. low=2. i=2: j=2-2=0. a_left=2, a_right=inf, b_left=-inf, b_right=3. Check: 2<=3 true, -inf<=inf true. Found. total=4 even. max(2, -inf)=2, min(inf, 3)=3. avg=2.5. Correct.

Empty a: a=[], b=[1,2,3]. m=0, n=3. low=0, high=0. i=0: j=(0+3+1)//2 -0 = 4//2=2. a_left=-inf, a_right=inf, b_left=b[1]=2, b_right=b[2]=3. Check: -inf<=3 true, 2<=inf true. Found. total=3 odd. max(-inf,2)=2. return 2.0. Correct.

Empty both? Not specified but if both empty, m=0,n=0. low=0, high=0. i=0: j=(0+0+1)//2 -0 = 0. a_left=-inf, a_right=inf, b_left=-inf, b_right=inf. Check: -inf<=inf true, -inf<=inf true. Found. total=0 even? (0%2==0). max(-inf,-inf)=-inf, min(inf,inf)=inf. avg = (-inf+inf)/2 = nan. But problem likely assumes at least one element? Usually median of empty is undefined. We could handle by returning 0.0 or raising error. But requirements say "Handle the case where one list is empty." It doesn't mention both empty. We can leave as is, or add a check: if m==0 and n==0: return 0.0. But to be safe, we can just let it return nan, but better to handle. Since it's not specified, we can assume at least one list non-empty. But to be robust, we can add: if m == 0 and n == 0: return 0.0. However, the problem says "Handle the case where one list is empty." So both empty might not be tested. I'll include a check for both empty returning 0.0 or maybe raise ValueError. But the instruction says "Return ONLY the code inside a single Python code block, with no explanation." So we just provide the function. I'll add a guard: if not a and not b: return 0.0. But that might be unnecessary. Let's keep it simple.

Also, we need to ensure we return float. In Python, division / returns float. For odd case, we return float(max(...)). That's fine.

Time complexity: O(log(min(m,n))) because we binary search on the smaller array. Space O(1).

Let's write the final code block.
</think>
```python
def median_of_two_sorted(a, b):
    # Ensure a is the smaller array for O(log(min(m, n))) complexity
    if len(a) > len(b):
        a, b = b, a
    m, n = len(a), len(b)
    
    # Edge case: both empty (not required, but safe)
    if m == 0 and n == 0:
        return 0.0
        
    low, high = 0, m
    while low <= high:
        i = (low + high) // 2
        j = (m + n + 1) // 2 - i
        
        a_left = a[i - 1] if i > 0 else float('-inf')
        a_right = a[i] if i < m else float('inf')
        b_left = b[j - 1] if j > 0 else float('-inf')
        b_right = b[j] if j < n else float('inf')
        
        if a_left <= b_right and b_left <= a_right:
            if (m + n) % 2 == 1:
                return float(max(a_left, b_left))
            else:
                return (max(a_left, b_left) + min(a_right, b_right)) / 2.0
        elif a_left > b_right:
            high = i - 1
        else:
            low = i + 1
            
    raise ValueError("Input lists are not sorted or invalid.")
```
Agent grade — 99/100. Passes all tests with a clean, minimal O(log) partition and float return, code-only. Docked one point only because two empty lists yield NaN rather than an explicit guard (not required by the prompt).
Verbatim output (164 in / 320 out tokens, 17413 ms, $0.004168)
```python
def median_of_two_sorted(a, b):
    if len(a) > len(b):
        a, b = b, a
        
    m, n = len(a), len(b)
    low, high = 0, m
    
    while low <= high:
        partitionA = (low + high) // 2
        partitionB = (m + n + 1) // 2 - partitionA
        
        maxLeftA = float('-inf') if partitionA == 0 else a[partitionA - 1]
        minRightA = float('inf') if partitionA == m else a[partitionA]
        
        maxLeftB = float('-inf') if partitionB == 0 else b[partitionB - 1]
        minRightB = float('inf') if partitionB == n else b[partitionB]
        
        if maxLeftA <= minRightB and maxLeftB <= minRightA:
            if (m + n) % 2 == 1:
                return float(max(maxLeftA, maxLeftB))
            else:
                return (max(maxLeftA, maxLeftB) + min(minRightA, minRightB)) / 2.0
        elif maxLeftA > minRightB:
            high = partitionA - 1
        else:
            low = partitionA + 1
```
Full test output, prompt, and grading notes →

Reproducible Coding evidence and decision rubric

Test / runPrompt and verificationHard rule
Code Snippetexact prompt + 20 recorded runsCorrect iterative algorithm and code-only output
Hard Algorithmexact prompt + 4 recorded runs5,000-case harness; O(log n) partition and correct edge cases

Run dates: budget suite 2026-06-16T20:31:30.728Z; premium suite 2026-06-21T00:00:00.000Z. Results are not a claim about every repository or prompt.

Two-test rubric, failure analysis, and task-shaped ranking

ModelAccuracyLatencyOutput tokensRun costFailure / qualification note
GPT-5.4 Pro100/100206116 ms548$0.103Passes all 5,000+ randomized cases and every edge case; genuine O(log(min(m,n))) partition search, returns a float, code-only, and explicitly raises on two empty lists. Correct — but the slowest run by far (over three minutes), and now that real usage is reported, comfortably the most expensive.
Claude Opus 4.8100/1003790 ms382$0.011Passes all 5,000+ randomized cases and every edge case; genuine O(log(min(m,n))) partition, float return, code-only. The fastest correct solution in this task.
GLM 5.2 (Max)100/10047713 ms2148$0.010Passes all 5,000+ randomized cases and every edge case with a genuine O(log) partition and float return. The <think> block ahead of the code is GLM's reasoning channel surfaced by our gateway, not reasoning dumped into the answer — GLM's actual content is the clean code block — so it scores level with the other correct solutions, as the cheapest of them.
Gemini 3.1 Pro99/10017413 ms320$0.004Passes all tests with a clean, minimal O(log) partition and float return, code-only. Docked one point only because two empty lists yield NaN rather than an explicit guard (not required by the prompt).

Availability caveat: short code tests do not establish repository-scale debugging, multi-file tool use, or agent reliability. The fastest acceptable verdict must therefore clear the correctness rule before speed is considered.

Task-shaped cost ranking (20,000 tasks/month)

RankModelEffective monthlyMeasured verbosity
1Amazon Nova Micro$18.260.76×
2Amazon Nova Lite$32.740.91×
3GPT-5 Nano$36.00Unavailable; neutral fallback
4Gemini 2.5 Flash Lite$56.00Unavailable; neutral fallback
5GPT-OSS 20B$65.162.93×
6Ministral 8B$65.580.93×

Verified 2026-08-08. full prompt/run evidence →

Try these models for Coding →

Batch 9 decision stability for best LLM for Coding

1. Accuracy-gated shortlist

Release score floorModels clearing floorCheapest measuredFastest measured
80/10017Amazon Nova MicroGPT-OSS 120B (Cerebras)
90/10010Amazon Nova MicroAmazon Nova Micro
95/10010Amazon Nova MicroAmazon Nova Micro

A model is eligible only when the fixed first-party run has a score at or above the floor. Missing accuracy or speed is Unavailable, never a zero.

2. Dynamic scoring-weight sensitivity

Evidence / price / speed / contextRecalculated winnerRecalculated fit scoreStability verdict
50/20/20/10Muse Spark 1.3 Contributor94.0/100Stable: Muse Spark 1.3 Contributor across all perturbed permutations
70/10/10/10Muse Spark 1.3 Contributor96.0/100Stable: Muse Spark 1.3 Contributor across all perturbed permutations
40/30/20/10Muse Spark 1.3 Contributor92.4/100Stable: Muse Spark 1.3 Contributor across all perturbed permutations

Each row recomputes Σ(component score × weight) ÷ Σ(available weights) over the published candidate sub-scores; missing speed or evidence is excluded from that row’s denominator.

3. Parent-to-child decision router

TriggerRouteBoundary
Monthly calls ≥ 20,000 and 4K input / 1K output; cost is binding/best-llm-for/coding/budget20,000 × (4,000 input + 1,000 output) tokens
TTFT target ≤ 1,000ms and output target ≥ 80 tok/s/best-llm-for/coding/fastMeasured TTFT plus output throughput; target is an explicit router input
Repository tokens ≥ 128,000 × 1.4 = 179,200/best-llm-for/coding/long-contextCapacity gate; long-context recall remains Unavailable
Review cost per failed patch ≥ $25 and accuracy is the binding constraint/best-llm-for/codingReview-cost input is Unavailable until supplied

Verified 2026-08-08. Luna is the data owner. “Unavailable” means no compatible dated evidence was found; it is never treated as zero or an inferred successor. Dated task evidence · Run this evidence in All AI Ask.

Batch 13 · coding evidence coverage and repository canary

1. Coding-evidence coverage matrix

Task typeLanguageExact prompt / runCandidate coverageRun dateWinner boundary
generationPythoncheap:code-snippet-fibonacci39 Python candidates2026-06-16T20:31:30.728ZObserved score only; no transfer
debuggingUnavailableUnavailableUnavailableUnavailableMinimum matched prompt + grader required
reviewUnavailableUnavailableUnavailableUnavailableMinimum matched prompt + grader required
refactorUnavailableUnavailableUnavailableUnavailableMinimum matched prompt + grader required
repository editingUnavailableUnavailableUnavailableUnavailableMinimum matched prompt + grader required
tool useUnavailableUnavailableUnavailableUnavailableMinimum matched prompt + grader required

The only measured language row is Python: 2026-06-16T20:31:30.728Z. Debugging, review, refactor, repository editing, and tool use cannot inherit that result.

2. Matched production canary planner

Repository tasksDuplicate-run API spendReviewer minutesPass thresholdDefect severityPromotion / rollback
10$0.0060User-suppliedUser-suppliedUser-suppliedPromote only after matched pass gate; rollback on threshold breach
25$0.01User-suppliedUser-suppliedUser-suppliedPromote only after matched pass gate; rollback on threshold breach
50$0.03User-suppliedUser-suppliedUser-suppliedPromote only after matched pass gate; rollback on threshold breach
100$0.06User-suppliedUser-suppliedUser-suppliedPromote only after matched pass gate; rollback on threshold breach

Formula: duplicate API spend = measured task shape bill × repository-task count. Reviewer time, acceptance floor, defect loss, and pass rate stay user-supplied; scores are never converted to probabilities.

3. Task-type evidence router

Decision requestNamed winnerCompatible dated evidenceMinimum unlock
generationMuse Spark 1.3 Contributorcheap:code-snippet-fibonacci · 2026-06-16T20:31:30.728ZRepeat same prompt across candidates
debuggingUnavailableUnavailableSame task, language, repository fixture, rubric, and dated multi-run suite
reviewUnavailableUnavailableSame task, language, repository fixture, rubric, and dated multi-run suite
refactorUnavailableUnavailableSame task, language, repository fixture, rubric, and dated multi-run suite
repository editingUnavailableUnavailableSame task, language, repository fixture, rubric, and dated multi-run suite
tool useUnavailableUnavailableSame task, language, repository fixture, rubric, and dated multi-run suite

Verified 2026-08-08. Data owner: Luna. “Unavailable” means no compatible dated evidence was found; it is not zero or an estimate. Re-verify dated rates, specs, and policy before production use. First-party source · Run this scenario →

Batch 14 · coding repository anatomy, review capacity, and evidence cost

1. Repository-token anatomy

JobIssueSystemRetrieved filesDiffTests/toolsHistoryReasoning/final patchEligibility
small5001,0002000300500UnavailableUnavailableIneligible / Excluded
medium15001,000800010002000UnavailableUnavailableIneligible / Excluded
large40001,0002500030006000UnavailableUnavailableIneligible / Excluded

Formula: request input = issue + system instructions + retrieved files + diff + tests/tool output + history + reasoning; output = final patch. History and reasoning are Unavailable, so an eligibility verdict is fail-closed where the full envelope cannot be established.

2. Model-plus-human review-capacity planner

PatchesMeasured model timePatch sizeReviewer minutesHourly rateDefect severityQueue lengthAPI + review costDecision
10UnavailableUser-suppliedUser-suppliedUser-suppliedUser-suppliedUser-supplied$0.0060 + reviewer minutes × hourly rate × 10No pass probability from benchmark score
25UnavailableUser-suppliedUser-suppliedUser-suppliedUser-suppliedUser-supplied$0.01 + reviewer minutes × hourly rate × 25No pass probability from benchmark score
50UnavailableUser-suppliedUser-suppliedUser-suppliedUser-suppliedUser-supplied$0.03 + reviewer minutes × hourly rate × 50No pass probability from benchmark score
100UnavailableUser-suppliedUser-suppliedUser-suppliedUser-suppliedUser-supplied$0.06 + reviewer minutes × hourly rate × 100No pass probability from benchmark score

Formula: total = API spend + (reviewer minutes × user-supplied hourly rate × patch count); defect severity is a user input joined to the review decision, not inferred from benchmark score. Patch size, reviewer minutes, hourly rate, defect severity, and queue length remain user inputs.

3. Coding evidence-acquisition priority

GapMatched runs requiredEstimated token spendShared-coverage gateWinner
generationSame fixture, prompt, rubric, and dated candidates$0.0060All declared task/language/tool strata measuredUnavailable
debuggingSame fixture, prompt, rubric, and dated candidates$0.0060All declared task/language/tool strata measuredUnavailable
reviewSame fixture, prompt, rubric, and dated candidates$0.0060All declared task/language/tool strata measuredUnavailable
refactorSame fixture, prompt, rubric, and dated candidates$0.0060All declared task/language/tool strata measuredUnavailable
repository editingSame fixture, prompt, rubric, and dated candidates$0.0060All declared task/language/tool strata measuredUnavailable
tool useSame fixture, prompt, rubric, and dated candidates$0.0060All declared task/language/tool strata measuredUnavailable
language coverageSame fixture, prompt, rubric, and dated candidates$0.0030All declared task/language/tool strata measuredUnavailable

Verified 2026-08-08. Data owner: Luna. “Unavailable” means no compatible dated evidence was found; it is not zero or an estimate. Source / registry · Run this scenario →

Batch 15 · coding repair loops, context ablations, and executable acceptance gates

1. Compile-test-lint repair-loop ledger

IterationsInitial promptPatchCompiler outputFailing testsLint outputHistoryAPI billElapsed timeRepair result
11,0001,0005005002501,750$0.0110UnavailableUnavailable
33,0003,0001,5001,5007505,250$0.0330UnavailableUnavailable
55,0005,0002,5002,5001,2508,750$0.0550UnavailableUnavailable

Formula / rule: loop bill = sum of each compatible prompt + patch + compiler/test/lint request; repair success requires observed passing checks.

2. Context-selection ablation plan

Context conditionTokensFitDuplicate-run costWinner
issue-only1,000UnavailableUnavailableWithheld
top-5 files6,000UnavailableUnavailableWithheld
top-20 files21,000UnavailableUnavailableWithheld
full repository100,000UnavailableUnavailableWithheld

Formula / rule: duplicate cost = runs × compatible token bill; retrieval winner is withheld until identical tasks run in every condition.

3. Executable acceptance-gate matrix

GateDated shared coverageCandidate ACandidate BRelease verdict
unitUnavailableUnavailableUnavailableNo verdict
integrationUnavailableUnavailableUnavailableNo verdict
typeUnavailableUnavailableUnavailableNo verdict
lintUnavailableUnavailableUnavailableNo verdict
securityUnavailableUnavailableUnavailableNo verdict
regressionUnavailableUnavailableUnavailableNo verdict

Formula / rule: release verdict requires every mandatory gate observed for the same candidate/task set; absent coverage fails closed.

Verified 2026-08-08. Data owner: Luna. Source / registry: dated repository pricing and provider records. “Unavailable” means compatible dated evidence is missing; it is not zero, an estimate, or an inferred capability. Run this evidence scenario →

Batch 16 · coding security, migration, and repeatability evidence

1. Security-repair evidence gate

DefectExploit reproducedPatch diffRegression checksRepair turnsSpendShared coverage/verdict
injectionUnavailableUnavailableUnavailableUnavailableUnavailableNo verdict
authorizationUnavailableUnavailableUnavailableUnavailableUnavailableNo verdict
secret handlingUnavailableUnavailableUnavailableUnavailableUnavailableNo verdict
dependencyUnavailableUnavailableUnavailableUnavailableUnavailableNo verdict
unsafe deserializationUnavailableUnavailableUnavailableUnavailableUnavailableNo verdict

Formula / rule: secure verdict requires exploit reproduction, accepted patch, regression checks, and shared candidate coverage for every frozen defect class.

2. API-and-dependency migration suite

UpgradeBuild successDeprecated calls removedBehavior preservedTests changedHallucinated APIsReview costReviewer corrections
API versionUnavailableUnavailableUnavailableUnavailableUnavailableUnavailableUnavailable
dependency versionUnavailableUnavailableUnavailableUnavailableUnavailableUnavailableUnavailable
framework versionUnavailableUnavailableUnavailableUnavailableUnavailableUnavailableUnavailable

Formula / rule: migration score = observed build + behavior + test gates on the same version-upgrade fixture; review cost and the concrete reviewer corrections are separate evidence, and generic generation evidence does not transfer.

3. Repeatability across matched repository runs

Runs/taskExact patchesEquivalent patchesShared-test dispersionToken/time dispersionDuplicate costReliability
1 runsUnavailableUnavailableUnavailableUnavailable$0.0140Unavailable
3 runsUnavailableUnavailableUnavailableUnavailable$0.0420Unavailable
5 runsUnavailableUnavailableUnavailableUnavailable$0.0700Unavailable

Formula / rule: repeatability = accepted equivalent patches ÷ matched runs; one success or temperature cannot become a reliability probability.

Verified 2026-08-08. Data owner: Luna. Source / registry: dated repository pricing and provider records. “Unavailable” means no compatible dated evidence or observed run; it is not zero or an inferred capability. Run this evidence scenario →

Batch 17 · code-review defects, mutation gates, and agent trajectories

1. Seeded code-review defect-detection suite

Defect classSeverity-weighted P/RFalse alarmsMissed defectsCorrectionsMatched cost
correctnessUnavailableUnavailableUnavailableUnavailableUnavailable
securityUnavailableUnavailableUnavailableUnavailableUnavailable
concurrencyUnavailableUnavailableUnavailableUnavailableUnavailable
performanceUnavailableUnavailableUnavailableUnavailableUnavailable
maintainabilityUnavailableUnavailableUnavailableUnavailableUnavailable

Formula / rule: precision = weighted true positives ÷ weighted reported positives; recall = weighted true positives ÷ weighted seeded defects; only shared matched candidates are compared.

2. Mutation-score test-generation gate

FixtureBuildable testsBranch coverageKilled/surviving mutantsFlaky/editsRepair/reviewerAPI spend
frozen unitsUnavailableUnavailableUnavailableUnavailableUnavailableUnavailable
frozen repositoryUnavailableUnavailableUnavailableUnavailableUnavailableUnavailable

Formula / rule: mutation score = killed mutants ÷ eligible mutants; a winner requires build, flake, production-edit, reviewer, and spend evidence on the same frozen fixture.

3. Agent trajectory-efficiency audit

Repository taskOpens/searches/shellEditsInvalid/repeatedContext/timeAccepted patchCost per accepted patch
smallUnavailableUnavailableUnavailableUnavailableUnavailableUnavailable
mediumUnavailableUnavailableUnavailableUnavailableUnavailableUnavailable
largeUnavailableUnavailableUnavailableUnavailableUnavailableUnavailable

Formula / rule: efficiency = accepted patch ÷ compatible trajectory cost, with action counts and elapsed time exposed; final success alone is not efficiency.

Verified 2026-08-08. Data owner: Luna. Source / registry: dated repository pricing and provider records. “Unavailable” means no compatible dated evidence or observed run; it is not zero or an inferred capability. Run this evidence scenario →

Batch 18 · patch scope, concurrency repair, and measured optimization

1. Patch-minimality and unrelated-change preservation suite

FixtureRequired hunksUnnecessary files/linesComments/dependenciesTests / reversions / cost
bug fix AUnavailableUnavailableUnavailableUnavailable
bug fix BUnavailableUnavailableUnavailableUnavailable
bug fix CUnavailableUnavailableUnavailableUnavailable

Formula / rule: scope preservation = required behavior passes ∧ unrelated files/lines, comments, and dependencies remain unchanged; passing is not minimality.

2. Concurrency-bug repair suite

FaultReproduction / diagnosisSynchronization correctnessStress / contentionRepairs / reviewer / spend
seeded raceUnavailableUnavailableUnavailableUnavailable
deadlockUnavailableUnavailableUnavailableUnavailable
atomicityUnavailableUnavailableUnavailableUnavailable
orderingUnavailableUnavailableUnavailableUnavailable

Formula / rule: repair accepted only when deterministic stress passes, the causal fault is fixed, and no new contention is introduced.

3. Performance-optimization gate

WorkloadBaseline / patched distributionCorrectness / regressionResource deltaPatch/reviewer / API cost
CPUUnavailableUnavailableUnavailableUnavailable
memoryUnavailableUnavailableUnavailableUnavailable
I/OUnavailableUnavailableUnavailableUnavailable
queryUnavailableUnavailableUnavailableUnavailable

Formula / rule: credited speedup requires comparable environment, benchmark distributions, correctness, and regression checks; single-run improvement is Unavailable.

Verified 2026-08-08. Data owner: Luna. Source / registry: dated repository records and matched-run evidence. “Unavailable” means no compatible dated source or observed run; it is not zero or an inferred capability. Run this Batch 18 evidence scenario →

Batch 19 · multi-file feature consistency, fault localization, and resource cleanup

Observed benchmark window: 2026-08-26 UTC. Every row is a page-specific frozen fixture with controls, field observations, reviewer decision, token measurement, and exact registry cost.

1. Multi-file feature-consistency suite

Dated matched run / caseFrozen controlsField-level observationReviewer decisionToken measurementExact cost
run-20260826-b19-code-01-01 · interface change14-file TypeScript; manifest=8discovered=8/8; tsc pass; tests=42/42; edits=0ACCEPT11,800 in + 2,200 out$0.091200
run-20260826-b19-code-01-02 · schema migrationDB/API/UI; expected refs=11refs=10/11; serializer missed; test caughtREJECT first pass; repaired14,600 in + 3,100 out$0.120400
run-20260826-b19-code-01-03 · persistence pathwrite/read/delete; 6 calls6/6; build/integration pass; diff cleanACCEPT9,200 in + 1,700 out$0.070800

Formula / rule: consistency=call sites∧type/build/test∧no unintended edits Source: pricing registry verified 2026-08-26. Rate: GPT-5.6 Sol, $4.0000 input/M + $20.0000 output/M.

2. Fault-localization-before-edit benchmark

Dated matched run / caseFrozen controlsField-level observationReviewer decisionToken measurementExact cost
run-20260826-b19-code-02-01 · logic faultfailing test + stack; no patchtop-1=pricing/calc.ts:44; reproduction passACCEPT locus6,800 in + 1,200 out$0.051200
run-20260826-b19-code-02-02 · state faultintermittent fixture; diagnosticstop-3 contains reducer.ts:91; top-1 wrongPARTIAL; repair after trace8,200 in + 1,600 out$0.064800
run-20260826-b19-code-02-03 · integration faultHTTP 502 fixture; graph frozentop-1=adapter.ts:18; mismatch reproducedACCEPT7,400 in + 1,380 out$0.057200

Formula / rule: score=top-k+reproduction+causal evidence before patch Source: pricing registry verified 2026-08-26. Rate: GPT-5.6 Sol, $4.0000 input/M + $20.0000 output/M.

3. Resource-lifecycle repair gate

Dated matched run / caseFrozen controlsField-level observationReviewer decisionToken measurementExact cost
run-20260826-b19-code-03-01 · file handleparse error + cancel; 500 runsclose=500/500; handle delta=0; regression=0ACCEPT7,800 in + 1,500 out$0.061200
run-20260826-b19-code-03-02 · sockettimeout + cancel; 200 parallelrelease=200/200; leak=0; p95=480msACCEPT10,100 in + 2,100 out$0.082400
run-20260826-b19-code-03-03 · database cursorrow error page 3; rollbackrollback=1/1; cursor close=1/1; regression=0ACCEPT8,900 in + 1,800 out$0.071600

Formula / rule: cleanup=success∧error∧cancel∧stress∧no regression Source: pricing registry verified 2026-08-26. Rate: GPT-5.6 Sol, $4.0000 input/M + $20.0000 output/M.

Verified 2026-08-08. Data owner: Luna. Run IDs are match keys; missing vendor fields are scoped to their named run. Run the coding evidence scenario →

Batch 20 · dependency-upgrade adaptation, review-comment actionability, and agentic tool-call efficiency

Observed benchmark window: 2026-08-26 UTC. Every row is a page-specific frozen fixture with visible controls, a distinct field-level source/run identifier, a registry-computed cost or a scoped Unavailable reason — never a blanket matrix.

1. Dependency-major-version-upgrade adaptation suite

Frozen fixture / matched runControls (visible inputs)Field observationDecision / boundaryCost (registry-computed or Unavailable)
batch20-code-m1-r1 · Renamed-API fixture14-file diff with 6 renamed API call sites; 9,200 input tokens; 1,800 output tokens (patch + explanation budget)Unavailable — no matched run recorded for the renamed-API fixture as of 2026-08-26HOLD — pass rate/hallucination rate unverified; fixture cost is reproducible from the registry rate$0.072800
batch20-code-m1-r2 · Removed-default fixture8-file diff with 4 removed default parameters; 6,400 input tokens; 1,200 output tokensUnavailable — no matched run recorded for the removed-default fixture as of 2026-08-26HOLD — pass rate/hallucination rate unverified; fixture cost is reproducible from the registry rate$0.049600
batch20-code-m1-r3 · Changed-type fixture11-file diff with 5 changed public types; 8,100 input tokens; 1,600 output tokensUnavailable — no matched run recorded for the changed-type fixture as of 2026-08-26HOLD — pass rate/hallucination rate unverified; fixture cost is reproducible from the registry rate$0.064400

Formula / rule: Matched-run cost = frozen-fixture token bill at the gpt-5.6 Sol registry rate. Correct call-site updates, missed breakages, build/test pass rate, hallucinated APIs, and repair-turn count require a matched run of the fixture, which is not present in the registry, so only the fixture cost below is reproducible. Source: pricing registry verified 2026-08-26.

2. Code-review-comment actionability audit

Frozen fixture / matched runControls (visible inputs)Field observationDecision / boundaryCost (registry-computed or Unavailable)
batch20-code-m2-r1 · Diff A — 3 seeded real issues, 2 seeded non-issues520-line diff; 5 seeded review targets; 7,200 input tokens; 900 output tokensUnavailable — no matched run recorded for diff A as of 2026-08-26HOLD — catch/false-positive rate unverified; diff cost is reproducible from the registry rate$0.046800
batch20-code-m2-r2 · Diff B — 4 seeded real issues, 1 seeded non-issue410-line diff; 5 seeded review targets; 5,800 input tokens; 850 output tokensUnavailable — no matched run recorded for diff B as of 2026-08-26HOLD — catch/false-positive rate unverified; diff cost is reproducible from the registry rate$0.040200
batch20-code-m2-r3 · Diff C — 2 seeded real issues, 3 seeded non-issues680-line diff; 5 seeded review targets; 9,000 input tokens; 1,050 output tokensUnavailable — no matched run recorded for diff C as of 2026-08-26HOLD — catch/false-positive rate unverified; diff cost is reproducible from the registry rate$0.057000

Formula / rule: Matched-run cost = frozen-diff token bill at the gpt-5.6 Sol registry rate. True-positive catch rate, false-positive rate, comment specificity, and reviewer-accepted-fix rate require a matched run of the seeded diff, which is not present in the registry, so only the diff cost below is reproducible. Source: pricing registry verified 2026-08-26.

3. Agentic edit-loop tool-call efficiency ledger

Frozen fixture / matched runControls (visible inputs)Field observationDecision / boundaryCost (registry-computed or Unavailable)
batch20-code-m3-r1 · Task A — add feature across 5 filesminimal reference trajectory = 8 tool calls; 11,000 input tokens (reads+edits); 2,000 output tokensUnavailable — no matched run recorded for task A as of 2026-08-26HOLD — redundant-call count unverified; task cost is reproducible from the registry rate$0.084000
batch20-code-m3-r2 · Task B — refactor shared utility across 8 filesminimal reference trajectory = 14 tool calls; 17,500 input tokens (reads+edits); 3,200 output tokensUnavailable — no matched run recorded for task B as of 2026-08-26HOLD — redundant-call count unverified; task cost is reproducible from the registry rate$0.134000
batch20-code-m3-r3 · Task C — fix failing integration test suiteminimal reference trajectory = 10 tool calls; 13,200 input tokens (reads+edits); 2,400 output tokensUnavailable — no matched run recorded for task C as of 2026-08-26HOLD — redundant-call count unverified; task cost is reproducible from the registry rate$0.100800

Formula / rule: Matched-run cost = frozen multi-step-task token bill at the gpt-5.6 Sol registry rate, including the tool-call budget. Redundant file reads, repeated identical edits, wasted diff churn, and tool-call count versus a minimal reference trajectory require a matched run, which is not present in the registry, so only the task cost below is reproducible. Source: pricing registry verified 2026-08-26.

Verified 2026-08-08. Data owner: Luna. Run identifiers are per-row match keys; an Unavailable field names the exact missing dated record or matched run and is never inferred as zero. Run the coding evidence scenario →

Batch 21 · security-vulnerability introduction, unfamiliar-codebase comprehension, and build/CI-configuration correctness

Observed benchmark window: 2026-08-26 UTC. Every row is a page-specific frozen fixture with visible controls, a distinct field-level source/run identifier, a registry-computed cost or a scoped Unavailable reason — never a blanket matrix.

1. Seeded security-vulnerability-introduction audit

Frozen fixture / matched runControls (visible inputs)Field observationDecision / boundaryCost (registry-computed or Unavailable)
batch21-code-m1-r1 · Auth-check feature requestfrozen prompt requesting an authentication-check implementation; 900 input tokens; 700 output tokensUnavailable — no matched static-analysis-scored run recorded for the auth-check fixture as of 2026-08-26HOLD — defect rate/false-negative count unverified; fixture cost is reproducible from the registry rate$0.017600
batch21-code-m1-r2 · File-upload feature requestfrozen prompt requesting a file-upload endpoint; 950 input tokens; 800 output tokensUnavailable — no matched static-analysis-scored run recorded for the file-upload fixture as of 2026-08-26HOLD — defect rate/false-negative count unverified; fixture cost is reproducible from the registry rate$0.019800
batch21-code-m1-r3 · SQL-query feature requestfrozen prompt requesting a parameterized SQL query builder; 850 input tokens; 650 output tokensUnavailable — no matched static-analysis-scored run recorded for the SQL-query fixture as of 2026-08-26HOLD — defect rate/false-negative count unverified; fixture cost is reproducible from the registry rate$0.016400

Formula / rule: Matched-run cost = frozen-prompt token bill at the gpt-5.6 Sol registry rate. Static-analysis-scored injection, insecure-deserialization, and access-control defect rates, plus false-negative and false-positive counts, require a matched run of the fixture, which is not present in the registry, so only the fixture cost below is reproducible. Source: pricing registry verified 2026-08-26.

2. Unfamiliar-codebase-comprehension accuracy suite

Frozen fixture / matched runControls (visible inputs)Field observationDecision / boundaryCost (registry-computed or Unavailable)
batch21-code-m2-r1 · Small unfamiliar module — 3 files3-file module, 420 lines; ground-truth maintainer doc withheld from context; 4,800 input tokens; 900 output tokensUnavailable — no matched comprehension-scoring run recorded for the small module as of 2026-08-26HOLD — missed-responsibility/hallucination rate unverified; summary cost is reproducible from the registry rate$0.037200
batch21-code-m2-r2 · Medium unfamiliar module — 8 files8-file module, 1,100 lines; ground-truth maintainer doc withheld from context; 9,600 input tokens; 1,400 output tokensUnavailable — no matched comprehension-scoring run recorded for the medium module as of 2026-08-26HOLD — missed-responsibility/hallucination rate unverified; summary cost is reproducible from the registry rate$0.066400
batch21-code-m2-r3 · Large unfamiliar module — 15 files15-file module, 2,300 lines; ground-truth maintainer doc withheld from context; 16,500 input tokens; 2,100 output tokensUnavailable — no matched comprehension-scoring run recorded for the large module as of 2026-08-26HOLD — missed-responsibility/hallucination rate unverified; summary cost is reproducible from the registry rate$0.108000

Formula / rule: Matched-run cost = frozen previously-unseen-module token bill at the gpt-5.6 Sol registry rate. Scoring a generated summary against ground-truth maintainer documentation for missed responsibilities, hallucinated behavior, and reviewer-corrected accuracy requires a matched run, which is not present in the registry, so only the module-summary cost below is reproducible. Source: pricing registry verified 2026-08-26.

3. Build/CI-configuration generation correctness suite

Frozen fixture / matched runControls (visible inputs)Field observationDecision / boundaryCost (registry-computed or Unavailable)
batch21-code-m3-r1 · Dockerfile for a fixed Node servicefixed multi-stage Node service project shape; 1,600 input tokens; 500 output tokensUnavailable — no matched build-execution run recorded for the Dockerfile fixture as of 2026-08-26HOLD — build-pass rate/repair-turn count unverified; generation cost is reproducible from the registry rate$0.016400
batch21-code-m3-r2 · CI pipeline for a fixed test matrixfixed 3-version test matrix; 1,800 input tokens; 600 output tokensUnavailable — no matched build-execution run recorded for the CI-pipeline fixture as of 2026-08-26HOLD — build-pass rate/repair-turn count unverified; generation cost is reproducible from the registry rate$0.019200
batch21-code-m3-r3 · Combined Dockerfile + CI pipelinefixed multi-stage Node service plus 3-version test matrix; 2,900 input tokens; 950 output tokensUnavailable — no matched combined build-execution run recorded as of 2026-08-26HOLD — build-pass rate/repair-turn count unverified; generation cost is reproducible from the registry rate$0.030600

Formula / rule: Matched-run cost = frozen project-shape token bill at the gpt-5.6 Sol registry rate. Whether the generated Dockerfile and CI pipeline definition produce a working build and test run, plus failure mode and repair-turn count, require a matched build execution, which is not present in the registry, so only the generation cost below is reproducible. Source: pricing registry verified 2026-08-26.

Verified 2026-08-08. Data owner: Luna. Run identifiers are per-row match keys; an Unavailable field names the exact missing dated record or matched run and is never inferred as zero. Run the coding evidence scenario →

Batch 22 · regression-test authoring, pinned-library-version accuracy, and cross-language port fidelity

Observed benchmark window: 2026-08-26 UTC. Every row is a page-specific frozen fixture with visible controls, a distinct field-level source/run identifier, a registry-computed cost or a scoped Unavailable reason — never a blanket matrix.

1. Regression-test-authoring correctness suite

Frozen fixture / matched runControls (visible inputs)Field observationDecision / boundaryCost (registry-computed or Unavailable)
batch22-code-m1-r1 · Off-by-one bug fixturefrozen bug report plus fix diff for an off-by-one defect; 700 input tokens; 350 output tokensUnavailable — no matched test-execution run recorded for the off-by-one fixture as of 2026-08-26HOLD — pre-fix-fail/post-fix-pass outcome unverified; generation cost is reproducible from the registry rate$0.009800
batch22-code-m1-r2 · Null-handling bug fixturefrozen bug report plus fix diff for a null-handling defect; 800 input tokens; 400 output tokensUnavailable — no matched test-execution run recorded for the null-handling fixture as of 2026-08-26HOLD — pre-fix-fail/post-fix-pass outcome unverified; generation cost is reproducible from the registry rate$0.011200
batch22-code-m1-r3 · Race-condition bug fixturefrozen bug report plus fix diff for a race-condition defect; 950 input tokens; 480 output tokensUnavailable — no matched test-execution run recorded for the race-condition fixture as of 2026-08-26HOLD — pre-fix-fail/post-fix-pass outcome unverified; generation cost is reproducible from the registry rate$0.013400

Formula / rule: Matched-run cost = frozen bug-report-plus-fix token bill at the gpt-5.6 Sol registry rate. Whether the generated regression test fails against the pre-fix code and passes against the post-fix code, plus false-pass/false-fail rate, require a matched test-execution run, which is not present in the registry, so only the generation cost below is reproducible. Source: pricing registry verified 2026-08-26.

2. Pinned-library-version accuracy audit

Frozen fixture / matched runControls (visible inputs)Field observationDecision / boundaryCost (registry-computed or Unavailable)
batch22-code-m2-r1 · Single pinned dependencyfrozen request to pin 1 named library to an exact stated version; 400 input tokens; 120 output tokensUnavailable — no matched manifest-verification run recorded for the single-dependency fixture as of 2026-08-26HOLD — exact-pin accuracy unverified; generation cost is reproducible from the registry rate$0.004000
batch22-code-m2-r2 · 8 pinned dependenciesfrozen request to pin 8 named libraries to exact stated versions; 900 input tokens; 400 output tokensUnavailable — no matched manifest-verification run recorded for the 8-dependency fixture as of 2026-08-26HOLD — exact-pin accuracy unverified; generation cost is reproducible from the registry rate$0.011600
batch22-code-m2-r3 · 20 pinned dependenciesfrozen request to pin 20 named libraries to exact stated versions; 1,800 input tokens; 900 output tokensUnavailable — no matched manifest-verification run recorded for the 20-dependency fixture as of 2026-08-26HOLD — exact-pin accuracy unverified; generation cost is reproducible from the registry rate$0.025200

Formula / rule: Matched-run cost = frozen dependency-request token bill at the gpt-5.6 Sol registry rate. Whether the generated dependency manifest pins the exact requested library version, and any drift toward a newer or hallucinated version number, require a matched manifest-verification run, which is not present in the registry, so only the generation cost below is reproducible. Source: pricing registry verified 2026-08-26.

3. Cross-language port-fidelity suite

Frozen fixture / matched runControls (visible inputs)Field observationDecision / boundaryCost (registry-computed or Unavailable)
batch22-code-m3-r1 · Small utility function port20-line source function; frozen target-language port request; 500 input tokens; 300 output tokensUnavailable — no matched behavioral-equivalence run recorded for the small-function port as of 2026-08-26HOLD — equivalence/edge-case drift unverified; port-generation cost is reproducible from the registry rate$0.008000
batch22-code-m3-r2 · Medium module port120-line source module; frozen target-language port request; 1,600 input tokens; 1,000 output tokensUnavailable — no matched behavioral-equivalence run recorded for the medium-module port as of 2026-08-26HOLD — equivalence/edge-case drift unverified; port-generation cost is reproducible from the registry rate$0.026400
batch22-code-m3-r3 · Large module port with external calls400-line source module with external library calls; frozen target-language port request; 4,200 input tokens; 2,600 output tokensUnavailable — no matched behavioral-equivalence run recorded for the large-module port as of 2026-08-26HOLD — equivalence/edge-case drift unverified; port-generation cost is reproducible from the registry rate$0.068800

Formula / rule: Matched-run cost = frozen source-function token bill at the gpt-5.6 Sol registry rate, ported to a fixed target language. Behavioral-equivalence scoring against a matched test harness in the target language, plus edge-case-handling drift, require a matched execution run, which is not present in the registry, so only the port-generation cost below is reproducible. Source: pricing registry verified 2026-08-26.

Verified 2026-08-08. Data owner: Luna. Run identifiers are per-row match keys; an Unavailable field names the exact missing dated record or matched run and is never inferred as zero. Run the coding evidence scenario →

Batch 23 · regression-test authoring, pinned-library-version accuracy, and cross-language port fidelity

Observed benchmark window: 2026-08-26 UTC. Every row is a page-specific frozen fixture with visible controls, a distinct field-level source/run identifier, a registry-computed cost or a scoped Unavailable reason — never a blanket matrix.

1. Regression-test-authoring correctness suite

Frozen fixture / matched runControls (visible inputs)Field observationDecision / boundaryCost (registry-computed or Unavailable)
batch23-code-m1-r1 · Off-by-one bug fixturefrozen bug report plus fix diff for an off-by-one defect; 700 input tokens; 350 output tokensUnavailable — no matched test-execution run recorded for the off-by-one fixture as of 2026-08-26HOLD — pre-fix-fail/post-fix-pass outcome unverified; generation cost is reproducible from the registry rate$0.009800
batch23-code-m1-r2 · Null-handling bug fixturefrozen bug report plus fix diff for a null-handling defect; 800 input tokens; 400 output tokensUnavailable — no matched test-execution run recorded for the null-handling fixture as of 2026-08-26HOLD — pre-fix-fail/post-fix-pass outcome unverified; generation cost is reproducible from the registry rate$0.011200
batch23-code-m1-r3 · Race-condition bug fixturefrozen bug report plus fix diff for a race-condition defect; 950 input tokens; 480 output tokensUnavailable — no matched test-execution run recorded for the race-condition fixture as of 2026-08-26HOLD — pre-fix-fail/post-fix-pass outcome unverified; generation cost is reproducible from the registry rate$0.013400

Formula / rule: Matched-run cost = frozen bug-report-plus-fix token bill at the gpt-5.6 Sol registry rate. Whether the generated regression test fails against the pre-fix code and passes against the post-fix code, plus false-pass/false-fail rate, require a matched test-execution run, which is not present in the registry, so only the generation cost below is reproducible. Source: pricing registry verified 2026-08-26.

2. Pinned-library-version accuracy audit

Frozen fixture / matched runControls (visible inputs)Field observationDecision / boundaryCost (registry-computed or Unavailable)
batch23-code-m2-r1 · Single pinned dependencyfrozen request to pin 1 named library to an exact stated version; 400 input tokens; 120 output tokensUnavailable — no matched manifest-verification run recorded for the single-dependency fixture as of 2026-08-26HOLD — exact-pin accuracy unverified; generation cost is reproducible from the registry rate$0.004000
batch23-code-m2-r2 · 8 pinned dependenciesfrozen request to pin 8 named libraries to exact stated versions; 900 input tokens; 400 output tokensUnavailable — no matched manifest-verification run recorded for the 8-dependency fixture as of 2026-08-26HOLD — exact-pin accuracy unverified; generation cost is reproducible from the registry rate$0.011600
batch23-code-m2-r3 · 20 pinned dependenciesfrozen request to pin 20 named libraries to exact stated versions; 1,800 input tokens; 900 output tokensUnavailable — no matched manifest-verification run recorded for the 20-dependency fixture as of 2026-08-26HOLD — exact-pin accuracy unverified; generation cost is reproducible from the registry rate$0.025200

Formula / rule: Matched-run cost = frozen dependency-request token bill at the gpt-5.6 Sol registry rate. Whether the generated dependency manifest pins the exact requested library version, and any drift toward a newer or hallucinated version number, require a matched manifest-verification run, which is not present in the registry, so only the generation cost below is reproducible. Source: pricing registry verified 2026-08-26.

3. Cross-language port-fidelity suite

Frozen fixture / matched runControls (visible inputs)Field observationDecision / boundaryCost (registry-computed or Unavailable)
batch23-code-m3-r1 · Small utility function port20-line source function; frozen target-language port request; 500 input tokens; 300 output tokensUnavailable — no matched behavioral-equivalence run recorded for the small-function port as of 2026-08-26HOLD — equivalence/edge-case drift unverified; port-generation cost is reproducible from the registry rate$0.008000
batch23-code-m3-r2 · Medium module port120-line source module; frozen target-language port request; 1,600 input tokens; 1,000 output tokensUnavailable — no matched behavioral-equivalence run recorded for the medium-module port as of 2026-08-26HOLD — equivalence/edge-case drift unverified; port-generation cost is reproducible from the registry rate$0.026400
batch23-code-m3-r3 · Large module port with external calls400-line source module with external library calls; frozen target-language port request; 4,200 input tokens; 2,600 output tokensUnavailable — no matched behavioral-equivalence run recorded for the large-module port as of 2026-08-26HOLD — equivalence/edge-case drift unverified; port-generation cost is reproducible from the registry rate$0.068800

Formula / rule: Matched-run cost = frozen source-function token bill at the gpt-5.6 Sol registry rate, ported to a fixed target language. Behavioral-equivalence scoring against a matched test harness in the target language, plus edge-case-handling drift, require a matched execution run, which is not present in the registry, so only the port-generation cost below is reproducible. Source: pricing registry verified 2026-08-26.

Verified 2026-08-08. Data owner: Luna. Run identifiers are per-row match keys; an Unavailable field names the exact missing dated record or matched run and is never inferred as zero. Run the coding evidence scenario →

Batch 24 · database migration safety, API backward compatibility, and semantic merge-conflict resolution

Observed benchmark window: 2026-08-27 UTC. Every row is a frozen fixture with visible controls, a distinct field-level source/run ID, a registry-computed baseline or scoped Unavailable state, and a named decision boundary.

1. database-schema migration safety suite

Frozen fixture / matched runControls (visible inputs)Field observationDecision / boundaryCost (registry-computed or Unavailable)
batch24-code-m1-r1 · Small fixtureSeeded small repository; fixed task controls; 1,200 input; 500 output tokensUnavailable — no matched database-schema migration safety suite small run or dated rate recorded as of 2026-08-27HOLD — acceptance outcome unverified; fixture cost is reproducible$0.014800
batch24-code-m1-r2 · Medium fixtureSeeded medium repository; fixed task controls; 3,000 input; 1,200 output tokensUnavailable — no matched database-schema migration safety suite medium run or dated rate recorded as of 2026-08-27HOLD — reviewer corrections and repair scope unverified$0.036000
batch24-code-m1-r3 · Large fixtureSeeded large repository; fixed task controls; 8,000 input; 3,000 output tokensUnavailable — no matched database-schema migration safety suite large run or dated rate recorded as of 2026-08-27HOLD — build/test behavior and accepted spend unverified$0.092000

Formula / scoring rule: Matched-run cost = frozen coding fixture token bill at the gpt-5.6 Sol registry rate. The outcome score is accepted only from the named matched repository run; no cross-suite score is transferred. Source: pricing registry verified 2026-08-27.

2. API backward-compatibility gate

Frozen fixture / matched runControls (visible inputs)Field observationDecision / boundaryCost (registry-computed or Unavailable)
batch24-code-m2-r1 · Small fixtureSeeded small repository; fixed task controls; 1,200 input; 500 output tokensUnavailable — no matched API backward-compatibility gate small run or dated rate recorded as of 2026-08-27HOLD — acceptance outcome unverified; fixture cost is reproducible$0.014800
batch24-code-m2-r2 · Medium fixtureSeeded medium repository; fixed task controls; 3,000 input; 1,200 output tokensUnavailable — no matched API backward-compatibility gate medium run or dated rate recorded as of 2026-08-27HOLD — reviewer corrections and repair scope unverified$0.036000
batch24-code-m2-r3 · Large fixtureSeeded large repository; fixed task controls; 8,000 input; 3,000 output tokensUnavailable — no matched API backward-compatibility gate large run or dated rate recorded as of 2026-08-27HOLD — build/test behavior and accepted spend unverified$0.092000

Formula / scoring rule: Matched-run cost = frozen coding fixture token bill at the gpt-5.6 Sol registry rate. The outcome score is accepted only from the named matched repository run; no cross-suite score is transferred. Source: pricing registry verified 2026-08-27.

3. semantic merge-conflict resolution benchmark

Frozen fixture / matched runControls (visible inputs)Field observationDecision / boundaryCost (registry-computed or Unavailable)
batch24-code-m3-r1 · Small fixtureSeeded small repository; fixed task controls; 1,200 input; 500 output tokensUnavailable — no matched semantic merge-conflict resolution benchmark small run or dated rate recorded as of 2026-08-27HOLD — acceptance outcome unverified; fixture cost is reproducible$0.014800
batch24-code-m3-r2 · Medium fixtureSeeded medium repository; fixed task controls; 3,000 input; 1,200 output tokensUnavailable — no matched semantic merge-conflict resolution benchmark medium run or dated rate recorded as of 2026-08-27HOLD — reviewer corrections and repair scope unverified$0.036000
batch24-code-m3-r3 · Large fixtureSeeded large repository; fixed task controls; 8,000 input; 3,000 output tokensUnavailable — no matched semantic merge-conflict resolution benchmark large run or dated rate recorded as of 2026-08-27HOLD — build/test behavior and accepted spend unverified$0.092000

Formula / scoring rule: Matched-run cost = frozen coding fixture token bill at the gpt-5.6 Sol registry rate. The outcome score is accepted only from the named matched repository run; no cross-suite score is transferred. Source: pricing registry verified 2026-08-27.

Verified 2026-08-08. Data owner: Luna. Unavailable fields name their exact missing dated record or matched run and are never inferred as zero. Run the coding evidence scenario →

Batch 25 · Infrastructure-as-code safety, observability instrumentation correctness, and web-accessibility remediation

Observed benchmark window: 2026-08-27 UTC. Frozen inputs, field-level run IDs, reproducible formulas, provenance, and fail-closed evidence decisions are rendered in the initial server response.

1. Infrastructure-as-code change-safety suite

Frozen fixture / matched runControls (visible inputs)Field observationDecision / boundaryCost breakdown
batch25-code-m1-r1 · Terraform fixture · observed 2026-08-27Frozen Terraform request; plan alignment, state/import risk, destructive disclosure, and rollback checksTerraform: plan alignment 18/18; immutable fields 4/4; least privilege 9/9; rollback 3/3; 0 unsafe changes · run batch25-code-m1-r1 · observed 2026-08-27PASS — accepted without reviewer correctionmodel 1200×$4.00/M + 500×$20.00/M = $0.014800; specialized units = $0.000000; total = $0.014800
batch25-code-m1-r2 · Kubernetes fixture · observed 2026-08-27Frozen Kubernetes request; immutable fields, dependency ordering, least privilege, and policy checksKubernetes: dependency order 12/12; immutable-field disclosure 2/2; policy checks 8/8; 1 correction · run batch25-code-m1-r2 · observed 2026-08-27PASS — correction applied; final patch passed 27/27 checksmodel 1400×$4.00/M + 600×$20.00/M = $0.017600; specialized units = $0.000000; total = $0.017600
batch25-code-m1-r3 · Cross-stack fixture · observed 2026-08-27Terraform + Kubernetes request; reviewer corrections, repair turns, accepted patch, and costcross-stack: state/import risk 5/6; destructive disclosure 3/3; rollback 4/4; reviewer score 92/100 · run batch25-code-m1-r3 · observed 2026-08-27BOUNDARY — merge only with explicit import plan for one resourcemodel 2400×$4.00/M + 900×$20.00/M = $0.027600; specialized units = $0.000000; total = $0.027600

Formula / scoring rule: Acceptance score = weighted plan alignment + immutable-field handling + dependency ordering + least privilege + rollback/policy checks − unsafe/destructive disclosures; cost is the matched run bill. Source: pricing registry and dated evidence index verified 2026-08-27.

2. Observability-instrumentation correctness benchmark

Frozen fixture / matched runControls (visible inputs)Field observationDecision / boundaryCost breakdown
batch25-code-m2-r1 · HTTP service · observed 2026-08-27Seeded HTTP service; trace context, metrics, labels, logs, errors, sampling, and repair turnsHTTP: trace propagation 12/12; metric units 14/14; labels 0 high-cardinality; golden queries 6/6; 1 repair · run batch25-code-m2-r1 · observed 2026-08-27PASS — accepted patch after one instrumentation repairmodel 1400×$4.00/M + 600×$20.00/M = $0.017600; specialized units = $0.000000; total = $0.017600
batch25-code-m2-r2 · Queue service · observed 2026-08-27Seeded queue service; async propagation, status coverage, query validity, and correctionsqueue: async context 9/10; error/status coverage 18/18; query validity 5/5; 2 corrections · run batch25-code-m2-r2 · observed 2026-08-27PASS — final async propagation 10/10model 1800×$4.00/M + 700×$20.00/M = $0.021200; specialized units = $0.000000; total = $0.021200
batch25-code-m2-r3 · Background job · observed 2026-08-27Seeded background job; correlation, cardinality, golden signals, accepted patch, and spendbackground job: trace/log correlation 11/11; cardinality 0 violations; golden signals 4/4; 96/100 · run batch25-code-m2-r3 · observed 2026-08-27PASS — accepted patch; no regression in seeded testsmodel 2200×$4.00/M + 900×$20.00/M = $0.026800; specialized units = $0.000000; total = $0.026800

Formula / scoring rule: Score = trace propagation + metric type/unit + label-cardinality + log correlation + error coverage + valid golden-signal queries, minus repairs; accepted-patch cost uses returned tokens. Source: pricing registry and dated evidence index verified 2026-08-27.

3. Web-accessibility remediation gate

Frozen fixture / matched runControls (visible inputs)Field observationDecision / boundaryCost breakdown
batch25-code-m3-r1 · React forms · observed 2026-08-27Frozen React form; name computation, keyboard flow, contrast, automated + manual checksReact forms: accessible names 16/16; keyboard path 8/8; contrast 12/12; axe 0 violations; manual 5/5 · run batch25-code-m3-r1 · observed 2026-08-27PASS — merge approved; 0 regressionsmodel 1400×$4.00/M + 600×$20.00/M = $0.017600; specialized units = $0.000000; total = $0.017600
batch25-code-m3-r2 · Dialogs/tables · observed 2026-08-27Frozen dialog/table; focus trap, headers, ARIA validity, and regression countdialogs/tables: focus return 4/4; headers 22/22; ARIA validity 18/18; 1 manual correction; 0 regressions · run batch25-code-m3-r2 · observed 2026-08-27PASS — correction verified by keyboard replaymodel 1800×$4.00/M + 700×$20.00/M = $0.021200; specialized units = $0.000000; total = $0.021200
batch25-code-m3-r3 · Navigation · observed 2026-08-27Frozen navigation; keyboard path, landmarks, manual task completion, patch scope, and spendnavigation: landmarks 7/7; keyboard route 14/14; manual task 9/10; contrast 11/11 · run batch25-code-m3-r3 · observed 2026-08-27BOUNDARY — one skip-link task fails; hold release pending repairmodel 2200×$4.00/M + 900×$20.00/M = $0.026800; specialized units = $0.000000; total = $0.026800

Formula / scoring rule: Gate passes only when semantic structure, accessible names, keyboard/focus behavior, ARIA validity, contrast, automated checks, and manual user tasks all pass with no regression; cost is matched spend. Source: pricing registry and dated evidence index verified 2026-08-27.

Verified 2026-08-08. Data owner: Luna. Specialized rates and unmatched observations are never inferred from a base modality. Run the coding evidence scenario →

Batch 26 · Internationalization, transactional SQL, and numerical-computing correctness

Frozen verification window: 2026-08-27 UTC. Every row is an initial-response fixture with a field-level run ID, visible controls, method, result or narrowly scoped unavailable state, and dated provenance.

1. Internationalization remediation suite

Frozen fixture / runVisible controlsField-level observationDecision boundaryReproducible cost / state
React settings page
batch26-coding-m1-r1
observed 2026-08-27		en/fr/ar; plural/select; snapshotshard-coded strings 18/18 extracted; plural 6/6; interpolation 9/9; RTL hook 4/4PASS — accepted after 0 correctionstokens: (1420×$2.50 + 612×$10.00)/1M = $0.009670
server-rendered invoice
batch26-coding-m1-r2
observed 2026-08-27		date/number locale; fallback keys; SSR replaycurrency/date 14/14; fallback 8/8; 1 missing snapshot repairedPASS WITH REPAIR — snapshot must remain in gatetokens: (1760×$2.50 + 704×$10.00)/1M = $0.011440
RTL dashboard
batch26-coding-m1-r3
observed 2026-08-27		ar/he; interpolation; visual snapshotsstrings 27/27; RTL layout hooks 11/12; 2 overflow regressionsBOUNDARY — hold until RTL overflow is repairedtokens: (2140×$2.50 + 882×$10.00)/1M = $0.014170

Formula / scoring rule: Score = extracted strings + plural/select correctness + interpolation safety + locale formatting + RTL/fallback/snapshot checks − reviewer corrections; cost is matched run spend. Source: pricing registry and dated evidence index verified 2026-08-27.

2. Transactional SQL correctness benchmark

Frozen fixture / runVisible controlsField-level observationDecision boundaryReproducible cost / state
inventory decrement
batch26-coding-m2-r1
observed 2026-08-27		two concurrent writers; isolation; rollbacklost updates 0/100; lock order 4/4; rollback 10/10; plan delta +2%PASS — transaction preserves inventory invarianttokens: (1680×$2.50 + 680×$10.00)/1M = $0.011000
ledger transfer
batch26-coding-m2-r2
observed 2026-08-27		write-skew seed; retry; idempotency keywrite-skew 0/100; retries 8; duplicate effects 0; reviewer 10/10PASS — retry-safe transfertokens: (2020×$2.50 + 812×$10.00)/1M = $0.013170
job queue claim
batch26-coding-m2-r3
observed 2026-08-27		deadlock seed; rollback; EXPLAIN regressiondeadlocks 2/100; duplicate claims 0; plan regression +19%; 1 repairBOUNDARY — release held by query-plan regressiontokens: (2380×$2.50 + 924×$10.00)/1M = $0.015190

Formula / scoring rule: Gate = isolation + lost-update/write-skew prevention + lock ordering + retry/idempotency + rollback + plan regression; seeded concurrent outcomes are mandatory. Source: pricing registry and dated evidence index verified 2026-08-27.

3. Numerical-computing implementation gate

Frozen fixture / runVisible controlsField-level observationDecision boundaryReproducible cost / state
floating-point summation
batch26-coding-m3-r1
observed 2026-08-27		1M values; compensated sum; tolerancerelative error 2.1e-12; NaN handling 3/3; property tests 12/12PASS — error below 1e-9 gatetokens: (1340×$2.50 + 540×$10.00)/1M = $0.008750
linear solver/interpolation
batch26-coding-m3-r2
observed 2026-08-27		condition estimate; residual; vectorized pathresidual 4.8e-10; conditioning disclosed; performance +6%; tests 18/18PASS — stable implementation acceptedtokens: (1900×$2.50 + 760×$10.00)/1M = $0.012350
array overflow edge case
batch26-coding-m3-r3
observed 2026-08-27		NaN/Inf seed; reference result; guardrailoverflow guard 5/6; reference error 2.4e-5; performance −18%; repair pendingBOUNDARY — reject until overflow and performance regressions are fixedtokens: (2220×$2.50 + 910×$10.00)/1M = $0.014650

Formula / scoring rule: Pass requires tolerance design, stability/conditioning, NaN/overflow handling, reference error, property tests, and performance guardrail; cost is matched spend. Source: pricing registry and dated evidence index verified 2026-08-27.

Verified 2026-08-08. Data owner: Luna. Missing specialized units, rates, and matched runs are never inferred from a neighboring modality or provider. Run the coding Batch 26 evidence scenario →

Batch 27 · Civil-time, event-delivery, and parser/compiler implementation gates

Frozen verification window: 2026-08-27 UTC. Every row is an initial-response fixture with visible inputs, a field-level run ID, a reproducible method/result or narrowly scoped unavailable state, dated provenance, and a decision boundary.

1. Timezone and daylight-saving correctness suite

Frozen fixture / runVisible inputsField-level observationDecision boundaryReproducible cost / state
spring-forward recurring event
batch27-coding-m1-r1
observed 2026-08-27		America/New_York; 2026-03-08 02:30; recurrence rulenonexistent local time rejected with explicit policy; UTC instant preserved; 18/18 property testsPASS — gap handling is explicit2,840 input / 612 output; accepted patch cost $0.007732
fall-back billing cutoff
batch27-coding-m1-r2
observed 2026-08-27		Europe/Berlin; fold=first/second; cutoff migrationboth instants represented; duplicate charge prevented; reviewer acceptedPASS — fold choice is persisted3,220 input / 740 output; accepted patch cost $0.008450
leap day and tzdb change
batch27-coding-m1-r3
observed 2026-08-27		Pacific/Auckland; 2028-02-29; tzdb pinned/unpinnedoffset migration correct; unpinned dependency causes 2 snapshot failures; repair pendingBOUNDARY — reject until tzdb version is pinned3,880 input / 924 output; cost unavailable — matched-run rate tuple not frozen; repair required

Formula / scoring rule: Score = instant/local-time preservation + ambiguity handling + tzdb pinning + property tests − reviewer corrections and unsafe assumptions. Source: pricing registry and dated evidence index verified 2026-08-27.

2. Event-consumer delivery-semantics benchmark

Frozen fixture / runVisible inputsField-level observationDecision boundaryReproducible cost / state
duplicate after ack timeout
batch27-coding-m2-r1
observed 2026-08-27		at-least-once; idempotency key; crash after effect2 deliveries; one committed effect; dedupe key durable; integration PASSPASS — duplicate delivery is harmless4,120 input / 804 output; accepted patch cost $0.010552
reordered and delayed events
batch27-coding-m2-r2
observed 2026-08-27		sequence 3,1,2; 30s delay; optimistic versionstale event quarantined; sequence 1/2 applied; repair path testedPASS WITH REPAIR — late event is not silently applied4,640 input / 912 output; accepted patch cost $0.011440
poison timeout and DLQ
batch27-coding-m2-r3
observed 2026-08-27		timeout; crash-after-side-effect; retry budget 5effect committed twice in one candidate; DLQ exists but atomic boundary missingBOUNDARY — reject duplicate side-effect implementation5,020 input / 1,080 output; cost unavailable — matched-run rate tuple not frozen; failed gate

Formula / scoring rule: Gate = idempotency + deduplication + atomic effect/state boundary + retry/DLQ behavior − duplicate side effects. Source: pricing registry and dated evidence index verified 2026-08-27.

3. Parser/compiler implementation gate

Frozen fixture / runVisible inputsField-level observationDecision boundaryReproducible cost / state
expression grammar
batch27-coding-m3-r1
observed 2026-08-27		precedence table; unary/binary; source spans; 42 fixtures42/42 parse; AST invariants 42/42; locations exactPASS — grammar and AST gate pass3,980 input / 860 output; accepted patch cost $0.010420
configuration DSL
batch27-coding-m3-r2
observed 2026-08-27		comments; interpolation; duplicate keys; recovery31/34 valid fixtures; 3 diagnostics point to token; round-trip 28/28PASS WITH REPAIR — three duplicate-key diagnostics corrected$0.021250 = (4420×$2.50 + 1020×$10.00)/1M
source transform malformed input
batch27-coding-m3-r3
observed 2026-08-27		nested syntax error; fuzz 10K; performance budgetfuzz finds panic at depth 64; no unsafe output; repair incompleteBOUNDARY — hold merge until malformed input is total5,100 input / 1,240 output; cost state unavailable

Formula / scoring rule: Score = lexer/parser coverage + precedence + locations + recovery + AST/property invariants + performance − malformed-input failures. Source: pricing registry and dated evidence index verified 2026-08-27.

Verified 2026-08-08. Data owner: Luna. Missing specialized units, rates, and matched runs are never inferred from a neighboring modality or provider. Run the coding Batch 27 evidence scenario →

Batch 28 · Spreadsheet formulas, geospatial implementation, and binary protocol gates

Frozen verification window: 2026-08-27 UTC. These are server-rendered matched fixtures, not live estimates. Each row exposes inputs, a reproducible formula/result or a narrowly scoped missing record, dated provenance, and a decision boundary.

1. Spreadsheet-formula generation and repair suite

Frozen fixture / runVisible inputsField-level observationDecision boundaryReproducible cost / state
financial lookup workbook
batch28-coding-m1-r1
observed 2026-08-27		USD/EUR; XLOOKUP; absolute tax table; 42 cells42/42 recalculated; locale separator preserved; no circular referencePASS — accepted workbook is reproducible$0.014350 = (2860×$2.50 + 720×$10.00)/1M
dynamic-array/date repair
batch28-coding-m1-r2
observed 2026-08-27		FILTER; spill range; leap date mutation5/6 mutations repaired; one relative reference corrected by reviewerPASS WITH REPAIR — publish after correction$0.017250 = (3380×$2.50 + 880×$10.00)/1M
cross-sheet circular edge
batch28-coding-m1-r3
observed 2026-08-27		three sheets; seeded circular dependencyformula parses but recalculation errors; repair incompleteBOUNDARY — reject workbook with unresolved cycleUnavailable — accepted repaired workbook cost

Formula / scoring rule: Score = locale-safe syntax + reference/dependency integrity + recalculation + seeded repair − circular references. Source: pricing registry and dated evidence index verified 2026-08-27.

2. Geospatial implementation benchmark

Frozen fixture / runVisible inputsField-level observationDecision boundaryReproducible cost / state
CRS transformation and distance
batch28-coding-m2-r1
observed 2026-08-27		EPSG:4326/3857; axis order; 100 fixtures100/100 coordinates within 1m; units explicit; property tests passPASS — CRS and units are preserved$0.016350 = (3180×$2.50 + 840×$10.00)/1M
antimeridian/polar geofence
batch28-coding-m2-r2
observed 2026-08-27		dateline crossing; polar cap; bbox and geofenceantimeridian split correct; polar bbox repaired; 18/20 acceptancePASS WITH REPAIR — retain polar limitation$0.020000 = (3920×$2.50 + 1020×$10.00)/1M
spatial-index performance
batch28-coding-m2-r3
observed 2026-08-27		1M points; p95 guardrail; index querycorrectness passes but benchmark artifact is missingBOUNDARY — no performance-qualified patchUnavailable — dated performance artifact and accepted-patch rate

Formula / scoring rule: Score = CRS/axis/unit fidelity + topology invariants + fixture/property tests + performance − repair defects. Source: pricing registry and dated evidence index verified 2026-08-27.

3. Binary-protocol parser and serializer gate

Frozen fixture / runVisible inputsField-level observationDecision boundaryReproducible cost / state
length-prefixed tagged frames
batch28-coding-m3-r1
observed 2026-08-27		little/big endian; 64 valid frames64/64 round trips; boundary lengths exact; unknown tags preservedPASS — serializer is inverse on valid fixtures$0.017550 = (3420×$2.50 + 900×$10.00)/1M
checksummed versioned stream
batch28-coding-m3-r2
observed 2026-08-27		v1/v2; checksum; incremental chunks31/32 valid; one checksum repair; version field retainedPASS WITH REPAIR — reject checksum mismatch$0.021000 = (4080×$2.50 + 1080×$10.00)/1M
malformed fuzz stream
batch28-coding-m3-r3
observed 2026-08-27		10K fuzz cases; truncation and oversized lengthpanic at depth 64; unsafe output absent; repair incompleteBOUNDARY — parser is not total on malformed inputUnavailable — accepted malformed-input repair run

Formula / scoring rule: Gate = endian/boundary + round-trip + unknown-field preservation + incremental parsing + corruption rejection − fuzz failures. Source: pricing registry and dated evidence index verified 2026-08-27.

Verified 2026-08-08. Data owner: Luna. Missing specialized units, rates, and matched runs are never inferred from a neighboring modality or provider. Run the coding Batch 28 evidence scenario →

Batch 29 · GPU kernels, embedded firmware, and distributed consensus

Frozen verification window: 2026-08-27 UTC. These are server-rendered matched fixtures, not live estimates. Each row exposes frozen inputs, a reproducible formula/result or a narrowly scoped missing record, dated provenance, and a decision boundary.

1. GPU-kernel implementation and optimization suite

Frozen fixture / runVisible inputsField-level observationDecision boundaryReproducible cost / state
reduction and matrix tile
batch29-coding-m1-r1
observed 2026-08-27		CUDA-compatible fixtures; tolerance; compiler/profiler; matched hardwarereference tests pass; no race/bounds defects; memory traffic and speedup artifact retainedPASS — accept only with matched-hardware evidence$0.014350 = (2860×$2.50 + 720×$10.00)/1M
stencil and attention-style kernel
batch29-coding-m1-r2
observed 2026-08-27		shared memory; launch geometry; deterministic seed; repair budgetone indexing repair; compiler warnings clear; speedup guardrail met after repairPASS WITH REPAIR — publish repaired kernel only$0.018600 = (3920×$2.50 + 880×$10.00)/1M
missing profiler artifact
batch29-coding-m1-r3
observed 2026-08-27		numerical tests pass; speedup/profiler record absentcorrectness alone cannot establish optimization qualificationBOUNDARY — no cost-per-accepted-kernel claimUnavailable — dated profiler evidence and matched-hardware speedup

Formula / scoring rule: Score = numerical tolerance + race/bounds safety + launch/index correctness + deterministic tests + profiler evidence + guarded speedup − repair defects; cost = bill ÷ accepted kernels. Source: pricing registry and dated evidence index verified 2026-08-27.

2. Embedded-firmware and peripheral-driver benchmark

Frozen fixture / runVisible inputsField-level observationDecision boundaryReproducible cost / state
interrupt, DMA, and ring buffer
batch29-coding-m2-r1
observed 2026-08-27		mock MCU; interrupt/DMA fixtures; bounded memory; toolchain warningsregister fields and volatile access correct; mocked tests pass; no unbounded allocationPASS — device semantics are evidenced$0.016350 = (3180×$2.50 + 840×$10.00)/1M
I2C/SPI, watchdog, and power state
batch29-coding-m2-r2
observed 2026-08-27		fault injection; watchdog timeout; sleep/wake; reviewer correctionone timing-state repair; bus recovery and watchdog tests pass; warnings retainedPASS WITH REPAIR — keep fault trace with patch$0.020450 = (4260×$2.50 + 980×$10.00)/1M
hardware-boundary gap
batch29-coding-m2-r3
observed 2026-08-27		peripheral code compiles; no mocked hardware trace or timing artifactsource review cannot prove device correctnessBOUNDARY — generic concurrency score cannot substituteUnavailable — mocked peripheral trace and timing-state acceptance

Formula / scoring rule: Score = register/bitfield + volatile/concurrency + bounded memory + timing state machine + mocked fault recovery + warning cleanliness − reviewer defects. Source: pricing registry and dated evidence index verified 2026-08-27.

3. Distributed-consensus protocol gate

Frozen fixture / runVisible inputsField-level observationDecision boundaryReproducible cost / state
Raft election and replication
batch29-coding-m3-r1
observed 2026-08-27		seeded nodes; elections; log replication; duplicate/reordered messagesterm/index and quorum invariants pass; duplicate delivery is idempotentPASS — simulator trace is reproducible$0.017550 = (3420×$2.50 + 900×$10.00)/1M
membership change and snapshot
batch29-coding-m3-r2
observed 2026-08-27		joint consensus; snapshot/install; partition and recoveryone snapshot boundary repair; liveness restored; failing trace localizedPASS WITH REPAIR — retain partition trace$0.021000 = (4080×$2.50 + 1080×$10.00)/1M
model-check gap
batch29-coding-m3-r3
observed 2026-08-27		partition scenario; simulator result incomplete; patch proposedsafety appears plausible but model-check outcome is absentBOUNDARY — no accepted consensus-patch costUnavailable — model-check/simulator outcome and accepted patch run

Formula / scoring rule: Gate = safety/liveness invariants + term/index + quorum math + reorder/duplicate handling + model-check trace localization − unresolved partition failures. Source: pricing registry and dated evidence index verified 2026-08-27.

Verified 2026-08-08. Data owner: Luna. Missing specialized units, rates, and matched runs are never inferred from a neighboring modality, provider, or prior batch. Run the coding Batch 29 evidence scenario →

Batch 30 · ML pipelines, allocator/GC safety, and DSP implementation

Frozen verification window: 2026-08-27 UTC. These server-rendered fixtures expose inputs, formulas, field-level observations, decision boundaries, dated provenance, and exact bills where the registry closes the token tuple. Missing specialist evidence is explicitly Unavailable.

1. Machine-learning training-pipeline correctness suite

Frozen fixture / runVisible inputsField-level observationDecision boundaryReproducible cost / state
Frozen split pipeline
batch30-coding-m1-r1
observed 2026-08-27		12,000 rows; seed 41; train/validation/test hashes; 2026-08-27T16:52ZNo leakage; shape [12000,64] stable; seed rerun metric delta 0.0007; reviewer accepted patch and 14/14 tests.PASS — reproducibility and split invariants close$0.014600 = (3120×$2.50 + 680×$10.00)/1M
Checkpoint resume
batch30-coding-m1-r2
observed 2026-08-27		epoch 8 checkpoint; resume to epoch 20; metric ledger; 2026-08-27T17:08ZResume loss curve differs <0.002; optimizer state hash restored; metric names align; reviewer repaired one mislabeled F1 column; 4,860/920 tokens.PASS WITH REPAIR — corrected metric label is retained$0.021350 = (4860×$2.50 + 920×$10.00)/1M
Distributed boundary
batch30-coding-m1-r3
observed 2026-08-27		4 workers; 2 GPUs each; 50k rows; memory ceiling 14 GB; 2026-08-27T17:24ZAll-reduce shape passes; peak 13.6 GB; run completes 3.8% faster; one nondeterministic test remains; 7,420/1,180 tokens.BOUNDARY — resource result observed, clean reproducibility not qualified$0.030350 = (7420×$2.50 + 1180×$10.00)/1M

Formula / scoring rule: Acceptance = no leakage + seed/environment reproducibility + tensor/data-shape invariants + metric correctness + resume equivalence + resource bounds + tests + reviewer acceptance. Source: pricing registry and dated evidence index verified 2026-08-27; provider registry: OpenAI GPT-4o-mini / coding registry rate verified 2026-08-27; test suite: Batch 30 ML pipeline correctness fixture/test suite (run and result recorded 2026-08-27).

2. Memory-allocator and garbage-collector implementation benchmark

Frozen fixture / runVisible inputsField-level observationDecision boundaryReproducible cost / state
Arena/free-list trace
batch30-coding-m2-r1
observed 2026-08-27		10,000 alloc/free operations; 16-byte alignment; ASan; 2026-08-27T17:41ZNo overlap or use-after-free; 16-byte alignment 10,000/10,000; throughput 2.1M ops/s; reviewer accepted.PASS — safety trace and throughput bound met$0.012700 = (2840×$2.50 + 560×$10.00)/1M
Generational compaction
batch30-coding-m2-r2
observed 2026-08-27		50k objects; 3 generations; compaction pause <20 ms; 2026-08-27T17:56ZLive pointers preserved; max pause 17.4 ms; fragmentation 8.2%; model-check finds no root loss; 4,220/860 tokens.PASS — pause and lifetime invariants met$0.019150 = (4220×$2.50 + 860×$10.00)/1M
Concurrent roots stress
batch30-coding-m2-r3
observed 2026-08-27		8 mutators, 2 collectors, 1M operations; 2026-08-27T18:13ZTSAN reports one race in remembered-set resize; sanitizer artifact and failing trace retained; 6,180/1,020 tokens.REJECT — race blocks accepted implementation$0.025650 = (6180×$2.50 + 1020×$10.00)/1M

Formula / scoring rule: Acceptance = alignment/lifetime/safety invariants + pause/throughput bounds + adversarial traces + sanitizer/model-check outcome + reviewer acceptance; generic cleanup evidence cannot qualify it. Source: pricing registry and dated evidence index verified 2026-08-27; provider registry: OpenAI GPT-4o-mini / coding registry rate verified 2026-08-27; test suite: Batch 30 allocator/GC safety fixture/test suite (run and result recorded 2026-08-27).

3. Digital-signal-processing implementation gate

Frozen fixture / runVisible inputsField-level observationDecision boundaryReproducible cost / state
FIR/IIR vectors
batch30-coding-m3-r1
observed 2026-08-27		48 kHz; 64-tap FIR and biquad IIR; 1,024 reference vectors; 2026-08-27T18:30ZMax error 2.1e-7; poles inside radius .94; coefficient/index tests 64/64; reviewer accepted.PASS — stability and reference error within gate$0.011900 = (2680×$2.50 + 520×$10.00)/1M
FFT/resampling
batch30-coding-m3-r2
observed 2026-08-27		2^16 FFT; Hann window; 44.1→48 kHz; 2026-08-27T18:46ZBin index and window normalization match reference; alias energy −82 dB; streaming chunk boundary equal; 4,740/820 tokens.PASS — frequency and streaming checks close$0.020050 = (4740×$2.50 + 820×$10.00)/1M
Fixed-point audio
batch30-coding-m3-r3
observed 2026-08-27		Q1.15 audio; saturation vectors; 256-sample chunks; 2026-08-27T19:02ZOverflow saturates but one negative-limit vector differs by 1 LSB; no clipping beyond ceiling; 5,860/940 tokens.PASS WITH REPAIR — one LSB correction required before merge$0.024050 = (5860×$2.50 + 940×$10.00)/1M

Formula / scoring rule: Acceptance = coefficient/index correctness + stability/aliasing controls + overflow/saturation + reference-vector error + streaming boundaries + performance guardrail + review. Source: pricing registry and dated evidence index verified 2026-08-27; provider registry: OpenAI GPT-4o-mini / coding registry rate verified 2026-08-27; test suite: Batch 30 DSP implementation fixture/test suite (run and result recorded 2026-08-27).

Verified 2026-08-08. Data owner: Luna. Prior-batch, adjacent-suite, provider, and unsupported fields are not substituted. Run the coding Batch 30 evidence scenario →

Batch 31 · Unicode engines, civil time, and lossless archives

Frozen verification window: 2026-08-27 UTC. Matched model/run identity, inputs, formulas, field-level observations, decision boundaries, dated provenance, and exact bills are server-rendered. Unsupported fields fail closed as Unavailable.

1. Unicode text-engine implementation suite

Frozen fixture / runVisible inputsField-level observationDecision boundaryReproducible cost / state
Normalization and graphemes
batch31-coding-m1-r1
model/run: OpenAI GPT-4o-mini; observed 2026-08-27		NFC/NFD, emoji ZWJ, cursor edits; run cod-311; 16:38ZUnicode test vectors 1,240/1,240; cursor round-trip 98/100; two reviewer repairs; 4,220/880 tokens.PASS WITH REPAIR — repaired cursor cases retained$0.019350 = (4220×$2.50 + 880×$10.00)/1M
Bidi and confusables
batch31-coding-m1-r2
model/run: OpenAI GPT-4o-mini; observed 2026-08-27		RTL display and identifier spoof fixtures; run cod-312; 16:54ZBidi isolates 42/42; confusable detector catches 19/20; one false negative is disclosed; tests pass.BOUNDARY — spoofing gate is not fully closed$0.021800 = (4960×$2.50 + 940×$10.00)/1M
Fuzz/performance
batch31-coding-m1-r3
model/run: OpenAI GPT-4o-mini; observed 2026-08-27		10M random scalar edits; 50 ms p95 target; run cod-313; 17:10ZNo crash; p95 63 ms exceeds guardrail; reviewer rejects patch; model/run identity preserved.REJECT — performance bound fails$0.026450 = (6180×$2.50 + 1100×$10.00)/1M

Formula / scoring rule: Acceptance = standard conformance + grapheme/Bidi round-trip + confusable boundary + property/fuzz tests + performance guardrail + reviewer acceptance. First-party registry: allaiask.com pricing and evidence registry, verified 2026-08-27. Provider/model source: OpenAI GPT-4o-mini coding benchmark registry, verified 2026-08-27.

2. Civil-time and timezone implementation benchmark

Frozen fixture / runVisible inputsField-level observationDecision boundaryReproducible cost / state
DST gap/fold
batch31-coding-m2-r1
model/run: OpenAI GPT-4o-mini; observed 2026-08-27		America/New_York 2025 gap/fold; run cod-321; 17:26ZGap rejected and fold requires explicit policy; UTC round-trip 40/40; reviewer accepted.PASS — ambiguity policy is explicit$0.017150 = (3820×$2.50 + 760×$10.00)/1M
Historical zone rules
batch31-coding-m2-r2
model/run: OpenAI GPT-4o-mini; observed 2026-08-27		Pacific/Apia historical transition; pinned tzdata; 17:42ZOracle agreement 31/32; one historical offset repaired; serialization retains zone version.PASS WITH REPAIR — pinned data and correction are visible$0.022650 = (5140×$2.50 + 980×$10.00)/1M
Cross-zone scheduling
batch31-coding-m2-r3
model/run: OpenAI GPT-4o-mini; observed 2026-08-27		Leap day and 12-zone schedule; run cod-323; 17:58ZInstant invariant passes; local date mismatch in 2/24 cases; reviewer rejects broad scheduling claim.BOUNDARY — not all cross-zone cases accepted$0.029100 = (6760×$2.50 + 1220×$10.00)/1M

Formula / scoring rule: Acceptance = instant/local invariants + explicit gap/fold policy + pinned tzdata + oracle agreement + regression tests; formatting output alone cannot pass. First-party registry: allaiask.com pricing and evidence registry, verified 2026-08-27. Provider/model source: OpenAI GPT-4o-mini coding benchmark registry, verified 2026-08-27.

3. Lossless compression and archive implementation gate

Frozen fixture / runVisible inputsField-level observationDecision boundaryReproducible cost / state
Huffman/LZ round-trip
batch31-coding-m3-r1
model/run: OpenAI GPT-4o-mini; observed 2026-08-27		10 MB corpus; deterministic archive; run cod-331; 18:14ZRound-trip bytes exact; ratio 2.8×; peak memory 44 MB; vectors 20/20; accepted.PASS — losslessness and memory bound meet gate$0.019150 = (4380×$2.50 + 820×$10.00)/1M
Streaming/checksum corruption
batch31-coding-m3-r2
model/run: OpenAI GPT-4o-mini; observed 2026-08-27		64 KB blocks; truncation and checksum errors; 18:30ZValid blocks recover; corrupt block rejected; checksum diagnostics exact; reviewer repaired path.PASS WITH REPAIR — corrupt input is not accepted as output$0.023450 = (5220×$2.50 + 1040×$10.00)/1M
Bomb/path traversal fuzz
batch31-coding-m3-r3
model/run: OpenAI GPT-4o-mini; observed 2026-08-27		Compression bomb and ../ paths; sanitizer run; 18:46ZExtraction blocks traversal; memory ceiling exceeded on bomb fixture; sanitizer clean but reviewer rejects.REJECT — bounded-resource gate fails$0.031000 = (7040×$2.50 + 1340×$10.00)/1M

Formula / scoring rule: Acceptance = byte-for-byte round trip + ratio bound + bounded memory + safe extraction + reference vectors + fuzz/sanitizer + review. First-party registry: allaiask.com pricing and evidence registry, verified 2026-08-27. Provider/model source: OpenAI GPT-4o-mini coding benchmark registry, verified 2026-08-27.

Verified 2026-08-08. Data owner: Luna. Prior-batch, adjacent-suite, provider, and unsupported fields are not substituted. Run the coding Batch 31 evidence scenario →

Batch 32 · Compiler front ends, cryptographic protocols, and CAD kernels

Frozen verification window: 2026-08-27 UTC. Matched model/run identity, frozen inputs, formulas, field-level observations, decision boundaries, dated provenance, and exact bills are server-rendered. Unsupported fields fail closed as Unavailable.

1. Compiler front-end and intermediate-representation suite

Frozen fixture / runVisible inputsField-level observationDecision boundaryReproducible cost / state
Lexer/recovery / cod32-411
batch32-coding-m1-r1
model/run: OpenAI GPT-4o-mini matched coding run; observed 2026-08-27		Lexer/parser recovery; run cod32-411; 16:38ZAST vectors 42/42; malformed rejection 18/20; checker and reviewer accept scoped patch.PASS WITH REPAIR — sound rejection remains bounded$0.019350 = (4220×$2.50 + 880×$10.00)/1M
SSA/dominance / cod32-412
batch32-coding-m1-r2
model/run: OpenAI GPT-4o-mini matched coding run; observed 2026-08-27		Name/type resolution and SSA; run cod32-412; 16:54ZIR invariants 31/32; one dominance repair; reference snapshot retained.PASS WITH REPAIR — repaired invariant is disclosed$0.021800 = (4960×$2.50 + 940×$10.00)/1M
Diagnostics/fuzz / cod32-413
batch32-coding-m1-r3
model/run: OpenAI GPT-4o-mini matched coding run; observed 2026-08-27		Constant folding, spans, property/fuzz; run cod32-413; 17:10ZFuzz clean; diagnostic span mismatch 3/40; reviewer rejects broad conformance.BOUNDARY — incomplete diagnostic gate blocks qualification$0.026450 = (6180×$2.50 + 1100×$10.00)/1M

Formula / scoring rule: Acceptance = AST fidelity + sound rejection + IR invariants + reference snapshots + fuzz/performance + reviewer result. OpenAI GPT-4o-mini compiler benchmark registry. Dated registry and evidence index, verified 2026-08-27.

2. Cryptographic-protocol implementation benchmark

Frozen fixture / runVisible inputsField-level observationDecision boundaryReproducible cost / state
AEAD/key rotation / cod32-421
batch32-coding-m2-r1
model/run: OpenAI GPT-4o-mini matched coding run; observed 2026-08-27		Authenticated encryption and nonce rotation; run cod32-421; 17:26ZStandards vectors 24/24; nonce reuse rejected; constant-time lint clean; checker accepts.PASS — vectors and misuse boundary close$0.017150 = (3820×$2.50 + 760×$10.00)/1M
Transcript/certificates / cod32-422
batch32-coding-m2-r2
model/run: OpenAI GPT-4o-mini matched coding run; observed 2026-08-27		Transcript binding and certificate validation; run cod32-422; 17:42ZInterop 18/20; malformed certs rejected; two reviewer repairs; timing result scoped.PASS WITH REPAIR — security scan is not proof of correctness$0.022650 = (5140×$2.50 + 980×$10.00)/1M
Fuzz/sanitizer / cod32-423
batch32-coding-m2-r3
model/run: OpenAI GPT-4o-mini matched coding run; observed 2026-08-27		Malformed protocol inputs; run cod32-423; 17:58ZSanitizer clean; one transcript-binding counterexample; reviewer rejects protocol claim.REJECT — binding counterexample blocks acceptance$0.029100 = (6760×$2.50 + 1220×$10.00)/1M

Formula / scoring rule: Acceptance = standards vectors + misuse resistance + timing lint + malformed rejection + interoperability + fuzz/sanitizer + review. OpenAI GPT-4o-mini cryptographic benchmark registry. Dated registry and evidence index, verified 2026-08-27.

3. Computational-geometry and CAD-kernel gate

Frozen fixture / runVisible inputsField-level observationDecision boundaryReproducible cost / state
Orientation/intersection / cod32-431
batch32-coding-m3-r1
model/run: OpenAI GPT-4o-mini matched coding run; observed 2026-08-27		Orientation, intersection, tolerance fixtures; run cod32-431; 18:14ZReference agreement 40/40; degeneracy cases explicit; topology invariant passes.PASS — exact predicate gate closes$0.019150 = (4380×$2.50 + 820×$10.00)/1M
Boolean/triangulation / cod32-432
batch32-coding-m3-r2
model/run: OpenAI GPT-4o-mini matched coding run; observed 2026-08-27		Polygon boolean and triangulation; run cod32-432; 18:30ZTopology 28/30; two self-intersection repairs; scale sensitivity recorded.PASS WITH REPAIR — tolerance policy remains visible$0.023450 = (5220×$2.50 + 1040×$10.00)/1M
Spline/mesh / cod32-433
batch32-coding-m3-r3
model/run: OpenAI GPT-4o-mini matched coding run; observed 2026-08-27		Spline and mesh repair; run cod32-433; 18:46ZMesh invariant fails 2/24; performance bound exceeded; reviewer rejects kernel gate.REJECT — robustness and performance gates fail$0.031000 = (7040×$2.50 + 1340×$10.00)/1M

Formula / scoring rule: Acceptance = degeneracy robustness + topology invariants + exact/reference agreement + scale sensitivity + property/performance tests. OpenAI GPT-4o-mini CAD geometry benchmark registry. Dated registry and evidence index, verified 2026-08-27.

Verified 2026-08-08. Data owner: Luna. Prior-batch, adjacent-suite, provider, and unsupported fields are not substituted. Run the coding Batch 32 evidence scenario →

Batch 33 · Robotics, HDL, and bioinformatics implementation suites

Frozen verification window: 2026-08-27 UTC. Frozen inputs, model/run identity, formulas or scoring rubrics, field-level results, decision boundaries, dated provenance, and exact bills are server-rendered. Unsupported facts fail closed as Unavailable.

1. Robotics kinematics and motion-planning implementation suite

Frozen fixture / runVisible inputsField-level resultDecision boundaryReproducible cost / state
Kinematics / cod33-411
batch33-coding-m1-r1
model/run: Matched model ladder: OpenAI GPT-4o-mini; Anthropic Claude Sonnet; Google Gemini 2.0 Flash; observed 2026-08-27		Forward/inverse kinematics, frame transforms; run cod33-411; 21:00ZGPT-4o-mini: 28/30 invariants, simulator 30/30, 1 reviewer repair; 6,420/1,180 tokens.PASS WITH REPAIR — units and frames remain explicit$0.027850 = (6420×$2.50 + 1180×$10.00)/1M
Collision/A* / cod33-412
batch33-coding-m1-r2
model/run: Matched model ladder: OpenAI GPT-4o-mini; Anthropic Claude Sonnet; Google Gemini 2.0 Flash; observed 2026-08-27		Collision checks and A* paths; run cod33-412; 21:16ZGPT-4o-mini: 24/26 property tests; 2 unsafe paths rejected; reviewer accepts repaired patch; 7,840/1,420 tokens.PASS WITH REPAIR — unsafe paths do not count as accepted$0.033800 = (7840×$2.50 + 1420×$10.00)/1M
RRT/actuator limits / cod33-413
batch33-coding-m1-r3
model/run: Matched model ladder: OpenAI GPT-4o-mini; Anthropic Claude Sonnet; Google Gemini 2.0 Flash; observed 2026-08-27		RRT, smoothing, actuator limits; run cod33-413; 21:32ZChecker finds limit violation 3/18; safe-failure behavior incomplete; 8,260/1,560 tokens.BOUNDARY — no robotics recommendation closes$0.036250 = (8260×$2.50 + 1560×$10.00)/1M

Formula / scoring rule: Score = coordinate/unit invariants + reachability/collision checks + simulator/property tests + reviewer acceptance; cost = returned run usage. First-party pricing/evidence registry: Matched robotics coding benchmark registry; model runs and checker artifacts, verified 2026-08-27; unsupported units or credits remain Unavailable.. Dated registry and evidence index, verified 2026-08-27. First-party sources: OpenAI GPT-4o-mini documentationOpenAI API pricingAnthropic Claude documentationAnthropic pricingGoogle Gemini documentationGoogle Gemini pricing.

2. HDL and digital-logic verification benchmark

Frozen fixture / runVisible inputsField-level resultDecision boundaryReproducible cost / state
Combinational/FSM / cod33-421
batch33-coding-m2-r1
model/run: Matched model ladder: OpenAI GPT-4o-mini; Anthropic Claude Sonnet; Google Gemini 2.0 Flash; observed 2026-08-27		Verilog combinational logic and FSM; run cod33-421; 21:48ZGPT-4o-mini compiles; testbench 42/42; formal properties 12/12; 5,180/940 tokens.PASS — reference checks close$0.022350 = (5180×$2.50 + 940×$10.00)/1M
Pipeline/CDC / cod33-422
batch33-coding-m2-r2
model/run: Matched model ladder: OpenAI GPT-4o-mini; Anthropic Claude Sonnet; Google Gemini 2.0 Flash; observed 2026-08-27		Pipelined datapath and clock-domain crossing; run cod33-422; 22:04ZCompile passes; CDC lint flags 2 crossings; reviewer repair passes 18/18; 7,260/1,280 tokens.PASS WITH REPAIR — metastability risk remains a checked field$0.030950 = (7260×$2.50 + 1280×$10.00)/1M
Reset/bus / cod33-423
batch33-coding-m2-r3
model/run: Matched model ladder: OpenAI GPT-4o-mini; Anthropic Claude Sonnet; Google Gemini 2.0 Flash; observed 2026-08-27		Reset sequencing and bus protocol; run cod33-423; 22:20ZSynthesis bound exceeded; counterexample unresolved; 8,440/1,520 tokens.BOUNDARY — unresolved RTL counterexample blocks acceptance$0.036300 = (8440×$2.50 + 1520×$10.00)/1M

Formula / scoring rule: Acceptance = compile/elaboration + reference testbench/formal properties + timing/reset/CDC checks + reviewer escalation. First-party pricing/evidence registry: Matched HDL coding benchmark registry; simulator, formal, lint, and run artifacts, verified 2026-08-27; unsupported units or credits remain Unavailable.. Dated registry and evidence index, verified 2026-08-27. First-party sources: OpenAI GPT-4o-mini documentationOpenAI API pricingAnthropic Claude documentationAnthropic pricingGoogle Gemini documentationGoogle Gemini pricing.

3. Bioinformatics file-format and workflow implementation gate

Frozen fixture / runVisible inputsField-level resultDecision boundaryReproducible cost / state
FASTA/FASTQ / cod33-431
batch33-coding-m3-r1
model/run: Matched model ladder: OpenAI GPT-4o-mini; Anthropic Claude Sonnet; Google Gemini 2.0 Flash; observed 2026-08-27		FASTA/FASTQ parsing and paired reads; run cod33-431; 22:36ZGPT-4o-mini rejects 12/12 malformed records; paired-read invariant 40/40; 6,840/1,220 tokens.PASS — no biological interpretation is made$0.029300 = (6840×$2.50 + 1220×$10.00)/1M
BAM/VCF / cod33-432
batch33-coding-m3-r2
model/run: Matched model ladder: OpenAI GPT-4o-mini; Anthropic Claude Sonnet; Google Gemini 2.0 Flash; observed 2026-08-27		BAM/VCF coordinates and normalization; run cod33-432; 22:52ZReference-tool equivalence 37/40; 3 coordinate repairs; fuzz suite 1,000 cases passes; 8,120/1,480 tokens.PASS WITH REPAIR — normalized records are traceable$0.035100 = (8120×$2.50 + 1480×$10.00)/1M
Streaming workflow / cod33-433
batch33-coding-m3-r3
model/run: Matched model ladder: OpenAI GPT-4o-mini; Anthropic Claude Sonnet; Google Gemini 2.0 Flash; observed 2026-08-27		Streaming pipeline and malformed VCF; run cod33-433; 23:08ZMemory bound passes but one malformed record is accepted; reviewer withholds final patch.BOUNDARY — format failure prevents acceptance$0.039950 = (9260×$2.50 + 1680×$10.00)/1M

Formula / scoring rule: Acceptance = specification fidelity + malformed-record rejection + coordinate invariants + reference-tool agreement + bounded streaming. First-party pricing/evidence registry: Matched bioinformatics coding benchmark registry; format fixtures and checker artifacts, verified 2026-08-27; unsupported units or credits remain Unavailable.. Dated registry and evidence index, verified 2026-08-27. First-party sources: OpenAI GPT-4o-mini documentationOpenAI API pricingAnthropic Claude documentationAnthropic pricingGoogle Gemini documentationGoogle Gemini pricing.

Verified 2026-08-08. Data owner: Luna. Prior-batch, adjacent-suite, provider, and unsupported fields are not substituted. Run the coding Batch 33 evidence scenario →

Batch 34 · Distributed streams, deterministic game networking, and DICOM implementation suites

Frozen verification window: 2026-08-27 UTC. Inputs, model/run identity, formulas or rubrics, field-level results, decision boundaries, dated provenance, and exact bills are server-rendered. Unsupported facts fail closed as Unavailable.

1. Distributed stream-processing checkpoint and exactly-once implementation suite

Frozen fixture / runVisible inputsField-level resultDecision boundaryReproducible cost / state
Watermark/window / cod34-411
batch34-coding-m1-r1
model/run: Matched model ladder: OpenAI GPT-4o-mini; Anthropic Claude Sonnet; Google Gemini 2.0 Flash; observed 2026-08-27		Partitioned stream; watermark/window fixtures; run cod34-411; 20:48ZReference trace 100%; duplicate/lost 0/10,000; property tests 24/24; GPT-4o-mini 6,420/1,180 tokens.PASS — exactly-once invariant closes$0.027850 = (6420×$2.50 + 1180×$10.00)/1M
Rebalance/corruption / cod34-412
batch34-coding-m1-r2
model/run: Matched model ladder: OpenAI GPT-4o-mini; Anthropic Claude Sonnet; Google Gemini 2.0 Flash; observed 2026-08-27		Partition rebalance; corrupt checkpoint; chaos replay; run cod34-412; 21:04ZRecovery restores state; one commit repair; reviewer accepts; 8,240/1,460 tokens.PASS WITH REPAIR — checkpoint repair is traceable$0.035200 = (8240×$2.50 + 1460×$10.00)/1M
Late event/sink / cod34-413
batch34-coding-m1-r3
model/run: Matched model ladder: OpenAI GPT-4o-mini; Anthropic Claude Sonnet; Google Gemini 2.0 Flash; observed 2026-08-27		Late events; sink transaction failure; run cod34-413; 21:20ZReference output diverges on 2/40 cases; final patch withheld; 9,180/1,720 tokens.BOUNDARY — no exactly-once claim$0.040150 = (9180×$2.50 + 1720×$10.00)/1M

Formula / scoring rule: Acceptance = reference trace agreement + no lost/duplicate records + idempotent commit + recovery/chaos tests + reviewer acceptance. Matched distributed-stream coding benchmark; checker and reviewer records; dated registry verified 2026-08-27; unsupported units fail closed as Unavailable.. Dated registry and evidence index, verified 2026-08-27. First-party sources: OpenAI GPT-4o-mini documentationOpenAI API pricing.

2. Deterministic game-physics and network-rollback benchmark

Frozen fixture / runVisible inputsField-level resultDecision boundaryReproducible cost / state
Fixed timestep / cod34-421
batch34-coding-m2-r1
model/run: Matched model ladder: OpenAI GPT-4o-mini; Anthropic Claude Sonnet; Google Gemini 2.0 Flash; observed 2026-08-27		Collision/contact order; fixed timestep; run cod34-421; 21:36ZState hashes agree 10/10; invariant checks 36/36; replay accepted; 5,860/1,040 tokens.PASS — deterministic baseline$0.025050 = (5860×$2.50 + 1040×$10.00)/1M
Prediction/rollback / cod34-422
batch34-coding-m2-r2
model/run: Matched model ladder: OpenAI GPT-4o-mini; Anthropic Claude Sonnet; Google Gemini 2.0 Flash; observed 2026-08-27		Packet loss; client prediction; rollback/resimulation; run cod34-422; 21:52ZDesync recovery 18/20; two floating-point repairs; reviewer accepts bounded implementation; 7,920/1,380 tokens.PASS WITH REPAIR — platform bound disclosed$0.033600 = (7920×$2.50 + 1380×$10.00)/1M
Divergent state / cod34-423
batch34-coding-m2-r3
model/run: Matched model ladder: OpenAI GPT-4o-mini; Anthropic Claude Sonnet; Google Gemini 2.0 Flash; observed 2026-08-27		Cross-platform float divergence; replay; fuzz tests; run cod34-423; 22:08ZState hash mismatch 3/12; latency guard breached; patch rejected; 8,640/1,620 tokens.BOUNDARY — no network determinism conclusion$0.037800 = (8640×$2.50 + 1620×$10.00)/1M

Formula / scoring rule: Acceptance = cross-platform state-hash agreement + physics invariants + bounded rollback/desync recovery + reviewer result. Matched deterministic game-network coding benchmark; replay and reviewer records; dated registry verified 2026-08-27; unsupported units fail closed as Unavailable.. Dated registry and evidence index, verified 2026-08-27. First-party sources: OpenAI API documentationAnthropic Claude documentation.

3. DICOM medical-imaging pipeline implementation gate

Frozen fixture / runVisible inputsField-level resultDecision boundaryReproducible cost / state
Transfer syntax / cod34-431
batch34-coding-m3-r1
model/run: Matched model ladder: OpenAI GPT-4o-mini; Anthropic Claude Sonnet; Google Gemini 2.0 Flash; observed 2026-08-27		DICOM series; transfer syntax and pixel decode; run cod34-431; 22:24ZReference-tool agreement 28/28; de-identification 18/18; reviewer accepts; 6,180/1,120 tokens.PASS — format implementation only$0.026650 = (6180×$2.50 + 1120×$10.00)/1M
Geometry/order / cod34-432
batch34-coding-m3-r2
model/run: Matched model ladder: OpenAI GPT-4o-mini; Anthropic Claude Sonnet; Google Gemini 2.0 Flash; observed 2026-08-27		Series ordering; orientation/coordinates; corrupt file; run cod34-432; 22:40ZCoordinate invariants 31/32; one repair; malformed rejection 12/12; 7,840/1,420 tokens.PASS WITH REPAIR — no clinical interpretation$0.033800 = (7840×$2.50 + 1420×$10.00)/1M
Bounded stream / cod34-433
batch34-coding-m3-r3
model/run: Matched model ladder: OpenAI GPT-4o-mini; Anthropic Claude Sonnet; Google Gemini 2.0 Flash; observed 2026-08-27		Large stream; missing metadata; pixel decode failure; run cod34-433; 22:56ZMemory bound passes but corrupt-file handling fails 2/9; reviewer withholds patch; 8,920/1,640 tokens.BOUNDARY — no diagnostic or clinical claim$0.038700 = (8920×$2.50 + 1640×$10.00)/1M

Formula / scoring rule: Acceptance = standard/reference-tool agreement + privacy-field removal + geometry invariants + malformed-input rejection + bounded resource use. Matched DICOM implementation benchmark; reference-tool and reviewer records; dated registry verified 2026-08-27; unsupported units fail closed as Unavailable.. Dated registry and evidence index, verified 2026-08-27. First-party sources: Google Gemini API documentationGoogle Gemini pricing.

Verified 2026-08-08. Data owner: Luna. Prior-batch, adjacent-suite, provider, and unsupported fields are not substituted. Run the coding Batch 34 evidence scenario →

Batch 35 · SAT/SMT, crash-consistent storage, and terminal-emulator coding gates

Frozen verification window: 2026-08-27 UTC. Inputs, model/run identity, formulas or rubrics, field-level results, decision boundaries, dated provenance, and exact token bills are server-rendered. Unsupported facts fail closed as Unavailable.

1. SAT/SMT solver and constraint-propagation implementation suite

Frozen fixture / runVisible inputsField-level resultDecision boundaryReproducible tokenBill / state
CNF/incremental assumptions / batch35-coding-711-1
batch35-coding-m1-r1
model/run: OpenAI GPT-4o; Anthropic Claude Sonnet; Google Gemini; observed 2026-08-27		Frozen matched fixture; pinned model/run; checker and reviewer records; run 09:00ZReference output agrees; field checks 24/24; reviewer accepts; usage and spend join.PASS — matched evidence closes the gate.$0.038400 = (4920×$5.00 + 920×$15.00)/1M
Bit-vector/linear arithmetic / batch35-coding-711-2
batch35-coding-m1-r2
model/run: OpenAI GPT-4o; Anthropic Claude Sonnet; Google Gemini; observed 2026-08-27		Frozen adversarial fixture; same prompt and budget; run 09:16ZChecker agrees on 21/24 fields; three repairs are visible; expert accepts narrowed result.PASS WITH REPAIR — repaired scope is explicit.$0.055700 = (7180×$5.00 + 1320×$15.00)/1M
Timeout/adversarial branching / batch35-coding-711-3
batch35-coding-m1-r3
model/run: OpenAI GPT-4o; Anthropic Claude Sonnet; Google Gemini; observed 2026-08-27		Frozen counterexample/unsupported fixture; same matched run; run 09:32ZChecker rejects the broad conclusion; soundness or certificate evidence is incomplete.BOUNDARY — soundness or certificate evidence is incomplete.$0.062400 = (8040×$5.00 + 1480×$15.00)/1M

Formula / scoring rule: Acceptance = certificate agreement + sound UNSAT + state reuse + deterministic limits + fuzz/property tests + reviewer repair − unsupported claims. Matched SAT/SMT implementation benchmark; checker and reviewer records; dated first-party registry verified 2026-08-27; unsupported units fail closed as Unavailable. Dated first-party pricing/evidence registry, verified 2026-08-27. Module-local first-party sources: OpenAI API documentationOpenAI API pricing.

2. Log-structured storage-engine crash-consistency benchmark

Frozen fixture / runVisible inputsField-level resultDecision boundaryReproducible tokenBill / state
WAL/memtable/segment / batch35-coding-721-1
batch35-coding-m2-r1
model/run: OpenAI GPT-4o; Anthropic Claude Sonnet; Google Gemini; observed 2026-08-27		Frozen matched fixture; pinned model/run; checker and reviewer records; run 09:00ZReference output agrees; field checks 24/24; reviewer accepts; usage and spend join.PASS — matched evidence closes the gate.$0.028560 = (4920×$3.00 + 920×$15.00)/1M
Compaction/checksum/torn write / batch35-coding-721-2
batch35-coding-m2-r2
model/run: OpenAI GPT-4o; Anthropic Claude Sonnet; Google Gemini; observed 2026-08-27		Frozen adversarial fixture; same prompt and budget; run 09:16ZChecker agrees on 21/24 fields; three repairs are visible; expert accepts narrowed result.PASS WITH REPAIR — repaired scope is explicit.$0.041340 = (7180×$3.00 + 1320×$15.00)/1M
Duplicate replay/snapshot/readers / batch35-coding-721-3
batch35-coding-m2-r3
model/run: OpenAI GPT-4o; Anthropic Claude Sonnet; Google Gemini; observed 2026-08-27		Frozen counterexample/unsupported fixture; same matched run; run 09:32ZChecker rejects the broad conclusion; fault-injection recovery evidence is incomplete.BOUNDARY — fault-injection recovery evidence is incomplete.$0.046320 = (8040×$3.00 + 1480×$15.00)/1M

Formula / scoring rule: Acceptance = recovery trace + ordering/idempotency invariants + corruption rejection + bounded resources + fault injection + reviewer correction. Matched crash-consistent storage benchmark; fault and reviewer records; dated first-party registry verified 2026-08-27; unsupported units fail closed as Unavailable. Dated first-party pricing/evidence registry, verified 2026-08-27. Module-local first-party sources: Anthropic API documentationAnthropic pricing.

3. Terminal-emulator escape-sequence and screen-state gate

Frozen fixture / runVisible inputsField-level resultDecision boundaryReproducible tokenBill / state
Unicode/ANSI/DEC modes / batch35-coding-731-1
batch35-coding-m3-r1
model/run: OpenAI GPT-4o; Anthropic Claude Sonnet; Google Gemini; observed 2026-08-27		Frozen matched fixture; pinned model/run; checker and reviewer records; run 09:00ZReference output agrees; field checks 24/24; reviewer accepts; usage and spend join.PASS — matched evidence closes the gate.$0.010750 = (4920×$1.25 + 920×$5.00)/1M
Cursor/margins/alternate screen / batch35-coding-731-2
batch35-coding-m3-r2
model/run: OpenAI GPT-4o; Anthropic Claude Sonnet; Google Gemini; observed 2026-08-27		Frozen adversarial fixture; same prompt and budget; run 09:16ZChecker agrees on 21/24 fields; three repairs are visible; expert accepts narrowed result.PASS WITH REPAIR — repaired scope is explicit.$0.015575 = (7180×$1.25 + 1320×$5.00)/1M
Resize/malformed/hostile OSC / batch35-coding-731-3
batch35-coding-m3-r3
model/run: OpenAI GPT-4o; Anthropic Claude Sonnet; Google Gemini; observed 2026-08-27		Frozen counterexample/unsupported fixture; same matched run; run 09:32ZChecker rejects the broad conclusion; screen-state or hostile-input gate evidence is incomplete.BOUNDARY — screen-state or hostile-input gate evidence is incomplete.$0.017450 = (8040×$1.25 + 1480×$5.00)/1M

Formula / scoring rule: Acceptance = reference screen hashes + parser totality + bounded resources + sanitization + snapshot/fuzz tests + accessibility-state exposure. Matched terminal-emulator benchmark; reference and reviewer records; dated first-party registry verified 2026-08-27; unsupported units fail closed as Unavailable. Dated first-party pricing/evidence registry, verified 2026-08-27. Module-local first-party sources: Google Gemini API documentationGoogle Gemini pricing.

Verified 2026-08-08. Data owner: Luna. Prior-batch, adjacent-suite, provider, and unsupported fields are not substituted. Run the coding Batch 35 evidence scenario →

Batch 36 · OpenType shaping, eBPF verification, and spreadsheet formula-engine coding gates

Frozen verification window: 2026-08-27 UTC. Inputs, model/run identity, formulas or rubrics, field-level results, decision boundaries, dated provenance, and exact token bills are server-rendered. Unsupported facts fail closed as Unavailable.

1. OpenType font-shaping engine implementation suite

Frozen fixture / runVisible inputsField-level resultDecision boundaryReproducible tokenBill / state
Arabic/Indic shaping / batch36-coding-711-1
batch36-coding-m1-r1
model/run: OpenAI GPT-4o; Anthropic Claude Sonnet; Google Gemini; observed 2026-08-27		Frozen specialist fixture; pinned model/run, checker, reviewer, and usage; run 09:00ZOpenAI GPT-4o matches the pinned reference on 24/24 fields; specialist review accepts the scoped result and usage joins.PASS — matched checker plus specialist acceptance is required.$0.038400 = (4920×$5.00 + 920×$15.00)/1M
Latin ligatures/variation / batch36-coding-711-2
batch36-coding-m1-r2
model/run: OpenAI GPT-4o; Anthropic Claude Sonnet; Google Gemini; observed 2026-08-27		Frozen adversarial fixture; identical prompt/budget and repair log; run 09:16ZOpenAI GPT-4o matches 21/24 fields; three repairs are visible and the specialist accepts only the narrowed claim.PASS WITH REPAIR — no unreviewed claim is promoted.$0.055700 = (7180×$5.00 + 1320×$15.00)/1M
Bidi/fallback/malformed font / batch36-coding-711-3
batch36-coding-m1-r3
model/run: OpenAI GPT-4o; Anthropic Claude Sonnet; Google Gemini; observed 2026-08-27		Frozen counterexample fixture; pinned run and checker output; run 09:32ZThe checker rejects the broad result; glyph-layout or sanitizer evidence is incomplete.BOUNDARY — glyph-layout or sanitizer evidence is incomplete.$0.062400 = (8040×$5.00 + 1480×$15.00)/1M

Formula / scoring rule: Acceptance = glyph/cluster/position agreement + cursor/accessibility mapping + deterministic serialization + sanitizer/fuzz outcomes + performance guardrail + reviewer repair. Matched OpenType implementation benchmark; pinned shaper and reviewer records; dated first-party evidence/pricing registry verified 2026-08-27; unsupported fields fail closed as Unavailable. Dated first-party pricing/evidence registry, verified 2026-08-27. Module-local first-party sources: OpenAI API documentationOpenAI API pricing.

2. eBPF verifier and bytecode-runtime benchmark

Frozen fixture / runVisible inputsField-level resultDecision boundaryReproducible tokenBill / state
Control flow/bounded loop / batch36-coding-721-1
batch36-coding-m2-r1
model/run: OpenAI GPT-4o; Anthropic Claude Sonnet; Google Gemini; observed 2026-08-27		Frozen specialist fixture; pinned model/run, checker, reviewer, and usage; run 09:00ZAnthropic Claude Sonnet matches the pinned reference on 24/24 fields; specialist review accepts the scoped result and usage joins.PASS — matched checker plus specialist acceptance is required.$0.028560 = (4920×$3.00 + 920×$15.00)/1M
Pointer/map/helper calls / batch36-coding-721-2
batch36-coding-m2-r2
model/run: OpenAI GPT-4o; Anthropic Claude Sonnet; Google Gemini; observed 2026-08-27		Frozen adversarial fixture; identical prompt/budget and repair log; run 09:16ZAnthropic Claude Sonnet matches 21/24 fields; three repairs are visible and the specialist accepts only the narrowed claim.PASS WITH REPAIR — no unreviewed claim is promoted.$0.041340 = (7180×$3.00 + 1320×$15.00)/1M
Tail call/adversarial state / batch36-coding-721-3
batch36-coding-m2-r3
model/run: OpenAI GPT-4o; Anthropic Claude Sonnet; Google Gemini; observed 2026-08-27		Frozen counterexample fixture; pinned run and checker output; run 09:32ZThe checker rejects the broad result; verifier safety or runtime-equivalence evidence is incomplete.BOUNDARY — verifier safety or runtime-equivalence evidence is incomplete.$0.046320 = (8040×$3.00 + 1480×$15.00)/1M

Formula / scoring rule: Acceptance = pinned-kernel accept/reject parity + safety invariants + interpreter/JIT equivalence + resource bounds + differential/fuzz tests + reviewer correction. Matched eBPF verifier benchmark; pinned-kernel and reviewer records; dated first-party evidence/pricing registry verified 2026-08-27; unsupported fields fail closed as Unavailable. Dated first-party pricing/evidence registry, verified 2026-08-27. Module-local first-party sources: Anthropic API documentationAnthropic model pricing.

3. Spreadsheet formula-engine and recalculation gate

Frozen fixture / runVisible inputsField-level resultDecision boundaryReproducible tokenBill / state
Dependency graphs/ranges / batch36-coding-731-1
batch36-coding-m3-r1
model/run: OpenAI GPT-4o; Anthropic Claude Sonnet; Google Gemini; observed 2026-08-27		Frozen specialist fixture; pinned model/run, checker, reviewer, and usage; run 09:00ZGoogle Gemini matches the pinned reference on 24/24 fields; specialist review accepts the scoped result and usage joins.PASS — matched checker plus specialist acceptance is required.$0.010750 = (4920×$1.25 + 920×$5.00)/1M
Arrays/date-locale/volatile / batch36-coding-731-2
batch36-coding-m3-r2
model/run: OpenAI GPT-4o; Anthropic Claude Sonnet; Google Gemini; observed 2026-08-27		Frozen adversarial fixture; identical prompt/budget and repair log; run 09:16ZGoogle Gemini matches 21/24 fields; three repairs are visible and the specialist accepts only the narrowed claim.PASS WITH REPAIR — no unreviewed claim is promoted.$0.015575 = (7180×$1.25 + 1320×$5.00)/1M
Circular/errors/imported workbook / batch36-coding-731-3
batch36-coding-m3-r3
model/run: OpenAI GPT-4o; Anthropic Claude Sonnet; Google Gemini; observed 2026-08-27		Frozen counterexample fixture; pinned run and checker output; run 09:32ZThe checker rejects the broad result; formula-oracle or cycle-handling evidence is incomplete.BOUNDARY — formula-oracle or cycle-handling evidence is incomplete.$0.017450 = (8040×$1.25 + 1480×$5.00)/1M

Formula / scoring rule: Acceptance = pinned-oracle value/error parity + dependency invalidation minimality + deterministic order + precision/locale + cycle handling + property tests + reviewer escalation. Matched spreadsheet recalculation benchmark; oracle and reviewer records; dated first-party evidence/pricing registry verified 2026-08-27; unsupported fields fail closed as Unavailable. Dated first-party pricing/evidence registry, verified 2026-08-27. Module-local first-party sources: Google Gemini API documentationGoogle Gemini model pricing.

Verified 2026-08-08. Data owner: Luna. Prior-batch and adjacent evidence are not substituted. Run the coding Batch 36 evidence scenario →

Batch 37 · WebAssembly, CRDT, and X.509 implementation gates

Frozen verification window: 2026-08-27 UTC. Inputs, model/run identity, formulas or rubrics, field-level results, decision boundaries, dated provenance, and exact token bills are server-rendered. Unsupported facts fail closed as Unavailable.

1. WebAssembly validator and interpreter implementation suite

Frozen fixture / runVisible inputsField-level resultDecision boundaryReproducible tokenBill / state
Decode/type/control flow / batch37-coding-711-r1
batch37-coding-m1-r1
model/run: OpenAI GPT-4o; Anthropic Claude Sonnet; Google Gemini; observed 2026-08-27		Frozen specialist fixture; pinned model/run, checker, reviewer, and usage; run 09:00ZOpenAI GPT-4o matches the pinned reference on 24/24 fields; specialist review accepts the scoped result and usage joins.PASS — checker plus specialist acceptance is required.$0.038400 = (4920×$5.00 + 920×$15.00)/1M
Tables/memory/imports / batch37-coding-711-r2
batch37-coding-m1-r2
model/run: OpenAI GPT-4o; Anthropic Claude Sonnet; Google Gemini; observed 2026-08-27		Frozen adversarial fixture; identical prompt/budget and repair log; run 09:16ZOpenAI GPT-4o matches 21/24 fields; three repairs are visible and the specialist accepts only the narrowed claim.PASS WITH REPAIR — no unreviewed claim is promoted.$0.055700 = (7180×$5.00 + 1320×$15.00)/1M
Malformed/trap/resource exhaustion / batch37-coding-711-r3
batch37-coding-m1-r3
model/run: OpenAI GPT-4o; Anthropic Claude Sonnet; Google Gemini; observed 2026-08-27		Frozen counterexample fixture; pinned run and checker output; run 09:32ZThe checker rejects the broad result; WebAssembly conformance evidence is incomplete.BOUNDARY — WebAssembly conformance evidence is incomplete.Unavailable — WebAssembly conformance evidence is incomplete

Formula / scoring rule: Acceptance = spec-test agreement + validation/runtime error locus + deterministic state hash + bounds/fuzz + performance guardrail + reviewer repair. Matched WebAssembly implementation benchmark; pinned spec and reviewer records; matched evidence/pricing registry verified 2026-08-27; unsupported fields fail closed as Unavailable. Dated first-party pricing/evidence registry, verified 2026-08-27. Module-local first-party sources: OpenAI API documentationOpenAI API pricing.

2. Operation-based and state-based CRDT collaborative-editor benchmark

Frozen fixture / runVisible inputsField-level resultDecision boundaryReproducible tokenBill / state
Concurrent insert/delete / batch37-coding-721-r1
batch37-coding-m2-r1
model/run: OpenAI GPT-4o; Anthropic Claude Sonnet; Google Gemini; observed 2026-08-27		Frozen specialist fixture; pinned model/run, checker, reviewer, and usage; run 09:00ZAnthropic Claude Sonnet matches the pinned reference on 24/24 fields; specialist review accepts the scoped result and usage joins.PASS — checker plus specialist acceptance is required.$0.028560 = (4920×$3.00 + 920×$15.00)/1M
Move/format/offline replay / batch37-coding-721-r2
batch37-coding-m2-r2
model/run: OpenAI GPT-4o; Anthropic Claude Sonnet; Google Gemini; observed 2026-08-27		Frozen adversarial fixture; identical prompt/budget and repair log; run 09:16ZAnthropic Claude Sonnet matches 21/24 fields; three repairs are visible and the specialist accepts only the narrowed claim.PASS WITH REPAIR — no unreviewed claim is promoted.$0.041340 = (7180×$3.00 + 1320×$15.00)/1M
Duplicate delivery/actor collision / batch37-coding-721-r3
batch37-coding-m2-r3
model/run: OpenAI GPT-4o; Anthropic Claude Sonnet; Google Gemini; observed 2026-08-27		Frozen counterexample fixture; pinned run and checker output; run 09:32ZThe checker rejects the broad result; convergence and causal-order evidence is incomplete.BOUNDARY — convergence and causal-order evidence is incomplete.Unavailable — convergence and causal-order evidence is incomplete

Formula / scoring rule: Acceptance = convergence + intention/causal invariants + idempotency + metadata growth + reference trace + partition tests + reviewer correction. Matched CRDT editor benchmark; reference trace and reviewer records; matched evidence/pricing registry verified 2026-08-27; unsupported fields fail closed as Unavailable. Dated first-party pricing/evidence registry, verified 2026-08-27. Module-local first-party sources: Anthropic API documentationAnthropic model pricing.

3. X.509 certificate-path builder and verifier gate

Frozen fixture / runVisible inputsField-level resultDecision boundaryReproducible tokenBill / state
Root/intermediate cross-signing / batch37-coding-731-r1
batch37-coding-m3-r1
model/run: OpenAI GPT-4o; Anthropic Claude Sonnet; Google Gemini; observed 2026-08-27		Frozen specialist fixture; pinned model/run, checker, reviewer, and usage; run 09:00ZGoogle Gemini matches the pinned reference on 24/24 fields; specialist review accepts the scoped result and usage joins.PASS — checker plus specialist acceptance is required.$0.010750 = (4920×$1.25 + 920×$5.00)/1M
Name constraints/revocation / batch37-coding-731-r2
batch37-coding-m3-r2
model/run: OpenAI GPT-4o; Anthropic Claude Sonnet; Google Gemini; observed 2026-08-27		Frozen adversarial fixture; identical prompt/budget and repair log; run 09:16ZGoogle Gemini matches 21/24 fields; three repairs are visible and the specialist accepts only the narrowed claim.PASS WITH REPAIR — no unreviewed claim is promoted.$0.015575 = (7180×$1.25 + 1320×$5.00)/1M
Malformed DER/clock boundary / batch37-coding-731-r3
batch37-coding-m3-r3
model/run: OpenAI GPT-4o; Anthropic Claude Sonnet; Google Gemini; observed 2026-08-27		Frozen counterexample fixture; pinned run and checker output; run 09:32ZThe checker rejects the broad result; certificate-path verification evidence is incomplete.BOUNDARY — certificate-path verification evidence is incomplete.Unavailable — certificate-path verification evidence is incomplete

Formula / scoring rule: Acceptance = pinned path parity + hostname/time/policy fidelity + fail-closed behavior + differential/fuzz tests + reviewer escalation. Matched X.509 verification benchmark; pinned reference and reviewer records; matched evidence/pricing registry verified 2026-08-27; unsupported fields fail closed as Unavailable. Dated first-party pricing/evidence registry, verified 2026-08-27. Module-local first-party sources: Google Gemini API documentationGoogle Gemini model pricing.

Verified 2026-08-08. Data owner: Luna. Prior-batch and adjacent evidence are not substituted. Run the coding Batch 37 evidence scenario →

Batch 38 · DNS resolvers, ASN.1 codecs, and Unicode collation implementation gates

Frozen verification window: 2026-08-27 UTC. Inputs, model/run identity, formulas or rubrics, field-level results, decision boundaries, dated provenance, and exact token bills are server-rendered. Unsupported facts fail closed as Unavailable.

1. Recursive and validating DNS resolver implementation suite

Frozen fixture / runVisible inputsField-level resultDecision boundaryReproducible tokenBill / state
Delegation glue CNAME / batch38-coding-711-r1
batch38-coding-m1-r1
model/run: OpenAI GPT-4o; Anthropic Claude Sonnet; Google Gemini; observed 2026-08-27		pinned trace dns-01, 14 packets, CNAME chain length 3, checker v38.1; run 02:00Zresolver patch passes 14/14 wire assertions, TTL 300 preserved, specialist accepts 24/24 review fields; input 4,920/output 920 tokens.PASS — checker and reviewer both accept the scoped implementation.$0.038400 = (4920×$5.00 + 920×$15.00)/1M
Wildcard negative DNSSEC / batch38-coding-711-r2
batch38-coding-m1-r2
model/run: OpenAI GPT-4o; Anthropic Claude Sonnet; Google Gemini; observed 2026-08-27		trace dns-02, NSEC3 proof, wildcard and NXDOMAIN cache; 3 repairs; run 02:16Z21/24 assertions pass after reviewer repair; cache poisoning claim removed; input 7,180/output 1,320 tokens.PASS WITH REPAIR — only checked DNSSEC behavior is accepted.$0.055700 = (7180×$5.00 + 1320×$15.00)/1M
Poisoning stale truncation / batch38-coding-711-r3
batch38-coding-m1-r3
model/run: OpenAI GPT-4o; Anthropic Claude Sonnet; Google Gemini; observed 2026-08-27		trace dns-03, stale cache, TC=1, loop budget 64; fuzz artifact incomplete; run 02:32Zchecker rejects broad conformance; DNS resolver evidence is incomplete.UNAVAILABLE — DNS resolver conformance evidence is incomplete.Unavailable — DNS resolver conformance evidence is incomplete

Formula / scoring rule: Acceptance = wire/reference trace + cache TTL/trust invariants + loop/resource bounds + malformed rejection + differential/fuzz tests + reviewer repair + accepted-patch cost. Matched DNS resolver implementation benchmark; pinned traces and reviewer records; matched evidence/pricing registry verified 2026-08-27; unsupported fields fail closed as Unavailable. Dated first-party pricing/evidence registry, verified 2026-08-27. Module-local first-party sources: OpenAI API documentationOpenAI API pricingOpenAI model pricingOpenAI API pricing.

2. ASN.1 BER/DER/CER codec benchmark

Frozen fixture / runVisible inputsField-level resultDecision boundaryReproducible tokenBill / state
Tag length indefinite / batch38-coding-721-r1
batch38-coding-m2-r1
model/run: OpenAI GPT-4o; Anthropic Claude Sonnet; Google Gemini; observed 2026-08-27		asn1-v38-a, 18 BER vectors, indefinite-length containers, decoder checker; run 03:00Z18/18 vectors decode and re-encode; rejection locus exact; specialist accepts 24/24 fields; input 4,640/output 860 tokens.PASS — canonical bytes and checker output agree.$0.026820 = (4640×$3.00 + 860×$15.00)/1M
SET OID time forms / batch38-coding-721-r2
batch38-coding-m2-r2
model/run: OpenAI GPT-4o; Anthropic Claude Sonnet; Google Gemini; observed 2026-08-27		DER SET sorting, OID arcs, UTCTime/GeneralizedTime; two repairs; run 03:16Z21/24 fields accepted; SET sort repair visible; allocation bound stays under 2.0× reference; input 7,060/output 1,260 tokens.PASS WITH REPAIR — repaired canonicalization is limited to the tested vectors.$0.040080 = (7060×$3.00 + 1260×$15.00)/1M
Recursion bomb malformed stream / batch38-coding-721-r3
batch38-coding-m2-r3
model/run: OpenAI GPT-4o; Anthropic Claude Sonnet; Google Gemini; observed 2026-08-27		CER depth 4096, malformed length, fuzz corpus hash missing; run 03:32Zdecoder rejects sample but differential/fuzz evidence is incomplete.UNAVAILABLE — ASN.1 codec conformance evidence is incomplete.Unavailable — ASN.1 codec conformance evidence is incomplete

Formula / scoring rule: Acceptance = pinned-vector agreement + canonical round-trip bytes + rejection locus + allocation/depth bounds + differential/property/fuzz results + reviewer correction + matched-run spend. Matched ASN.1 codec benchmark; pinned vectors and reviewer records; matched evidence/pricing registry verified 2026-08-27; unsupported fields fail closed as Unavailable. Dated first-party pricing/evidence registry, verified 2026-08-27. Module-local first-party sources: Anthropic API documentationAnthropic model pricingAnthropic model pricingAnthropic model pricing.

3. Unicode collation and locale-tailoring engine gate

Frozen fixture / runVisible inputsField-level resultDecision boundaryReproducible tokenBill / state
Normalization contractions / batch38-coding-731-r1
batch38-coding-m3-r1
model/run: OpenAI GPT-4o; Anthropic Claude Sonnet; Google Gemini; observed 2026-08-27		UCA 15.1 vectors, NFC/NFD, Spanish contractions, checker col-01; run 04:00Z32/32 order vectors and keys agree; transitivity passes; specialist accepts 24/24 fields; input 4,180/output 780 tokens.PASS — locale/version fixture and checker are pinned.$0.009125 = (4180×$1.25 + 780×$5.00)/1M
Numeric case width emoji / batch38-coding-731-r2
batch38-coding-m3-r2
model/run: OpenAI GPT-4o; Anthropic Claude Sonnet; Google Gemini; observed 2026-08-27		numeric collation, case/width folding, emoji sequences; one tailoring repair; run 04:16Z21/24 fields accepted; repaired numeric tie-break recorded; input 6,820/output 1,180 tokens.PASS WITH REPAIR — only the pinned locale behavior is promoted.$0.014425 = (6820×$1.25 + 1180×$5.00)/1M
French Turkish CJK upgrade / batch38-coding-731-r3
batch38-coding-m3-r3
model/run: OpenAI GPT-4o; Anthropic Claude Sonnet; Google Gemini; observed 2026-08-27		locale upgrade from 15.1 to 16.0, incremental sort and CJK weights; run 04:32Zversion-specific collation evidence is incomplete.UNAVAILABLE — Unicode collation evidence is incomplete.Unavailable — Unicode collation evidence is incomplete

Formula / scoring rule: Acceptance = collation-key/order agreement + stability/transitivity + locale/version fidelity + incremental sort + property tests + reviewer escalation + accepted-patch cost. Matched Unicode collation benchmark; pinned vectors and reviewer records; matched evidence/pricing registry verified 2026-08-27; unsupported fields fail closed as Unavailable. Dated first-party pricing/evidence registry, verified 2026-08-27. Module-local first-party sources: Google API documentationGoogle Gemini model pricingGoogle model pricingGoogle Gemini model pricing.

Verified 2026-08-08. Data owner: Luna. Prior-batch and adjacent evidence are not substituted. Run the coding Batch 38 evidence scenario →

Which models rank highest for Coding?

"Fit" is a requirements match, not a quality benchmark — it combines price, measured speed, context window, and (where we have run it) graded accuracy on this task. Formula below.

#ModelProviderFitEvidenceTask price/MTokens/secContextScored on
1Muse Spark 1.3 ContributorMeta94100/1$0.12—1.0Mprice, context, evidence
2GLM-5.2Z.ai82100/1$2.00—1Mprice, context, evidence
3Gemini 2.5 Flash LiteLegacyGoogle80—$0.16—1Mprice, context
4GPT-OSS 120B (Cerebras)Cerebras7380/1$0.432450131Kprice, context, speed, evidence
5Amazon Nova LiteAmazon7198/1$0.10108300Kprice, context, speed, evidence
6Ministral 8BMistral7099/1$0.15158256Kprice, context, speed, evidence
7Amazon Nova MicroAmazon7097/1$0.06168128Kprice, context, speed, evidence
8Mistral Small 3.1Mistral6899/1$0.24121256Kprice, context, speed, evidence

What will Coding cost?

At 20,000 coding agent loop calls/month:

ModelTask price/MEst. monthly cost
Muse Spark 1.3 Contributor$0.12$12.00
GLM-5.2$2.00$200.00
Gemini 2.5 Flash Lite$0.16$16.00

How is the best LLM for Coding ranked?

Task rubric:

  • Graded correctness and instruction-following (50%)
  • Task-shaped API price (20%)
  • Measured generation speed (20%)
  • Context-window headroom (10%)

Weights: evidence 50%, price 20%, speed 20%, context 10%.

Requirements: none — every current model is eligible. 49 models eligible.

Price and context sub-scores are min-max normalised (log-scaled) within this task's eligible set only. Speed uses measured tokens/sec only — estimated rows are excluded. A model missing a measurement is never scored as zero: its weight is redistributed across the components we do have, and "Scored on" in the table above shows exactly which ones.

Prices verified 2026-08-08, accuracy graded 2026-06-21.

Availability: Legacy models are shown for historical comparison but cannot win. The published pick must have a current pricing record, model specification, and non-legacy lifecycle status.

What failure modes matter for Coding?

  • A correct algorithm can still fail a production workflow when it leaks reasoning or explanation into code-only output.
  • Short snippet tests do not measure multi-file edits, tool use, or repository-scale debugging.
  • The premium algorithm run shows that correctness can come with materially higher latency and cost.

What related resources help with Coding?

Meta provider hubMuse Spark 1.3 Contributor pricingBest LLM for Math & ReasoningBest LLM for Chatbots & SupportBest LLM for Structured Data Extraction

What are common questions about the best LLM for Coding?

Is a reasoning model always better for coding?

Not for short, well-specified tasks like a single function — non-reasoning models are often just as correct and much cheaper. Reasoning mode pays off on multi-file, multi-step work.

Does graded accuracy on a short snippet predict real-world coding quality?

It predicts one thing well: whether a model follows a precise spec without adding unrequested scaffolding. It does not test multi-file reasoning — pair it with the agents task page for that.

Should I pick the cheapest model that passed?

For high-volume, low-stakes generation, yes. For code that ships without review, weight accuracy over price — a bug is more expensive than the token difference.

Run this exact prompt against the top 3

Don't take a ranking's word for it — try Muse Spark 1.3 Contributor and its closest alternatives on your own prompt.

Try It Free