Best LLM for Math & Reasoning in 2026
For math & reasoning, GLM-5.2 is our pick: $3.40/M tokens on a Reasoning-heavy query workload, 1M context, graded 99/100 across 1 run. Muse Spark 1.3 Contributor is the value alternative at $0.17/M.
Multi-step math and logic puzzles punish models that guess instead of verifying intermediate steps. We require a reasoning mode and weight graded accuracy heavily — price is a secondary concern for work this failure-sensitive.
Quick answer: What is the best LLM for math and reasoning?
GLM-5.2, from Z.ai, is the best fit for math & reasoning at $3.40 per million task tokens on a Reasoning-heavy query workload, with a 1M-token context window. Muse Spark 1.3 Contributor is the cheaper value alternative at $0.17 per million tokens.
Can't use GLM-5.2? See GLM-5.2 alternatives.
What evidence supports the Math & Reasoning recommendation?
Graded on 3 of 28 eligible models, run 2026-06-21. Full prompts, verbatim outputs, and grading notes below.
Solving a Constraint Logic Puzzle
A five-house constraint-satisfaction puzzle with five interlocking clues and a single valid solution. The model must reason through both cases, eliminate the dead end, and report the exact arrangement and count.
| # | Model | Accuracy ↓ | Speed ↕ | Cost ↕ | Output |
|---|---|---|---|---|---|
| 1 | GPT-5.4 ProOpenAI · gpt-5.4-pro | 100★ | 21.7 t/s37818 ms | $0.1533$180/M out | |
Agent grade — 100/100. Correct unique arrangement and count (1) with tight, rigorous case analysis on both yellow positions, final count on its own line. Now that the live API reports real usage for this model, its true cost is shown — by far the priciest run here. Verbatim output (178 in / 822 out tokens, 37818 ms, $0.1533) Let the houses be positions 1–5. - Since the **green house is immediately to the right of the white house**, the pair must be one of: - (1,2) = white, green - (2,3) = white, green - (3,4) = white, green - (4,5) = white, green - The **yellow house is at an end** (position 1 or 5), and there are **exactly two houses between green and yellow**, so green must be 3 positions away from yellow. ## Case 1: Yellow at 1 Then green must be at 4. So white must be at 3. Arrangement so far: 1. yellow 2. ? 3. white 4. green 5. ? The remaining colors are red and blue. Since **red is left of blue**, we must have: 2. red 5. blue But **blue cannot be at either end**, and position 5 is an end. So this case is impossible. ## Case 2: Yellow at 5 Then green must be at 2. So white must be at 1. Arrangement so far: 1. white 2. green 3. ? 4. ? 5. yellow The remaining colors are red and blue. Since **red is left of blue**, we must have: 3. red 4. blue Blue is not at an end, so this works. ## Valid arrangement 1. white, 2. green, 3. red, 4. blue, 5. yellow 1 | |||||
| 2 | Claude Opus 4.8Anthropic · claude-opus-4-8 | 100★ | 97.3 t/s10119 ms | $0.025855$25/M out | |
Agent grade — 100/100. Correct arrangement and count via tight, fully-verified case analysis on both yellow positions, with a final five-clue check. The leading <think> block is Opus 4.8's reasoning channel surfaced by our gateway, not dumped into the answer — the final solution itself is clean and correct. Verbatim output (246 in / 985 out tokens, 10119 ms, $0.025855) <think> Houses 1-5. Colors: red, blue, green, yellow, white. Clue 1: green = white+1 (white immediately left of green). Clue 2: red < blue. Clue 3: yellow at position 1 or 5. Clue 4: |green - yellow| = 3. Clue 5: blue not at 1 or 5. Let me enumerate by yellow position. Case yellow=1: Clue 4: green = 1±3 = 4 (since -2 invalid). So green=4. Clue 1: white=3. Remaining: red, blue for positions 2,5. Clue 5: blue not at 5, so blue=2, red=5. But clue 2: red<blue → 5<2 false. Invalid. So no valid with yellow=1. Case yellow=5: Clue 4: green = 5±3 = 2 or 8. green=2. Clue 1: white=1. Remaining: red, blue for positions 3,4. Clue 5: blue not at 1,5, ok both. Clue 2: red<blue. Positions 3,4: red=3, blue=4. Valid. Arrangement: 1 white, 2 green, 3 red, 4 blue, 5 yellow. Check all clues: green(2)=white(1)+1 ✓. red(3)<blue(4) ✓. yellow=5 end ✓. |green-yellow|=|2-5|=3 ✓. blue=4 not end ✓. Only one arrangement. </think> ## Reasoning **Clue constraints:** - Green = White + 1 (adjacent) - Red < Blue - Yellow at position 1 or 5 - |Green − Yellow| = 3 - Blue at position 2, 3, or 4 **Case: Yellow = 1** - Clue 4: Green = 4 (only valid option). So White = 3. - Remaining positions 2 and 5 get red and blue. - Clue 5 forbids blue at 5, so Blue = 2, Red = 5. - But Clue 2 needs Red < Blue → 5 < 2, false. ❌ No solution. **Case: Yellow = 5** - Clue 4: Green = 2 (only valid option). So White = 1. - Remaining positions 3 and 4 get red and blue. - Clue 2 (Red < Blue): Red = 3, Blue = 4. - Check Clue 5: Blue = 4, not an end. ✓ This gives: **House 1: White, House 2: Green, House 3: Red, House 4: Blue, House 5: Yellow** Verify all clues: Green(2)=White(1)+1 ✓; Red(3)<Blue(4) ✓; Yellow at end ✓; |2−5|=3 ✓; Blue not at end ✓. ## Valid Arrangement(s) 1. White, Green, Red, Blue, Yellow **1** | |||||
| 3 | GLM 5.2 (Max)Z.ai · glm-5.2 | 99 | 73.7 t/s38370 ms | $0.012698$4.4/M out | |
Agent grade — 99/100. Reaches the correct arrangement and count with sound, fully-verified case analysis. The long <think> section is GLM's reasoning surfaced by our gateway from its separate reasoning channel (not dumped into the answer), so it isn't penalised — the final solution itself is clean and correct. Verbatim output (185 in / 2827 out tokens, 38370 ms, $0.012698) <think> The problem asks for the number of distinct color arrangements of 5 houses in a row, numbered 1 to 5 from left to right. The colors are red, blue, green, yellow, white. Clues: 1. Green is immediately to the right of White. (W, G) 2. Red is somewhere to the left of Blue. 3. Yellow is at one of the two ends (house 1 or house 5). 4. There are exactly two houses between Green and Yellow. 5. Blue is not at either end (so Blue is in house 2, 3, or 4). Let's analyze the positions of Yellow and the W-G block. Yellow is at 1 or 5. Case 1: Yellow is at 1. Since there are exactly two houses between Green and Yellow, Green must be at 4. If Green is at 4, White must be at 3 (since Green is immediately to the right of White). So we have: 1: Yellow 2: ? 3: White 4: Green 5: ? The remaining colors are Red and Blue. They must go in houses 2 and 5. Clue 5 says Blue is not at either end. So Blue cannot be at 5. Thus, Blue must be at 2, and Red must be at 5. But Clue 2 says Red is somewhere to the left of Blue. If Blue is at 2 and Red is at 5, Red is to the right of Blue. This violates Clue 2. So Case 1 yields no valid arrangements. Case 2: Yellow is at 5. Since there are exactly two houses between Green and Yellow, Green must be at 2. If Green is at 2, White must be at 1 (since Green is immediately to the right of White). So we have: 1: White 2: Green 3: ? 4: ? 5: Yellow The remaining colors are Red and Blue. They must go in houses 3 and 4. Clue 5 says Blue is not at either end. Houses 3 and 4 are not ends, so this is satisfied. Clue 2 says Red is to the left of Blue. So Red must be at 3, and Blue must be at 4. Let's check all clues for this arrangement: 1: White 2: Green 3: Red 4: Blue 5: Yellow Clue 1: Green is immediately to the right of White. (W at 1, G at 2) - Yes. Clue 2: Red is to the left of Blue. (R at 3, B at 4) - Yes. Clue 3: Yellow is at one of the two ends. (Y at 5) - Yes. Clue 4: Exactly two houses between Green and Yellow. (G at 2, Y at 5, houses 3 and 4 are between) - Yes. Clue 5: Blue is not at either end. (B at 4) - Yes. Are there any other possibilities? Let's double check. Could Yellow be at 1 and Green at 4? We checked this, it leads to W at 3, Y at 1. Remaining houses 2, 5 for R, B. B cannot be at 5, so B at 2, R at 5. But R must be left of B. Contradiction. Could Yellow be at 5 and Green at 2? We checked this, it leads to W at 1, Y at 5. Remaining houses 3, 4 for R, B. B cannot be at 5 (already satisfied). R must be left of B, so R at 3, B at 4. This works. Wait, are there any other interpretations of "exactly two houses between"? If Yellow is at 1, Green is at 4. Houses 2 and 3 are between. That's exactly two houses. If Yellow is at 5, Green is at 2. Houses 3 and 4 are between. That's exactly two houses. Are there any other positions for Yellow? No, clue 3 says Yellow is at one of the two ends. Are there any other positions for Green? No, because the distance between 1 and 4 is 2 houses, and between 2 and 5 is 2 houses. So Green must be at 2 or 4. If Green is at 4, White is at 3. If Green is at 2, White is at 1. Let's quickly review the steps. Colors: R, B, G, Y, W. Positions: 1, 2, 3, 4, 5. Y is at 1 or 5. If Y=1, G=4. W=3. Remaining: 2, 5 for R, B. B cannot be at 5 (end), so B=2, R=5. But R must be left of B. 5 is not left of 2. Invalid. If Y=5, G=2. W=1. Remaining: 3, 4 for R, B. B cannot be at ends (3 and 4 are not ends, so both valid). R must be left of B, so R=3, B=4. Valid. Is there any ambiguity in "The green house is immediately to the right of the white house"? No, it means W is at x, G is at x+1. Is there any ambiguity in "exactly two houses between"? No, |pos(G) - pos(Y)| = 3. Is there any ambiguity in "The red house is somewhere to the left of the blue house"? No, pos(R) < pos(B). Let's check if there's any other way. What if Yellow is at 1, Green is at 4, White is at 3. Houses: 1: Y, 2: ?, 3: W, 4: G, 5: ? Colors left: R, B. If B=2, R=5. R is not left of B. If B=5, R=2. B is at an end, which violates clue 5. So no solution for Y=1. What if Yellow is at 5, Green is at 2, White is at 1. Houses: 1: W, 2: G, 3: ?, 4: ?, 5: Y. Colors left: R, B. If R=3, B=4. R is left of B. B is not at an end. Valid. If R=4, B=3. R is not left of B. Invalid. So there is exactly 1 valid arrangement. The arrangement is: House 1: White House 2: Green House 3: Red House 4: Blue House 5: Yellow Let me re-read the prompt to make sure I didn't miss anything. "Five houses stand in a row, numbered 1 to 5 from left to right." "Each house is painted a different color, one of: red, blue, green, yellow, white." "1. The green house is immediately to the right of the white house." -> W, G "2. The red house is somewhere to the left of the blue house (not necessarily adjacent)." -> R < B "3. The yellow house is at one of the two ends of the row." -> Y=1 or Y=5 "4. There are exactly two houses between the green house and the yellow house." -> |G - Y| = 3 "5. The blue house is not at either end." -> B != 1, B != 5 Everything seems perfectly straightforward. Let's write down the solution clearly. - State the possible positions for Yellow based on Clue 3 (1 or 5). - Use Clue 4 to find the corresponding positions for Green (4 or 2). - Use Clue 1 to find the corresponding positions for White (3 or 1). - Evaluate the two cases: - Case 1: Y=1, G=4, W=3. Remaining positions for R and B are 2 and 5. Clue 5 says B cannot be 5, so B=2, R=5. But Clue 2 says R must be left of B, which is violated. So this case is invalid. - Case 2: Y=5, G=2, W=1. Remaining positions for R and B are 3 and 4. Clue 2 says R must be left of B, so R=3, B=4. Clue 5 is satisfied since B is not at an end. This case is valid. - Conclude there is exactly 1 valid arrangement. - List the arrangement as house 1 through house 5. - State the final count on its own line. The arrangement: House 1: White House 2: Green House 3: Red House 4: Blue House 5: Yellow Final count: 1 Let me double check the "exactly two houses between" part. If Y is at 1, and G is at 4. Houses between 1 and 4 are 2 and 3. That's exactly two houses. If Y is at 5, and G is at 2. Houses between 2 and 5 are 3 and 4. That's exactly two houses. Yes, this is correct. What if "exactly two houses between" means the distance is 2? No, "two houses between" means there are two houses separating them. For example, between house 1 and house 4, there are houses 2 and 3. That's two houses. If it meant distance is 2, it would be "one house between". So |G - Y| = 3 is correct. Let's do a quick sanity check. Colors: R, B, G, Y, W. Arrangement: W, G, R, B, Y. 1. Green immediately right of White: W at 1, G at 2. Yes. 2. Red left of Blue: R at 3, B at 4. Yes. 3. Yellow at an end: Y at 5. Yes. 4. Exactly two houses between Green and Yellow: G at 2, Y at 5. Houses 3 and 4 are between. Yes. 5. Blue not at either end: B at 4. Yes. Everything is consistent. The logic is solid. No other interpretations seem likely. I will structure the output to show the reasoning, list the valid arrangement, and then state the final count. </think> To solve this logic puzzle, we can deduce the positions of the houses step-by-step using the given clues: **Step 1: Determine the possible positions for the Yellow and Green houses.** * Clue 3 states that the yellow house is at one of the two ends (house 1 or house 5). * Clue 4 states that there are exactly two houses between the green house and the yellow house. This gives us two possible scenarios: * **Scenario A:** Yellow is in house 1, which means Green must be in house 4. * **Scenario B:** Yellow is in house 5, which means Green must be in house 2. **Step 2: Place the White house.** * Clue 1 states that the green house is immediately to the right of the white house (meaning they are a pair: White, Green). * In **Scenario A** (Green in 4), White must be in house 3. * In **Scenario B** (Green in 2), White must be in house 1. **Step 3: Place the Red and Blue houses.** * Clue 5 states that the blue house is not at either end, meaning it cannot be in house 1 or house 5. Let's test **Scenario A**: * Houses: 1: Yellow, 2: ?, 3: White, 4: Green, 5: ? * The remaining colors for houses 2 and 5 are Red and Blue. * Since Blue cannot be at an end (house 5), Blue must be in house 2, leaving Red in house 5. * However, Clue 2 says the red house is to the left of the blue house. Here, Red (5) is to the right of Blue (2), which violates the clue. * Therefore, Scenario A is invalid. Let's test **Scenario B**: * Houses: 1: White, 2: Green, 3: ?, 4: ?, 5: Yellow * The remaining colors for houses 3 and 4 are Red and Blue. * Clue 2 says Red is to the left of Blue. Therefore, Red must be in house 3 and Blue must be in house 4. * Let's check Clue 5: Blue is in house 4, which is not an end. This fits perfectly. The only valid arrangement is: House 1: White House 2: Green House 3: Red House 4: Blue House 5: Yellow 1 | |||||
| 4 | Gemini 3.1 ProGoogle · gemini-3.1-pro | 98 | 22.7 t/s22828 ms | $0.006576$12/M out | |
Agent grade — 98/100. Correct arrangement and count via clean step-by-step deduction. Slightly lighter on explicit verification than Opus, but fully sound and the cheapest correct answer here. Verbatim output (180 in / 518 out tokens, 22828 ms, $0.006576) To find the valid color arrangements, we can deduce the positions of the houses step by step (numbered 1 to 5 from left to right): 1. **Analyze Yellow and Green's positions:** Clue 3 states the yellow house is at one of the two ends (Position 1 or Position 5). Clue 4 states there are exactly two houses between the green house and the yellow house. - *Case A:* If the yellow house is at Position 1, the green house must be at Position 4. - *Case B:* If the yellow house is at Position 5, the green house must be at Position 2. 2. **Analyze White's position:** Clue 1 states the green house is immediately to the right of the white house. This means the white house is exactly one spot to the left of the green house. - *Case A:* If green is at 4, white must be at 3. (Arrangement so far: 1:Yellow, 2:Empty, 3:White, 4:Green, 5:Empty) - *Case B:* If green is at 2, white must be at 1. (Arrangement so far: 1:White, 2:Green, 3:Empty, 4:Empty, 5:Yellow) 3. **Analyze Red and Blue's positions:** The remaining colors to place are red and blue. Clue 2 states the red house is somewhere to the left of the blue house. - *Case A:* The empty spots are 2 and 5. For red to be to the left of blue, red must be at 2 and blue must be at 5. However, Clue 5 states the blue house cannot be at either end. Since Position 5 is an end, **Case A is invalid.** - *Case B:* The empty spots are 3 and 4. For red to be to the left of blue, red must be at 3 and blue must be at 4. Let's check Clue 5: Blue is at Position 4, which is not an end. This works perfectly! This leaves us with exactly one valid arrangement that satisfies all the clues. **Valid Arrangement:** House 1: White, House 2: Green, House 3: Red, House 4: Blue, House 5: Yellow 1 | |||||
Reproducible Math & Reasoning evidence and decision rubric
| Test / run | Prompt and verification | Hard rule |
|---|---|---|
| Code Snippet | exact prompt + 20 recorded runs | Correct iterative algorithm and code-only output |
| Logic Puzzle | exact prompt + 4 recorded runs | Exhaustive search: only White, Green, Red, Blue, Yellow; count = 1 |
Run dates: budget suite 2026-06-16T20:31:30.728Z; premium suite 2026-06-21T00:00:00.000Z. Results are not a claim about every repository or prompt.
Reasoning-only accuracy / cost / latency frontier
| Model | Accuracy | Latency | Output tokens | Run cost | Failure / qualification note |
|---|---|---|---|---|---|
| GPT-5.4 Pro | 100/100 | 37818 ms | 822 | $0.153 | Correct unique arrangement and count (1) with tight, rigorous case analysis on both yellow positions, final count on its own line. Now that the live API reports real usage for this model, its true cost is shown — by far the priciest run here. |
| Claude Opus 4.8 | 100/100 | 10119 ms | 985 | $0.026 | Correct arrangement and count via tight, fully-verified case analysis on both yellow positions, with a final five-clue check. The leading <think> block is Opus 4.8's reasoning channel surfaced by our gateway, not dumped into the answer — the final solution itself is clean and correct. |
| GLM 5.2 (Max) | 99/100 | 38370 ms | 2827 | $0.013 | Reaches the correct arrangement and count with sound, fully-verified case analysis. The long <think> section is GLM's reasoning surfaced by our gateway from its separate reasoning channel (not dumped into the answer), so it isn't penalised — the final solution itself is clean and correct. |
| Gemini 3.1 Pro | 98/100 | 22828 ms | 518 | $0.007 | Correct arrangement and count via clean step-by-step deduction. Slightly lighter on explicit verification than Opus, but fully sound and the cheapest correct answer here. |
Scope: the puzzle result is exhaustively checked over 120 permutations, but it demonstrates constraint solving—not theorem proving, numerical math, or general reasoning. Only models with a current pricing record enter the frontier.
Task-shaped cost ranking (20,000 tasks/month)
| Rank | Model | Effective monthly | Measured verbosity |
|---|---|---|---|
| 1 | Amazon Nova Micro | $4.23 | 0.76× |
| 2 | Amazon Nova Lite | $7.97 | 0.91× |
| 3 | GPT-5 Nano | $11.00 | Unavailable; neutral fallback |
| 4 | Ministral 8B | $11.79 | 0.93× |
| 5 | Gemini 2.5 Flash Lite | $14.00 | Unavailable; neutral fallback |
| 6 | Mistral Small 3.1 | $19.20 | 0.85× |
Verified 2026-08-08. full prompt/run evidence →
Try these models for Math & Reasoning →Batch 9 decision stability for best LLM for Math & Reasoning
1. Accuracy-gated shortlist
| Release score floor | Models clearing floor | Cheapest measured | Fastest measured |
|---|---|---|---|
| 80/100 | 3 | GLM-5.2 | Claude Opus 4.8 |
| 90/100 | 3 | GLM-5.2 | Claude Opus 4.8 |
| 95/100 | 3 | GLM-5.2 | Claude Opus 4.8 |
A model is eligible only when the fixed first-party run has a score at or above the floor. Missing accuracy or speed is Unavailable, never a zero.
2. Dynamic scoring-weight sensitivity
| Evidence / price / speed / context | Recalculated winner | Recalculated fit score | Stability verdict |
|---|---|---|---|
| 50/20/20/10 | Muse Spark 1.3 Contributor | 92.1/100 | Switches: Muse Spark 1.3 Contributor vs GLM-5.2 |
| 70/10/10/10 | GLM-5.2 | 90.6/100 | Switches: Muse Spark 1.3 Contributor vs GLM-5.2 |
| 40/30/20/10 | Muse Spark 1.3 Contributor | 94.1/100 | Switches: Muse Spark 1.3 Contributor vs GLM-5.2 |
Each row recomputes Σ(component score × weight) ÷ Σ(available weights) over the published candidate sub-scores; missing speed or evidence is excluded from that row’s denominator.
3. Parent-to-child decision router
| Trigger | Route | Boundary |
|---|---|---|
| Budget is binding for repeated reasoning calls | /best-llm-for/math-and-reasoning/budget | Recalculate at 1K input / 2K output |
| Prompt needs long context | /best-llm-for/math-and-reasoning/long-context | Capacity is separate from reasoning accuracy |
| Explicit reasoning is mandatory | /best-llm-for/math-and-reasoning/reasoning | Models without reasoning capability remain excluded |
Math-and-reasoning verifier-first audit
The verifier exhaustively checked all 120 permutations of the five colors; exactly 1 permutation satisfies all 5 clues.
| Clue | Verifier check |
|---|---|
| Clue 1 (green right of white) | |G - W| = 1 and G > W |
| Clue 2 (red left of blue) | R < B |
| Clue 3 (yellow at end) | Y = 1 or Y = 5 |
| Clue 4 (two houses between green and yellow) | |G - Y| = 3 |
| Clue 5 (blue not at end) | B != 1 and B != 5 |
| Exhaustive Search | 1 of 120 permutations valid |
| Model | Actual final answer from premiumTests.ts | Verifier result |
|---|---|---|
| GPT-5.4 Pro | Let the houses be positions 1–5. - Since the **green house is immediately to the right of the white house**, the pair must be one of: - (1,2) = white, green - (2,3) = white, green - (3,4) = white, green - (4,5) = white, green - The **yellow house is at an end** (position 1 or 5), and there are **exactly two houses between green and yellow**, so green must be 3 positions away from yellow. ## Case 1: Yellow at 1 Then green must be at 4. So white must be at 3. Arrangement so far: 1. yellow 2. ? 3. white 4. green 5. ? The remaining colors are red and blue. Since **red is left of blue**, we must have: 2. red 5. blue But **blue cannot be at either end**, and position 5 is an end. So this case is impossible. ## Case 2: Yellow at 5 Then green must be at 2. So white must be at 1. Arrangement so far: 1. white 2. green 3. ? 4. ? 5. yellow The remaining colors are red and blue. Since **red is left of blue**, we must have: 3. red 4. blue Blue is not at an end, so this works. ## Valid arrangement 1. white, 2. green, 3. red, 4. blue, 5. yellow 1 | Matches Ground Truth |
| Claude Opus 4.8 | <think> Houses 1-5. Colors: red, blue, green, yellow, white. Clue 1: green = white+1 (white immediately left of green). Clue 2: red < blue. Clue 3: yellow at position 1 or 5. Clue 4: |green - yellow| = 3. Clue 5: blue not at 1 or 5. Let me enumerate by yellow position. Case yellow=1: Clue 4: green = 1±3 = 4 (since -2 invalid). So green=4. Clue 1: white=3. Remaining: red, blue for positions 2,5. Clue 5: blue not at 5, so blue=2, red=5. But clue 2: red<blue → 5<2 false. Invalid. So no valid with yellow=1. Case yellow=5: Clue 4: green = 5±3 = 2 or 8. green=2. Clue 1: white=1. Remaining: red, blue for positions 3,4. Clue 5: blue not at 1,5, ok both. Clue 2: red<blue. Positions 3,4: red=3, blue=4. Valid. Arrangement: 1 white, 2 green, 3 red, 4 blue, 5 yellow. Check all clues: green(2)=white(1)+1 ✓. red(3)<blue(4) ✓. yellow=5 end ✓. |green-yellow|=|2-5|=3 ✓. blue=4 not end ✓. Only one arrangement. </think> ## Reasoning **Clue constraints:** - Green = White + 1 (adjacent) - Red < Blue - Yellow at position 1 or 5 - |Green − Yellow| = 3 - Blue at position 2, 3, or 4 **Case: Yellow = 1** - Clue 4: Green = 4 (only valid option). So White = 3. - Remaining positions 2 and 5 get red and blue. - Clue 5 forbids blue at 5, so Blue = 2, Red = 5. - But Clue 2 needs Red < Blue → 5 < 2, false. ❌ No solution. **Case: Yellow = 5** - Clue 4: Green = 2 (only valid option). So White = 1. - Remaining positions 3 and 4 get red and blue. - Clue 2 (Red < Blue): Red = 3, Blue = 4. - Check Clue 5: Blue = 4, not an end. ✓ This gives: **House 1: White, House 2: Green, House 3: Red, House 4: Blue, House 5: Yellow** Verify all clues: Green(2)=White(1)+1 ✓; Red(3)<Blue(4) ✓; Yellow at end ✓; |2−5|=3 ✓; Blue not at end ✓. ## Valid Arrangement(s) 1. White, Green, Red, Blue, Yellow **1** | Matches Ground Truth |
| Gemini 3.1 Pro | To find the valid color arrangements, we can deduce the positions of the houses step by step (numbered 1 to 5 from left to right): 1. **Analyze Yellow and Green's positions:** Clue 3 states the yellow house is at one of the two ends (Position 1 or Position 5). Clue 4 states there are exactly two houses between the green house and the yellow house. - *Case A:* If the yellow house is at Position 1, the green house must be at Position 4. - *Case B:* If the yellow house is at Position 5, the green house must be at Position 2. 2. **Analyze White's position:** Clue 1 states the green house is immediately to the right of the white house. This means the white house is exactly one spot to the left of the green house. - *Case A:* If green is at 4, white must be at 3. (Arrangement so far: 1:Yellow, 2:Empty, 3:White, 4:Green, 5:Empty) - *Case B:* If green is at 2, white must be at 1. (Arrangement so far: 1:White, 2:Green, 3:Empty, 4:Empty, 5:Yellow) 3. **Analyze Red and Blue's positions:** The remaining colors to place are red and blue. Clue 2 states the red house is somewhere to the left of the blue house. - *Case A:* The empty spots are 2 and 5. For red to be to the left of blue, red must be at 2 and blue must be at 5. However, Clue 5 states the blue house cannot be at either end. Since Position 5 is an end, **Case A is invalid.** - *Case B:* The empty spots are 3 and 4. For red to be to the left of blue, red must be at 3 and blue must be at 4. Let's check Clue 5: Blue is at Position 4, which is not an end. This works perfectly! This leaves us with exactly one valid arrangement that satisfies all the clues. **Valid Arrangement:** House 1: White, House 2: Green, House 3: Red, House 4: Blue, House 5: Yellow 1 | Matches Ground Truth |
| GLM 5.2 (Max) | <think> The problem asks for the number of distinct color arrangements of 5 houses in a row, numbered 1 to 5 from left to right. The colors are red, blue, green, yellow, white. Clues: 1. Green is immediately to the right of White. (W, G) 2. Red is somewhere to the left of Blue. 3. Yellow is at one of the two ends (house 1 or house 5). 4. There are exactly two houses between Green and Yellow. 5. Blue is not at either end (so Blue is in house 2, 3, or 4). Let's analyze the positions of Yellow and the W-G block. Yellow is at 1 or 5. Case 1: Yellow is at 1. Since there are exactly two houses between Green and Yellow, Green must be at 4. If Green is at 4, White must be at 3 (since Green is immediately to the right of White). So we have: 1: Yellow 2: ? 3: White 4: Green 5: ? The remaining colors are Red and Blue. They must go in houses 2 and 5. Clue 5 says Blue is not at either end. So Blue cannot be at 5. Thus, Blue must be at 2, and Red must be at 5. But Clue 2 says Red is somewhere to the left of Blue. If Blue is at 2 and Red is at 5, Red is to the right of Blue. This violates Clue 2. So Case 1 yields no valid arrangements. Case 2: Yellow is at 5. Since there are exactly two houses between Green and Yellow, Green must be at 2. If Green is at 2, White must be at 1 (since Green is immediately to the right of White). So we have: 1: White 2: Green 3: ? 4: ? 5: Yellow The remaining colors are Red and Blue. They must go in houses 3 and 4. Clue 5 says Blue is not at either end. Houses 3 and 4 are not ends, so this is satisfied. Clue 2 says Red is to the left of Blue. So Red must be at 3, and Blue must be at 4. Let's check all clues for this arrangement: 1: White 2: Green 3: Red 4: Blue 5: Yellow Clue 1: Green is immediately to the right of White. (W at 1, G at 2) - Yes. Clue 2: Red is to the left of Blue. (R at 3, B at 4) - Yes. Clue 3: Yellow is at one of the two ends. (Y at 5) - Yes. Clue 4: Exactly two houses between Green and Yellow. (G at 2, Y at 5, houses 3 and 4 are between) - Yes. Clue 5: Blue is not at either end. (B at 4) - Yes. Are there any other possibilities? Let's double check. Could Yellow be at 1 and Green at 4? We checked this, it leads to W at 3, Y at 1. Remaining houses 2, 5 for R, B. B cannot be at 5, so B at 2, R at 5. But R must be left of B. Contradiction. Could Yellow be at 5 and Green at 2? We checked this, it leads to W at 1, Y at 5. Remaining houses 3, 4 for R, B. B cannot be at 5 (already satisfied). R must be left of B, so R at 3, B at 4. This works. Wait, are there any other interpretations of "exactly two houses between"? If Yellow is at 1, Green is at 4. Houses 2 and 3 are between. That's exactly two houses. If Yellow is at 5, Green is at 2. Houses 3 and 4 are between. That's exactly two houses. Are there any other positions for Yellow? No, clue 3 says Yellow is at one of the two ends. Are there any other positions for Green? No, because the distance between 1 and 4 is 2 houses, and between 2 and 5 is 2 houses. So Green must be at 2 or 4. If Green is at 4, White is at 3. If Green is at 2, White is at 1. Let's quickly review the steps. Colors: R, B, G, Y, W. Positions: 1, 2, 3, 4, 5. Y is at 1 or 5. If Y=1, G=4. W=3. Remaining: 2, 5 for R, B. B cannot be at 5 (end), so B=2, R=5. But R must be left of B. 5 is not left of 2. Invalid. If Y=5, G=2. W=1. Remaining: 3, 4 for R, B. B cannot be at ends (3 and 4 are not ends, so both valid). R must be left of B, so R=3, B=4. Valid. Is there any ambiguity in "The green house is immediately to the right of the white house"? No, it means W is at x, G is at x+1. Is there any ambiguity in "exactly two houses between"? No, |pos(G) - pos(Y)| = 3. Is there any ambiguity in "The red house is somewhere to the left of the blue house"? No, pos(R) < pos(B). Let's check if there's any other way. What if Yellow is at 1, Green is at 4, White is at 3. Houses: 1: Y, 2: ?, 3: W, 4: G, 5: ? Colors left: R, B. If B=2, R=5. R is not left of B. If B=5, R=2. B is at an end, which violates clue 5. So no solution for Y=1. What if Yellow is at 5, Green is at 2, White is at 1. Houses: 1: W, 2: G, 3: ?, 4: ?, 5: Y. Colors left: R, B. If R=3, B=4. R is left of B. B is not at an end. Valid. If R=4, B=3. R is not left of B. Invalid. So there is exactly 1 valid arrangement. The arrangement is: House 1: White House 2: Green House 3: Red House 4: Blue House 5: Yellow Let me re-read the prompt to make sure I didn't miss anything. "Five houses stand in a row, numbered 1 to 5 from left to right." "Each house is painted a different color, one of: red, blue, green, yellow, white." "1. The green house is immediately to the right of the white house." -> W, G "2. The red house is somewhere to the left of the blue house (not necessarily adjacent)." -> R < B "3. The yellow house is at one of the two ends of the row." -> Y=1 or Y=5 "4. There are exactly two houses between the green house and the yellow house." -> |G - Y| = 3 "5. The blue house is not at either end." -> B != 1, B != 5 Everything seems perfectly straightforward. Let's write down the solution clearly. - State the possible positions for Yellow based on Clue 3 (1 or 5). - Use Clue 4 to find the corresponding positions for Green (4 or 2). - Use Clue 1 to find the corresponding positions for White (3 or 1). - Evaluate the two cases: - Case 1: Y=1, G=4, W=3. Remaining positions for R and B are 2 and 5. Clue 5 says B cannot be 5, so B=2, R=5. But Clue 2 says R must be left of B, which is violated. So this case is invalid. - Case 2: Y=5, G=2, W=1. Remaining positions for R and B are 3 and 4. Clue 2 says R must be left of B, so R=3, B=4. Clue 5 is satisfied since B is not at an end. This case is valid. - Conclude there is exactly 1 valid arrangement. - List the arrangement as house 1 through house 5. - State the final count on its own line. The arrangement: House 1: White House 2: Green House 3: Red House 4: Blue House 5: Yellow Final count: 1 Let me double check the "exactly two houses between" part. If Y is at 1, and G is at 4. Houses between 1 and 4 are 2 and 3. That's exactly two houses. If Y is at 5, and G is at 2. Houses between 2 and 5 are 3 and 4. That's exactly two houses. Yes, this is correct. What if "exactly two houses between" means the distance is 2? No, "two houses between" means there are two houses separating them. For example, between house 1 and house 4, there are houses 2 and 3. That's two houses. If it meant distance is 2, it would be "one house between". So |G - Y| = 3 is correct. Let's do a quick sanity check. Colors: R, B, G, Y, W. Arrangement: W, G, R, B, Y. 1. Green immediately right of White: W at 1, G at 2. Yes. 2. Red left of Blue: R at 3, B at 4. Yes. 3. Yellow at an end: Y at 5. Yes. 4. Exactly two houses between Green and Yellow: G at 2, Y at 5. Houses 3 and 4 are between. Yes. 5. Blue not at either end: B at 4. Yes. Everything is consistent. The logic is solid. No other interpretations seem likely. I will structure the output to show the reasoning, list the valid arrangement, and then state the final count. </think> To solve this logic puzzle, we can deduce the positions of the houses step-by-step using the given clues: **Step 1: Determine the possible positions for the Yellow and Green houses.** * Clue 3 states that the yellow house is at one of the two ends (house 1 or house 5). * Clue 4 states that there are exactly two houses between the green house and the yellow house. This gives us two possible scenarios: * **Scenario A:** Yellow is in house 1, which means Green must be in house 4. * **Scenario B:** Yellow is in house 5, which means Green must be in house 2. **Step 2: Place the White house.** * Clue 1 states that the green house is immediately to the right of the white house (meaning they are a pair: White, Green). * In **Scenario A** (Green in 4), White must be in house 3. * In **Scenario B** (Green in 2), White must be in house 1. **Step 3: Place the Red and Blue houses.** * Clue 5 states that the blue house is not at either end, meaning it cannot be in house 1 or house 5. Let's test **Scenario A**: * Houses: 1: Yellow, 2: ?, 3: White, 4: Green, 5: ? * The remaining colors for houses 2 and 5 are Red and Blue. * Since Blue cannot be at an end (house 5), Blue must be in house 2, leaving Red in house 5. * However, Clue 2 says the red house is to the left of the blue house. Here, Red (5) is to the right of Blue (2), which violates the clue. * Therefore, Scenario A is invalid. Let's test **Scenario B**: * Houses: 1: White, 2: Green, 3: ?, 4: ?, 5: Yellow * The remaining colors for houses 3 and 4 are Red and Blue. * Clue 2 says Red is to the left of Blue. Therefore, Red must be in house 3 and Blue must be in house 4. * Let's check Clue 5: Blue is in house 4, which is not an end. This fits perfectly. The only valid arrangement is: House 1: White House 2: Green House 3: Red House 4: Blue House 5: Yellow 1 | Matches Ground Truth |
| Model | Output tokens | Amplification vs Gemini 3.1 Pro (518 tok) | Latency | Run cost |
|---|---|---|---|---|
| GPT-5.4 Pro | 822 | 1.59x | 37.8s | $0.153300 |
| Claude Opus 4.8 | 985 | 1.90x | 10.1s | $0.025855 |
| Gemini 3.1 Pro | 518 | 1.00x | 22.8s | $0.006576 |
| GLM 5.2 (Max) | 2,827 | 5.46x | 38.4s | $0.012698 |
| Verifier scope | Result | Boundary |
|---|---|---|
| Exhaustive assignment search | 120 permutations checked; 1 satisfies all 5 clues | This verifies the puzzle answer, not general reasoning ability |
| Arithmetic consistency | Unavailable | No independent arithmetic proof |
| Evidence class | Supported by verifier transfer | Unsupported boundary | Exact promotion test |
|---|---|---|---|
| Constraint solving | Yes — same 120-permutation verifier and prompt | Different verifier/model/version is not transferable | Re-run exact prompt and require the same unique permutation |
| Arithmetic | No | No independent arithmetic proof | Add a deterministic arithmetic verifier and pass every assertion |
| Theorem proving | No | No formal proof kernel evidence | Replay in a named proof assistant with accepted proof artifact |
| Tool-assisted math | No | No tool trace or sandbox evidence | Replay with tool trace, tool outputs, and independently checked final answer |
Evidence transfers only at the same prompt, verifier, model version, and dated run boundary. Output-token amplification is recorded output tokens divided by Gemini 3.1 Pro’s 518-token baseline.
Verified 2026-08-08. Luna is the data owner. “Unavailable” means no compatible dated evidence was found; it is never treated as zero or an inferred successor. Dated task evidence · Run this evidence in All AI Ask.
Batch 13 · math verification topology and transfer boundary
1. Verification-topology planner
| Topology | Samples | Model token cost | Latency | Checker / human success | Decision |
|---|---|---|---|---|---|
| Single sample | 1 | $0.01 | Unavailable | User-supplied | Select only with measured checker/human result |
| 3-sample vote | 3 | $0.03 | Unavailable | User-supplied | Select only with measured checker/human result |
| 5-sample vote | 5 | $0.05 | Unavailable | User-supplied | Select only with measured checker/human result |
| Model + symbolic checker | 1 | $0.01 | Unavailable | User-supplied | Select only with measured checker/human result |
| Human review | 1 | $0.01 | Unavailable | User-supplied | Select only with measured checker/human result |
The exhaustive constraint-puzzle observation remains an observed result. Checker and human success rates are not inferred from it.
2. Reasoning-output budget and truncation surface
| Output expansion | Output tokens | Calculated token bill | Context / max output | Measured speed | Stop rule |
|---|---|---|---|---|---|
| 1× | 2000 | $0.01 | 1,000,000 / 64,000 | Unavailable | Stop at cap/truncation; no quality uplift assigned |
| 2× | 4000 | $0.02 | 1,000,000 / 64,000 | Unavailable | Stop at cap/truncation; no quality uplift assigned |
| 4× | 8000 | $0.04 | 1,000,000 / 64,000 | Unavailable | Stop at cap/truncation; no quality uplift assigned |
| 8× | 16000 | $0.07 | 1,000,000 / 64,000 | Unavailable | Stop at cap/truncation; no quality uplift assigned |
3. Domain-transfer evidence ledger
| Domain | Current evidence | Candidate verdict | Matched unlock test / rubric |
|---|---|---|---|
| arithmetic | Unavailable | Unavailable | Freeze arithmetic prompt, grader, tool policy, and dated multi-model run |
| algebra | Unavailable | Unavailable | Freeze algebra prompt, grader, tool policy, and dated multi-model run |
| geometry | Unavailable | Unavailable | Freeze geometry prompt, grader, tool policy, and dated multi-model run |
| theorem proof | Unavailable | Unavailable | Freeze theorem proof prompt, grader, tool policy, and dated multi-model run |
| constraint solving | premium:constraint-logic-puzzle · 2026-06-16T20:31:30.728Z | GLM-5.2 | Repeat puzzle with independent samples |
| tool-assisted math | Unavailable | Unavailable | Freeze tool-assisted math prompt, grader, tool policy, and dated multi-model run |
Verified 2026-08-08. Data owner: Luna. “Unavailable” means no compatible dated evidence was found; it is not zero or an estimate. Re-verify dated rates, specs, and policy before production use. First-party source · Run this scenario →
Batch 14 · math solver-verifier pairing, evidence strata, and policy loss
1. Solver-verifier pairing matrix
| Solver | Verifier | Generation cost | Verification cost | Retry cost | Human review | Success input / decision |
|---|---|---|---|---|---|---|
| cheap solver | model verifier | $0.01 | $0.0050 | Unavailable | N/A | Unavailable |
| cheap solver | symbolic verifier | $0.01 | Unavailable | Unavailable | N/A | Unavailable |
| cheap solver | human verifier | $0.01 | Unavailable | Unavailable | User-supplied | Unavailable |
| premium solver | model verifier | $0.01 | $0.0050 | Unavailable | N/A | Unavailable |
| premium solver | symbolic verifier | $0.01 | Unavailable | Unavailable | N/A | Unavailable |
| premium solver | human verifier | $0.01 | Unavailable | Unavailable | User-supplied | Unavailable |
Total pairing cost = generation + verification + retry + human review. Success rates are observed, user-supplied, or Unavailable; no broad winner is emitted from a cost-only row.
2. Difficulty-stratified evidence plan
| Difficulty | Prompt count | Arithmetic | Algebra | Geometry | Proof | Constraint/tool | Duplicate runs | Duplicate-run spend | Explicit rubric | Minimum gate |
|---|---|---|---|---|---|---|---|---|---|---|
| easy | 20 | 4 | 4 | 4 | 4 | 4 | 2 | $0.80 | Correctness 0/1; proof validity; constraint satisfaction; no unsupported claim; abstain when unsolved | Shared dated runs across every declared stratum |
| medium | 20 | 4 | 4 | 4 | 4 | 4 | 2 | $0.80 | Correctness 0/1; proof validity; constraint satisfaction; no unsupported claim; abstain when unsolved | Shared dated runs across every declared stratum |
| hard | 20 | 4 | 4 | 4 | 4 | 4 | 2 | $0.80 | Correctness 0/1; proof validity; constraint satisfaction; no unsupported claim; abstain when unsolved | Shared dated runs across every declared stratum |
The plan freezes 20 prompts per difficulty (four per declared stratum), two duplicate runs, the displayed duplicate-run spend, and the rubric before testing. A winner requires the shared-coverage gate; puzzle evidence cannot transfer to other domains.
3. Answer-policy loss surface
| Action | Wrong-answer loss | Delay loss | Review cost | Confidence calibration | Expected loss / decision |
|---|---|---|---|---|---|
| answer | User-supplied | User-supplied | N/A | Unavailable | Unavailable |
| abstain | User-supplied | User-supplied | N/A | Unavailable | Unavailable |
| retry | User-supplied | User-supplied | N/A | Unavailable | Unavailable |
| escalate | User-supplied | User-supplied | User-supplied | Unavailable | Unavailable |
Expected loss = wrong-answer cost + delay cost + retry/review cost, weighted by user-supplied outcome frequencies. Calibration is Unavailable until a dated calibration run exists.
Verified 2026-08-08. Data owner: Luna. “Unavailable” means no compatible dated evidence was found; it is not zero or an estimate. Source / registry · Run this scenario →
Batch 15 · math equivalence, tool-assisted evidence, and confidence calibration
1. Answer-equivalence verifier matrix
| Answer form | False accept | False reject | Review cost | Rule |
|---|---|---|---|---|
| exact integer | Unavailable | Unavailable | Unavailable | Frozen cases required |
| fraction | Unavailable | Unavailable | Unavailable | Frozen cases required |
| decimal/tolerance | Unavailable | Unavailable | Unavailable | Frozen cases required |
| units | Unavailable | Unavailable | Unavailable | Frozen cases required |
| symbolic form | Unavailable | Unavailable | Unavailable | Frozen cases required |
| proof obligation | Unavailable | Unavailable | Unavailable | Frozen cases required |
Formula / rule: report rates over the frozen cases; string mismatch is not model failure when the verifier proves equivalence.
2. Model-only versus calculator/Python/symbolic-tool matched runs
| Task family | Model-only | Tool invocation | Latency/errors/retries | Winner |
|---|---|---|---|---|
| arithmetic | Unavailable | Unavailable | Unavailable | Withheld |
| algebra | Unavailable | Unavailable | Unavailable | Withheld |
| geometry | Unavailable | Unavailable | Unavailable | Withheld |
| constraint | Unavailable | Unavailable | Unavailable | Withheld |
Formula / rule: tool-assisted winner requires matched arithmetic/algebra/geometry/constraint runs and separately attributed tool, token, latency, error, retry, and review evidence.
3. Confidence-calibration protocol
| Score bin | Frozen n | Predictions/outcomes | Brier/ECE | Duplicate-run cost | Routing |
|---|---|---|---|---|---|
| 0–.2 | 20 | Unavailable | Unavailable | Unavailable | Prohibited |
| .2–.4 | 20 | Unavailable | Unavailable | Unavailable | Prohibited |
| .4–.6 | 20 | Unavailable | Unavailable | Unavailable | Prohibited |
| .6–.8 | 20 | Unavailable | Unavailable | Unavailable | Prohibited |
| .8–1.0 | 20 | Unavailable | Unavailable | Unavailable | Prohibited |
Formula / rule: Brier = mean((p − outcome)^2); ECE = sum(bin count / n × |mean p − accuracy|). No confidence routing until every bin has dated predictions and outcomes.
Verified 2026-08-08. Data owner: Luna. Source / registry: dated repository pricing and provider records. “Unavailable” means compatible dated evidence is missing; it is not zero, an estimate, or an inferred capability. Run this evidence scenario →
Batch 16 · reasoning diversity, proof localization, and numeric precision
1. Self-consistency diversity audit
| Samples | Distinct paths | Correlated wrong answers | Vote outcome | Verifier | Token/time cost | Decision |
|---|---|---|---|---|---|---|
| 1 | Unavailable | Unavailable | Unavailable | Unavailable | Unavailable | No quality verdict |
| 3 | Unavailable | Unavailable | Unavailable | Unavailable | Unavailable | No quality verdict |
| 5 | Unavailable | Unavailable | Unavailable | Unavailable | Unavailable | No quality verdict |
| 10 | Unavailable | Unavailable | Unavailable | Unavailable | Unavailable | No quality verdict |
Formula / rule: agreement = winning vote share, not correctness; correctness requires an independent verifier on the same frozen problem.
2. Proof-step localization suite
| Derivation | First invalid step | Justification | False alarms | Repair success | Reviewer effort | Coverage/verdict |
|---|---|---|---|---|---|---|
| valid derivation | Unavailable | Unavailable | Unavailable | Unavailable | Unavailable | Shared coverage required |
| injected invalid step | Unavailable | Unavailable | Unavailable | Unavailable | Unavailable | Shared coverage required |
| repair after localization | Unavailable | Unavailable | Unavailable | Unavailable | Unavailable | Shared coverage required |
Formula / rule: localization requires identifying the first invalid step on valid and injected-invalid derivations; final-answer match alone is insufficient.
3. Numeric precision and dimensional-analysis ladder
| Case | Equivalence rule | Tool/retries | Latency | Spend | Verdict |
|---|---|---|---|---|---|
| exact | Unavailable | Unavailable | Unavailable | Unavailable | No quantitative verdict |
| floating point | Unavailable | Unavailable | Unavailable | Unavailable | No quantitative verdict |
| tolerance | Unavailable | Unavailable | Unavailable | Unavailable | No quantitative verdict |
| significant figures | Unavailable | Unavailable | Unavailable | Unavailable | No quantitative verdict |
| unit conversion | Unavailable | Unavailable | Unavailable | Unavailable | No quantitative verdict |
| overflow/cancellation | Unavailable | Unavailable | Unavailable | Unavailable | No quantitative verdict |
Formula / rule: numeric result is accepted only under the declared exact/tolerance/unit rule; precision handling is not inferred from a final numeric string.
Verified 2026-08-08. Data owner: Luna. Source / registry: dated repository pricing and provider records. “Unavailable” means no compatible dated evidence or observed run; it is not zero or an inferred capability. Run this evidence scenario →
Batch 17 · representation invariance, triage, and error localization
1. Representation-invariance suite
| Transformation | Answer equivalence | Proof-path stability | Regression | Retries/latency | Spend |
|---|---|---|---|---|---|
| variable rename | Unavailable | Unavailable | Unavailable | Unavailable | Unavailable |
| equation reorder | Unavailable | Unavailable | Unavailable | Unavailable | Unavailable |
| unit change | Unavailable | Unavailable | Unavailable | Unavailable | Unavailable |
| diagram to text | Unavailable | Unavailable | Unavailable | Unavailable | Unavailable |
| irrelevant format | Unavailable | Unavailable | Unavailable | Unavailable | Unavailable |
Formula / rule: invariance = equivalent answers across matched transformations, with proof-path stability reported separately; formatting agreement is not correctness.
2. Underspecified-and-inconsistent problem triage
| Case | Solve/clarify/abstain | Hallucinated assumptions | False refusals | Corrections | Cost |
|---|---|---|---|---|---|
| solvable | Unavailable | Unavailable | Unavailable | Unavailable | Unavailable |
| missing data | Unavailable | Unavailable | Unavailable | Unavailable | Unavailable |
| contradictory | Unavailable | Unavailable | Unavailable | Unavailable | Unavailable |
| multiple answer | Unavailable | Unavailable | Unavailable | Unavailable | Unavailable |
Formula / rule: triage correctness = expected solve, clarify, or abstain behavior under the frozen case label; unsupported assumptions are failures, not successful solves.
3. Extraction-to-verification error attribution
| Stage | Correctness | Tool calls | Repair scope | Reviewer effort | Total cost |
|---|---|---|---|---|---|
| fact extraction | Unavailable | Unavailable | Unavailable | Unavailable | Unavailable |
| equation setup | Unavailable | Unavailable | Unavailable | Unavailable | Unavailable |
| plan | Unavailable | Unavailable | Unavailable | Unavailable | Unavailable |
| calculation | Unavailable | Unavailable | Unavailable | Unavailable | Unavailable |
| unit handling | Unavailable | Unavailable | Unavailable | Unavailable | Unavailable |
| final answer | Unavailable | Unavailable | Unavailable | Unavailable | Unavailable |
| checker | Unavailable | Unavailable | Unavailable | Unavailable | Unavailable |
Formula / rule: error localization assigns the first failed stage among extraction, setup, plan, calculation, units, final answer, and checker; final-answer match alone is insufficient.
Verified 2026-08-08. Data owner: Luna. Source / registry: dated repository pricing and provider records. “Unavailable” means no compatible dated evidence or observed run; it is not zero or an inferred capability. Run this evidence scenario →
Batch 18 · counterexamples, independently checked certificates, and probability reasoning
1. False-conjecture counterexample suite
| Domain | Valid witness / minimality | Premises satisfied | Verifier rejection | Retries / reviewer / cost |
|---|---|---|---|---|
| arithmetic | Unavailable | Unavailable | Unavailable | Unavailable |
| algebra | Unavailable | Unavailable | Unavailable | Unavailable |
| geometry | Unavailable | Unavailable | Unavailable | Unavailable |
| combinatorics | Unavailable | Unavailable | Unavailable | Unavailable |
| program invariant | Unavailable | Unavailable | Unavailable | Unavailable |
Formula / rule: counterexample accepted only when a valid minimal witness satisfies all premises and an independent verifier accepts it.
2. Answer-plus-certificate gate
| Task | Answer/objective | Certificate completeness | Independent checker | Gap/invalid witness / spend |
|---|---|---|---|---|
| constraint | Unavailable | Unavailable | Unavailable | Unavailable |
| optimization | Unavailable | Unavailable | Unavailable | Unavailable |
| satisfiability | Unavailable | Unavailable | Unavailable | Unavailable |
Formula / rule: gate = answer correct ∧ certificate complete ∧ independent checker accepts; unverifiable final answers do not pass.
3. Probabilistic and base-rate reasoning suite
| Case | Equation / numeric answer | Assumptions | Simulation/tool / verifier | Failures/retries / cost |
|---|---|---|---|---|
| conditional probability | Unavailable | Unavailable | Unavailable | Unavailable |
| expected value | Unavailable | Unavailable | Unavailable | Unavailable |
| dependence | Unavailable | Unavailable | Unavailable | Unavailable |
| calibration | Unavailable | Unavailable | Unavailable | Unavailable |
| rare event | Unavailable | Unavailable | Unavailable | Unavailable |
Formula / rule: correctness requires the stated equation, compatible assumptions, numeric result, and verifier agreement; confidence alone is not correctness.
Verified 2026-08-08. Data owner: Luna. Source / registry: dated repository records and matched-run evidence. “Unavailable” means no compatible dated source or observed run; it is not zero or an inferred capability. Run this Batch 18 evidence scenario →
Batch 19 · causal inference, combinatorial counting, and optimization sensitivity
Observed benchmark window: 2026-08-26 UTC. Every row is a page-specific frozen fixture with controls, field observations, reviewer decision, token measurement, and exact registry cost.
1. Causal-inference suite
| Dated matched run / case | Frozen controls | Field-level observation | Reviewer decision | Token measurement | Exact cost |
|---|---|---|---|---|---|
| run-20260826-b19-math-01-01 · observational | DAG confounder; backdoor estimand | graph valid; set={Z}; estimate=1.84; checker agree | ACCEPT | 6,200 in + 1,420 out | $0.053200 |
| run-20260826-b19-math-01-02 · collider | DAG X→C←Y; forbid conditioning C | conditioned first; checker flagged; repair removed C | ACCEPT repaired; claim false | 7,100 in + 1,660 out | $0.061600 |
| run-20260826-b19-math-01-03 · mediation | direct/indirect; assumptions disclosed | simulator delta=0.02; estimand valid | ACCEPT | 8,300 in + 1,940 out | $0.072000 |
Formula / rule: pass=graph∧adjustment∧estimand∧numeric/checker∧assumptions Source: pricing registry verified 2026-08-26. Rate: GPT-5.6 Sol, $4.0000 input/M + $20.0000 output/M.
2. Combinatorial-counting gate
| Dated matched run / case | Frozen controls | Field-level observation | Reviewer decision | Token measurement | Exact cost |
|---|---|---|---|---|---|
| run-20260826-b19-math-02-01 · permutations | 7 books; 2 adjacent | 5!×2×6=1,440; brute force=1,440 | ACCEPT certificate | 4,200 in + 920 out | $0.035200 |
| run-20260826-b19-math-02-02 · occupancy | 8 balls / 3 nonempty | inclusion-exclusion=5,796; verifier=5,796 | ACCEPT | 5,600 in + 1,280 out | $0.048000 |
| run-20260826-b19-math-02-03 · recurrence | F(n)=F(n−1)+F(n−2); F10 | closed form=89; verifier=89 | ACCEPT | 3,800 in + 810 out | $0.031400 |
Formula / rule: accepted=decomposition∧closed form equals verifier Source: pricing registry verified 2026-08-26. Rate: GPT-5.6 Sol, $4.0000 input/M + $20.0000 output/M.
3. Optimization-sensitivity suite
| Dated matched run / case | Frozen controls | Field-level observation | Reviewer decision | Token measurement | Exact cost |
|---|---|---|---|---|---|
| run-20260826-b19-math-03-01 · linear | 3 coefficients +10%; LP certificate | objective 124→131.2; active set unchanged | ACCEPT local | 5,100 in + 1,100 out | $0.042400 |
| run-20260826-b19-math-03-02 · integer | capacity bound −1; integer feasibility | active set changed; objective +7; certificate valid | ACCEPT change | 6,400 in + 1,380 out | $0.053200 |
| run-20260826-b19-math-03-03 · scheduling | 2 jobs unavailable; recognize infeasible | status=infeasible; certificate matched | ACCEPT no stability claim | 5,800 in + 1,260 out | $0.048400 |
Formula / rule: perturbation joins solution+certificate+active-set/objective delta Source: pricing registry verified 2026-08-26. Rate: GPT-5.6 Sol, $4.0000 input/M + $20.0000 output/M.
Verified 2026-08-08. Data owner: Luna. Run IDs are match keys; missing vendor fields are scoped to their named run. Run the math evidence scenario →
Batch 20 · numerical-precision propagation, graph-algorithmic verification, and hypothesis-test correctness
Observed benchmark window: 2026-08-26 UTC. Every row is a page-specific frozen fixture with visible controls, a distinct field-level source/run identifier, a registry-computed cost or a scoped Unavailable reason — never a blanket matrix.
1. Numerical-precision and rounding-error propagation suite
| Frozen fixture / matched run | Controls (visible inputs) | Field observation | Decision / boundary | Cost (registry-computed or Unavailable) |
|---|---|---|---|---|
| batch20-math-m1-r1 · 3-step arithmetic chain | 3-step chain with 2-decimal intermediate rounding specified; 400 input tokens; 350 output tokens | Unavailable — no matched precision-propagation run recorded for the 3-step chain as of 2026-08-26 | HOLD — deviation/attribution unverified; fixture cost is reproducible from the registry rate | $0.008600 |
| batch20-math-m1-r2 · 6-step arithmetic chain | 6-step chain with 2-decimal intermediate rounding specified; 650 input tokens; 600 output tokens | Unavailable — no matched precision-propagation run recorded for the 6-step chain as of 2026-08-26 | HOLD — deviation/attribution unverified; fixture cost is reproducible from the registry rate | $0.014600 |
| batch20-math-m1-r3 · 10-step arithmetic chain | 10-step chain with 2-decimal intermediate rounding specified; 950 input tokens; 900 output tokens | Unavailable — no matched precision-propagation run recorded for the 10-step chain as of 2026-08-26 | HOLD — deviation/attribution unverified; fixture cost is reproducible from the registry rate | $0.021800 |
Formula / rule: Matched-run cost = frozen arithmetic-chain token bill at the gpt-5.6 Sol registry rate. Final-answer deviation from an exact-arithmetic reference, error-source attribution to a specific step, and unsupported precision claims require a matched run, which is not present in the registry, so only the fixture cost below is reproducible. Source: pricing registry verified 2026-08-26.
2. Graph-theory algorithmic-reasoning suite
| Frozen fixture / matched run | Controls (visible inputs) | Field observation | Decision / boundary | Cost (registry-computed or Unavailable) |
|---|---|---|---|---|
| batch20-math-m2-r1 · Shortest-path instance — 12-node graph | 12-node weighted graph; requires distance + path certificate; 700 input tokens; 500 output tokens | Unavailable — no matched shortest-path verification run recorded as of 2026-08-26 | HOLD — correctness/certificate validity unverified; instance cost is reproducible from the registry rate | $0.012800 |
| batch20-math-m2-r2 · Bipartite-matching instance — 10×10 graph | 10×10 bipartite graph; requires matching + augmenting-path certificate; 900 input tokens; 650 output tokens | Unavailable — no matched bipartite-matching verification run recorded as of 2026-08-26 | HOLD — correctness/certificate validity unverified; instance cost is reproducible from the registry rate | $0.016600 |
| batch20-math-m2-r3 · Max-flow instance — 15-node network | 15-node flow network; requires max-flow value + min-cut certificate; 1,100 input tokens; 800 output tokens | Unavailable — no matched max-flow verification run recorded as of 2026-08-26 | HOLD — correctness/certificate validity unverified; instance cost is reproducible from the registry rate | $0.020400 |
Formula / rule: Matched-run cost = frozen graph-instance token bill at the gpt-5.6 Sol registry rate. Solution correctness against an independent solver, certificate validity, and false-optimality claims require a matched run, which is not present in the registry, so only the instance cost below is reproducible. Source: pricing registry verified 2026-08-26.
3. Statistical-hypothesis-test suite
| Frozen fixture / matched run | Controls (visible inputs) | Field observation | Decision / boundary | Cost (registry-computed or Unavailable) |
|---|---|---|---|---|
| batch20-math-m3-r1 · Two-sample scenario — α = 0.05 | two-sample dataset, n=30/32; declared significance level 0.05; 550 input tokens; 480 output tokens | Unavailable — no matched hypothesis-test run recorded for the two-sample scenario as of 2026-08-26 | HOLD — test-selection correctness unverified; scenario cost is reproducible from the registry rate | $0.011800 |
| batch20-math-m3-r2 · Paired scenario — α = 0.01 | paired dataset, n=24; declared significance level 0.01; 500 input tokens; 460 output tokens | Unavailable — no matched hypothesis-test run recorded for the paired scenario as of 2026-08-26 | HOLD — test-selection correctness unverified; scenario cost is reproducible from the registry rate | $0.011200 |
| batch20-math-m3-r3 · Multiple-comparison scenario — 5 groups, α = 0.05 | 5-group dataset, n=20 each; declared family-wise significance level 0.05; 700 input tokens; 620 output tokens | Unavailable — no matched multiple-comparison run recorded as of 2026-08-26 | HOLD — correction-application correctness unverified; scenario cost is reproducible from the registry rate | $0.015200 |
Formula / rule: Matched-run cost = frozen scenario-data token bill at the gpt-5.6 Sol registry rate. Correct test selection, assumption-checking disclosure, p-value interpretation accuracy, multiple-comparison correction, and misinterpretation rate require a matched run, which is not present in the registry, so only the scenario cost below is reproducible. Source: pricing registry verified 2026-08-26.
Verified 2026-08-08. Data owner: Luna. Run identifiers are per-row match keys; an Unavailable field names the exact missing dated record or matched run and is never inferred as zero. Run the math-and-reasoning evidence scenario →
Batch 21 · symbolic-algebra correctness, unit-conversion/dimensional-analysis correctness, and seeded-error detection-and-recovery
Observed benchmark window: 2026-08-26 UTC. Every row is a page-specific frozen fixture with visible controls, a distinct field-level source/run identifier, a registry-computed cost or a scoped Unavailable reason — never a blanket matrix.
1. Symbolic-algebra manipulation correctness suite
| Frozen fixture / matched run | Controls (visible inputs) | Field observation | Decision / boundary | Cost (registry-computed or Unavailable) |
|---|---|---|---|---|
| batch21-math-m1-r1 · Equation-simplification task | frozen rational-expression simplification task; 350 input tokens; 300 output tokens | Unavailable — no matched CAS-reference-scoring run recorded for the simplification task as of 2026-08-26 | HOLD — correctness/false-simplification rate unverified; fixture cost is reproducible from the registry rate | $0.007400 |
| batch21-math-m1-r2 · Factoring task | frozen polynomial-factoring task; 380 input tokens; 320 output tokens | Unavailable — no matched CAS-reference-scoring run recorded for the factoring task as of 2026-08-26 | HOLD — correctness/false-simplification rate unverified; fixture cost is reproducible from the registry rate | $0.007920 |
| batch21-math-m1-r3 · Expansion task | frozen multi-term expansion task; 400 input tokens; 340 output tokens | Unavailable — no matched CAS-reference-scoring run recorded for the expansion task as of 2026-08-26 | HOLD — correctness/false-simplification rate unverified; fixture cost is reproducible from the registry rate | $0.008400 |
Formula / rule: Matched-run cost = frozen equation-task token bill at the gpt-5.6 Sol registry rate. Final-expression correctness and intermediate-step validity against an independent computer-algebra-system reference, plus false-simplification rate, require a matched scoring run, which is not present in the registry, so only the fixture cost below is reproducible. Source: pricing registry verified 2026-08-26.
2. Unit-conversion and dimensional-analysis correctness suite
| Frozen fixture / matched run | Controls (visible inputs) | Field observation | Decision / boundary | Cost (registry-computed or Unavailable) |
|---|---|---|---|---|
| batch21-math-m2-r1 · Physics word problem — mixed SI/imperial units | frozen kinematics problem requiring mixed SI/imperial conversion; 450 input tokens; 380 output tokens | Unavailable — no matched dimensional-analysis-scoring run recorded for the physics problem as of 2026-08-26 | HOLD — conversion-factor correctness unverified; problem cost is reproducible from the registry rate | $0.009400 |
| batch21-math-m2-r2 · Engineering word problem — multi-step unit chain | frozen fluid-flow problem requiring a 3-step unit-conversion chain; 520 input tokens; 420 output tokens | Unavailable — no matched dimensional-analysis-scoring run recorded for the engineering problem as of 2026-08-26 | HOLD — conversion-factor correctness unverified; problem cost is reproducible from the registry rate | $0.010480 |
| batch21-math-m2-r3 · Compound-unit word problem — derived-unit consistency | frozen energy/power problem requiring derived-unit consistency; 560 input tokens; 460 output tokens | Unavailable — no matched dimensional-analysis-scoring run recorded for the compound-unit problem as of 2026-08-26 | HOLD — conversion-factor correctness unverified; problem cost is reproducible from the registry rate | $0.011440 |
Formula / rule: Matched-run cost = frozen word-problem token bill at the gpt-5.6 Sol registry rate. Dimensional consistency, correct conversion-factor application, and final-answer accuracy against a worked reference solution require a matched scoring run, which is not present in the registry, so only the problem cost below is reproducible. Source: pricing registry verified 2026-08-26.
3. Seeded-error-detection-and-recovery suite
| Frozen fixture / matched run | Controls (visible inputs) | Field observation | Decision / boundary | Cost (registry-computed or Unavailable) |
|---|---|---|---|---|
| batch21-math-m3-r1 · 3-step solution with a seeded sign error | frozen 3-step solution, error seeded at step 2; 500 input tokens; 300 output tokens | Unavailable — no matched error-detection-scoring run recorded for the 3-step solution as of 2026-08-26 | HOLD — detection/propagation outcome unverified; presented-solution cost is reproducible from the registry rate | $0.008000 |
| batch21-math-m3-r2 · 6-step solution with a seeded unit-conversion error | frozen 6-step solution, error seeded at step 4; 750 input tokens; 450 output tokens | Unavailable — no matched error-detection-scoring run recorded for the 6-step solution as of 2026-08-26 | HOLD — detection/propagation outcome unverified; presented-solution cost is reproducible from the registry rate | $0.012000 |
| batch21-math-m3-r3 · 10-step solution with a seeded algebraic-substitution error | frozen 10-step solution, error seeded at step 7; 1,050 input tokens; 620 output tokens | Unavailable — no matched error-detection-scoring run recorded for the 10-step solution as of 2026-08-26 | HOLD — detection/propagation outcome unverified; presented-solution cost is reproducible from the registry rate | $0.016600 |
Formula / rule: Matched-run cost = frozen multi-step-solution token bill at the gpt-5.6 Sol registry rate, presenting a deliberately incorrect intermediate step. Whether the model detects, names, and corrects the error versus silently propagating it to a wrong final answer requires a matched scoring run, which is not present in the registry, so only the presented-solution cost below is reproducible. Source: pricing registry verified 2026-08-26.
Verified 2026-08-08. Data owner: Luna. Run identifiers are per-row match keys; an Unavailable field names the exact missing dated record or matched run and is never inferred as zero. Run the math-and-reasoning evidence scenario →
Batch 22 · answer-format compliance, matrix/linear-algebra operation correctness, and distribution parameter estimation
Observed benchmark window: 2026-08-26 UTC. Every row is a page-specific frozen fixture with visible controls, a distinct field-level source/run identifier, a registry-computed cost or a scoped Unavailable reason — never a blanket matrix.
1. Answer-format compliance suite
| Frozen fixture / matched run | Controls (visible inputs) | Field observation | Decision / boundary | Cost (registry-computed or Unavailable) |
|---|---|---|---|---|
| batch22-math-m1-r1 · Exact-fraction format required | frozen arithmetic problem; declared exact-fraction output format; 300 input tokens; 200 output tokens | Unavailable — no matched format-compliance-scoring run recorded for the exact-fraction fixture as of 2026-08-26 | HOLD — format-compliance rate unverified; fixture cost is reproducible from the registry rate | $0.005200 |
| batch22-math-m1-r2 · Decimal-to-3-places format required | frozen arithmetic problem; declared 3-decimal-place output format; 320 input tokens; 210 output tokens | Unavailable — no matched format-compliance-scoring run recorded for the decimal-places fixture as of 2026-08-26 | HOLD — format-compliance rate unverified; fixture cost is reproducible from the registry rate | $0.005480 |
| batch22-math-m1-r3 · Boxed-final-answer format required | frozen multi-step problem; declared boxed-final-answer output format; 450 input tokens; 380 output tokens | Unavailable — no matched format-compliance-scoring run recorded for the boxed-answer fixture as of 2026-08-26 | HOLD — format-compliance rate unverified; fixture cost is reproducible from the registry rate | $0.009400 |
Formula / rule: Matched-run cost = frozen problem-plus-format-instruction token bill at the gpt-5.6 Sol registry rate against a declared output format (exact fraction, decimal-to-N-places, boxed final answer). Adherence to the declared format versus a numerically correct but non-compliant answer requires a matched format-scoring run, which is not present in the registry, so only the fixture cost below is reproducible. Source: pricing registry verified 2026-08-26.
2. Matrix/linear-algebra operation correctness suite
| Frozen fixture / matched run | Controls (visible inputs) | Field observation | Decision / boundary | Cost (registry-computed or Unavailable) |
|---|---|---|---|---|
| batch22-math-m2-r1 · 2×2 matrix — determinant and inverse | frozen 2×2 matrix task requesting determinant and inverse; 300 input tokens; 250 output tokens | Unavailable — no matched linear-algebra-reference-scoring run recorded for the 2×2 fixture as of 2026-08-26 | HOLD — correctness unverified; fixture cost is reproducible from the registry rate | $0.006200 |
| batch22-math-m2-r2 · 4×4 matrix — eigenvalues | frozen 4×4 matrix task requesting the eigenvalue set; 450 input tokens; 400 output tokens | Unavailable — no matched linear-algebra-reference-scoring run recorded for the 4×4 fixture as of 2026-08-26 | HOLD — correctness unverified; fixture cost is reproducible from the registry rate | $0.009800 |
| batch22-math-m2-r3 · 6×6 matrix — determinant, inverse, and eigenvalues | frozen 6×6 matrix task requesting determinant, inverse, and eigenvalue set; 650 input tokens; 600 output tokens | Unavailable — no matched linear-algebra-reference-scoring run recorded for the 6×6 fixture as of 2026-08-26 | HOLD — correctness and numerical-stability drift unverified; fixture cost is reproducible from the registry rate | $0.014600 |
Formula / rule: Matched-run cost = frozen matrix-task token bill at the gpt-5.6 Sol registry rate. Correctness of the computed determinant, inverse, or eigenvalue set against an independent linear-algebra reference, and numerical-stability drift at larger matrix sizes, require a matched scoring run, which is not present in the registry, so only the fixture cost below is reproducible. Source: pricing registry verified 2026-08-26.
3. Distribution parameter-estimation correctness suite
| Frozen fixture / matched run | Controls (visible inputs) | Field observation | Decision / boundary | Cost (registry-computed or Unavailable) |
|---|---|---|---|---|
| batch22-math-m3-r1 · 10-point dataset — normal-distribution mean/variance | frozen 10-point dataset; requested mean and variance estimate; 350 input tokens; 220 output tokens | Unavailable — no matched parameter-estimation-scoring run recorded for the 10-point dataset as of 2026-08-26 | HOLD — estimate correctness unverified; fixture cost is reproducible from the registry rate | $0.005800 |
| batch22-math-m3-r2 · 30-point dataset — linear-regression coefficients | frozen 30-point dataset; requested slope/intercept estimate; 600 input tokens; 380 output tokens | Unavailable — no matched parameter-estimation-scoring run recorded for the 30-point dataset as of 2026-08-26 | HOLD — estimate correctness unverified; fixture cost is reproducible from the registry rate | $0.010000 |
| batch22-math-m3-r3 · 80-point dataset — Poisson-rate parameter | frozen 80-point count dataset; requested Poisson rate-parameter estimate; 900 input tokens; 500 output tokens | Unavailable — no matched parameter-estimation-scoring run recorded for the 80-point dataset as of 2026-08-26 | HOLD — estimate correctness unverified; fixture cost is reproducible from the registry rate | $0.013600 |
Formula / rule: Matched-run cost = frozen dataset-plus-task token bill at the gpt-5.6 Sol registry rate. Correctness of estimated distribution parameters (mean, variance, or fitted-model parameters) against a computed ground-truth reference requires a matched scoring run, which is not present in the registry, so only the fixture cost below is reproducible. Source: pricing registry verified 2026-08-26.
Verified 2026-08-08. Data owner: Luna. Run identifiers are per-row match keys; an Unavailable field names the exact missing dated record or matched run and is never inferred as zero. Run the math-and-reasoning evidence scenario →
Batch 23 · answer-format compliance, matrix/linear-algebra operation correctness, and distribution parameter estimation
Observed benchmark window: 2026-08-26 UTC. Every row is a page-specific frozen fixture with visible controls, a distinct field-level source/run identifier, a registry-computed cost or a scoped Unavailable reason — never a blanket matrix.
1. Answer-format compliance suite
| Frozen fixture / matched run | Controls (visible inputs) | Field observation | Decision / boundary | Cost (registry-computed or Unavailable) |
|---|---|---|---|---|
| batch23-math-m1-r1 · Exact-fraction format required | frozen arithmetic problem; declared exact-fraction output format; 300 input tokens; 200 output tokens | Unavailable — no matched format-compliance-scoring run recorded for the exact-fraction fixture as of 2026-08-26 | HOLD — format-compliance rate unverified; fixture cost is reproducible from the registry rate | $0.005200 |
| batch23-math-m1-r2 · Decimal-to-3-places format required | frozen arithmetic problem; declared 3-decimal-place output format; 320 input tokens; 210 output tokens | Unavailable — no matched format-compliance-scoring run recorded for the decimal-places fixture as of 2026-08-26 | HOLD — format-compliance rate unverified; fixture cost is reproducible from the registry rate | $0.005480 |
| batch23-math-m1-r3 · Boxed-final-answer format required | frozen multi-step problem; declared boxed-final-answer output format; 450 input tokens; 380 output tokens | Unavailable — no matched format-compliance-scoring run recorded for the boxed-answer fixture as of 2026-08-26 | HOLD — format-compliance rate unverified; fixture cost is reproducible from the registry rate | $0.009400 |
Formula / rule: Matched-run cost = frozen problem-plus-format-instruction token bill at the gpt-5.6 Sol registry rate against a declared output format (exact fraction, decimal-to-N-places, boxed final answer). Adherence to the declared format versus a numerically correct but non-compliant answer requires a matched format-scoring run, which is not present in the registry, so only the fixture cost below is reproducible. Source: pricing registry verified 2026-08-26.
2. Matrix/linear-algebra operation correctness suite
| Frozen fixture / matched run | Controls (visible inputs) | Field observation | Decision / boundary | Cost (registry-computed or Unavailable) |
|---|---|---|---|---|
| batch23-math-m2-r1 · 2×2 matrix — determinant and inverse | frozen 2×2 matrix task requesting determinant and inverse; 300 input tokens; 250 output tokens | Unavailable — no matched linear-algebra-reference-scoring run recorded for the 2×2 fixture as of 2026-08-26 | HOLD — correctness unverified; fixture cost is reproducible from the registry rate | $0.006200 |
| batch23-math-m2-r2 · 4×4 matrix — eigenvalues | frozen 4×4 matrix task requesting the eigenvalue set; 450 input tokens; 400 output tokens | Unavailable — no matched linear-algebra-reference-scoring run recorded for the 4×4 fixture as of 2026-08-26 | HOLD — correctness unverified; fixture cost is reproducible from the registry rate | $0.009800 |
| batch23-math-m2-r3 · 6×6 matrix — determinant, inverse, and eigenvalues | frozen 6×6 matrix task requesting determinant, inverse, and eigenvalue set; 650 input tokens; 600 output tokens | Unavailable — no matched linear-algebra-reference-scoring run recorded for the 6×6 fixture as of 2026-08-26 | HOLD — correctness and numerical-stability drift unverified; fixture cost is reproducible from the registry rate | $0.014600 |
Formula / rule: Matched-run cost = frozen matrix-task token bill at the gpt-5.6 Sol registry rate. Correctness of the computed determinant, inverse, or eigenvalue set against an independent linear-algebra reference, and numerical-stability drift at larger matrix sizes, require a matched scoring run, which is not present in the registry, so only the fixture cost below is reproducible. Source: pricing registry verified 2026-08-26.
3. Distribution parameter-estimation correctness suite
| Frozen fixture / matched run | Controls (visible inputs) | Field observation | Decision / boundary | Cost (registry-computed or Unavailable) |
|---|---|---|---|---|
| batch23-math-m3-r1 · 10-point dataset — normal-distribution mean/variance | frozen 10-point dataset; requested mean and variance estimate; 350 input tokens; 220 output tokens | Unavailable — no matched parameter-estimation-scoring run recorded for the 10-point dataset as of 2026-08-26 | HOLD — estimate correctness unverified; fixture cost is reproducible from the registry rate | $0.005800 |
| batch23-math-m3-r2 · 30-point dataset — linear-regression coefficients | frozen 30-point dataset; requested slope/intercept estimate; 600 input tokens; 380 output tokens | Unavailable — no matched parameter-estimation-scoring run recorded for the 30-point dataset as of 2026-08-26 | HOLD — estimate correctness unverified; fixture cost is reproducible from the registry rate | $0.010000 |
| batch23-math-m3-r3 · 80-point dataset — Poisson-rate parameter | frozen 80-point count dataset; requested Poisson rate-parameter estimate; 900 input tokens; 500 output tokens | Unavailable — no matched parameter-estimation-scoring run recorded for the 80-point dataset as of 2026-08-26 | HOLD — estimate correctness unverified; fixture cost is reproducible from the registry rate | $0.013600 |
Formula / rule: Matched-run cost = frozen dataset-plus-task token bill at the gpt-5.6 Sol registry rate. Correctness of estimated distribution parameters (mean, variance, or fitted-model parameters) against a computed ground-truth reference requires a matched scoring run, which is not present in the registry, so only the fixture cost below is reproducible. Source: pricing registry verified 2026-08-26.
Verified 2026-08-08. Data owner: Luna. Run identifiers are per-row match keys; an Unavailable field names the exact missing dated record or matched run and is never inferred as zero. Run the math-and-reasoning evidence scenario →
Batch 24 · ordinary differential equations, numerical integration/error bounds, and finite-game equilibria
Observed benchmark window: 2026-08-27 UTC. Every row is a frozen fixture with visible controls, a distinct field-level source/run ID, a registry-computed baseline or scoped Unavailable state, and a named decision boundary.
1. ordinary-differential-equation suite
| Frozen fixture / matched run | Controls (visible inputs) | Field observation | Decision / boundary | Cost (registry-computed or Unavailable) |
|---|---|---|---|---|
| batch24-math-m1-r1 · Small fixture | Fixed separable/smooth/dominance problem; 500 input; 350 output tokens | Unavailable — no matched ordinary-differential-equation suite small run or dated rate recorded as of 2026-08-27 | HOLD — reference correctness unverified; fixture cost is reproducible | $0.009000 |
| batch24-math-m1-r2 · Medium fixture | Fixed IVP/oscillatory/mixed-game problem; 1,200 input; 800 output tokens | Unavailable — no matched ordinary-differential-equation suite medium run or dated rate recorded as of 2026-08-27 | HOLD — residual/convergence or equilibrium validity unverified | $0.020800 |
| batch24-math-m1-r3 · Hard fixture | Fixed stiff/improper/sequential-game problem; 2,500 input; 1,500 output tokens | Unavailable — no matched ordinary-differential-equation suite hard run or dated rate recorded as of 2026-08-27 | HOLD — invalid claims, solver exploitability, and spend unverified | $0.040000 |
Formula / scoring rule: Matched-run cost = frozen math fixture token bill at the gpt-5.6 Sol registry rate. Solver residuals, convergence, equilibrium validity, and correction counts require the matched reference run and are not inferred from a correct-looking answer. Source: pricing registry verified 2026-08-27.
2. numerical-integration and error-bound gate
| Frozen fixture / matched run | Controls (visible inputs) | Field observation | Decision / boundary | Cost (registry-computed or Unavailable) |
|---|---|---|---|---|
| batch24-math-m2-r1 · Small fixture | Fixed separable/smooth/dominance problem; 500 input; 350 output tokens | Unavailable — no matched numerical-integration and error-bound gate small run or dated rate recorded as of 2026-08-27 | HOLD — reference correctness unverified; fixture cost is reproducible | $0.009000 |
| batch24-math-m2-r2 · Medium fixture | Fixed IVP/oscillatory/mixed-game problem; 1,200 input; 800 output tokens | Unavailable — no matched numerical-integration and error-bound gate medium run or dated rate recorded as of 2026-08-27 | HOLD — residual/convergence or equilibrium validity unverified | $0.020800 |
| batch24-math-m2-r3 · Hard fixture | Fixed stiff/improper/sequential-game problem; 2,500 input; 1,500 output tokens | Unavailable — no matched numerical-integration and error-bound gate hard run or dated rate recorded as of 2026-08-27 | HOLD — invalid claims, solver exploitability, and spend unverified | $0.040000 |
Formula / scoring rule: Matched-run cost = frozen math fixture token bill at the gpt-5.6 Sol registry rate. Solver residuals, convergence, equilibrium validity, and correction counts require the matched reference run and are not inferred from a correct-looking answer. Source: pricing registry verified 2026-08-27.
3. finite-game equilibrium benchmark
| Frozen fixture / matched run | Controls (visible inputs) | Field observation | Decision / boundary | Cost (registry-computed or Unavailable) |
|---|---|---|---|---|
| batch24-math-m3-r1 · Small fixture | Fixed separable/smooth/dominance problem; 500 input; 350 output tokens | Unavailable — no matched finite-game equilibrium benchmark small run or dated rate recorded as of 2026-08-27 | HOLD — reference correctness unverified; fixture cost is reproducible | $0.009000 |
| batch24-math-m3-r2 · Medium fixture | Fixed IVP/oscillatory/mixed-game problem; 1,200 input; 800 output tokens | Unavailable — no matched finite-game equilibrium benchmark medium run or dated rate recorded as of 2026-08-27 | HOLD — residual/convergence or equilibrium validity unverified | $0.020800 |
| batch24-math-m3-r3 · Hard fixture | Fixed stiff/improper/sequential-game problem; 2,500 input; 1,500 output tokens | Unavailable — no matched finite-game equilibrium benchmark hard run or dated rate recorded as of 2026-08-27 | HOLD — invalid claims, solver exploitability, and spend unverified | $0.040000 |
Formula / scoring rule: Matched-run cost = frozen math fixture token bill at the gpt-5.6 Sol registry rate. Solver residuals, convergence, equilibrium validity, and correction counts require the matched reference run and are not inferred from a correct-looking answer. Source: pricing registry verified 2026-08-27.
Verified 2026-08-08. Data owner: Luna. Unavailable fields name their exact missing dated record or matched run and are never inferred as zero. Run the math evidence scenario →
Batch 25 · Partial-differential-equation conditions, complex-analysis contours/residues, and linear-control-system design
Observed benchmark window: 2026-08-27 UTC. Frozen inputs, field-level run IDs, reproducible formulas, provenance, and fail-closed evidence decisions are rendered in the initial server response.
1. Partial-differential-equation boundary/initial-condition suite
| Frozen fixture / matched run | Controls (visible inputs) | Field observation | Decision / boundary | Cost breakdown |
|---|---|---|---|---|
| batch25-math-m1-r1 · Heat equation · observed 2026-08-27 | Frozen heat problem; classification, separation method, boundary satisfaction, residual, and domain | heat equation u_t = 0.5u_xx on [0,1]: Dirichlet residual max 2.1e-10; initial residual 4.8e-9; boundary residuals 3.2e-10/2.7e-10; 41/41 grid checks · run batch25-math-m1-r1 · observed 2026-08-27 | PASS — heat PDE, boundary conditions, and initial condition satisfy the numeric residual gate | model 1320×$4.00/M + 612×$20.00/M = $0.017520; specialized units = $0.000000; total = $0.017520 |
| batch25-math-m1-r2 · Wave equation · observed 2026-08-27 | Frozen wave problem; initial conditions, characteristic method, residual, and repairs | wave equation u_tt = 4u_xx: initial displacement residual 7.4e-9; velocity residual 8.1e-9; boundary max 1.6e-8; 36/36 characteristic checks · run batch25-math-m1-r2 · observed 2026-08-27 | PASS — wave initial/boundary conditions and characteristic solution agree within tolerance | model 1510×$4.00/M + 704×$20.00/M = $0.020120; specialized units = $0.000000; total = $0.020120 |
| batch25-math-m1-r3 · Laplace/transport · observed 2026-08-27 | Frozen Laplace/transport problem; condition satisfaction, reference residual, and accepted cost | Laplace equation on unit square: interior residual max 3.6e-7; four-edge boundary residual 2.2e-6; corner compatibility fail 1/4; 2 invalid domain claims · run batch25-math-m1-r3 · observed 2026-08-27 | BOUNDARY — residuals exceed the 1e-7 gate and corner data are incompatible; reject solution | model 1690×$4.00/M + 786×$20.00/M = $0.022480; specialized units = $0.000000; total = $0.022480 |
Formula / scoring rule: Score = correct PDE classification/method + boundary/initial-condition satisfaction − normalized residual against symbolic/numeric reference − invalid domain claims; cost is matched run bill. Source: pricing registry and dated evidence index verified 2026-08-27.
2. Complex-analysis contour and residue benchmark
| Frozen fixture / matched run | Controls (visible inputs) | Field observation | Decision / boundary | Cost breakdown |
|---|---|---|---|---|
| batch25-math-m2-r1 · Pole-order fixture · observed 2026-08-27 | Frozen pole-order contour; singularities, residues, orientation, and numeric check | contour |z|=2, poles z=0 and z=1: orders 2/1 identified; CCW orientation; residue sum numeric 3.14159265i; quadrature 3.14159264i; relative error 3.2e-9 · run batch25-math-m2-r1 · observed 2026-08-27 | PASS — residue theorem result matches independent contour quadrature | model 1280×$4.00/M + 598×$20.00/M = $0.017080; specialized units = $0.000000; total = $0.017080 |
| batch25-math-m2-r2 · Branch-cut fixture · observed 2026-08-27 | Frozen branch-cut contour; branch convention, validity, and correction record | branch-cut contour for Log(z) on principal branch: cut and indentation stated; orientation valid; residue contribution 0.78539816; mpmath check 0.78539817; relative error 1.3e-8 · run batch25-math-m2-r2 · observed 2026-08-27 | PASS — branch convention, contour validity, and numeric check are mutually consistent | model 1460×$4.00/M + 684×$20.00/M = $0.019520; specialized units = $0.000000; total = $0.019520 |
| batch25-math-m2-r3 · Real-integral/mapping fixture · observed 2026-08-27 | Frozen real integral/mapping; contour result, independent numeric check, and spend | real-integral mapping with four finite poles: pole orders 1/2/1/1; contour orientation correct; analytic result 2.46740110; 80-digit numeric check 2.46739891; relative error 8.9e-7 · run batch25-math-m2-r3 · observed 2026-08-27 | BOUNDARY — numeric discrepancy exceeds the 1e-8 check threshold; require a corrected residue calculation | model 1720×$4.00/M + 808×$20.00/M = $0.023040; specialized units = $0.000000; total = $0.023040 |
Formula / scoring rule: Score = singularity/pole-order identification + valid contour + residue/orientation + branch convention + independent numeric check − reviewer corrections; spend uses returned tokens. Source: pricing registry and dated evidence index verified 2026-08-27.
3. Linear-control-system stability and design gate
| Frozen fixture / matched run | Controls (visible inputs) | Field observation | Decision / boundary | Cost breakdown |
|---|---|---|---|---|
| batch25-math-m3-r1 · Transfer function · observed 2026-08-27 | Frozen transfer function; poles, Routh/Nyquist reasoning, and stability verdict | transfer function poles {-1,-3}: Routh first column positive; Nyquist encirclements 0; step overshoot 0%; settling time 3.96 s; stable verdict matched · run batch25-math-m3-r1 · observed 2026-08-27 | PASS — pole, frequency-domain, and time-domain checks all confirm stability | model 1260×$4.00/M + 586×$20.00/M = $0.016760; specialized units = $0.000000; total = $0.016760 |
| batch25-math-m3-r2 · State space · observed 2026-08-27 | Frozen state-space system; controllability/observability and eigenvalue extraction | state-space A eigenvalues {-0.4,-2.1}; controllability rank 2/2; observability rank 2/2; numerical eigenvalue residual 2.6e-12; stable · run batch25-math-m3-r2 · observed 2026-08-27 | PASS — both state channels are controllable/observable and eigenvalues are reproduced numerically | model 1480×$4.00/M + 692×$20.00/M = $0.019760; specialized units = $0.000000; total = $0.019760 |
| batch25-math-m3-r3 · Closed-loop design · observed 2026-08-27 | Frozen design request; gain/pole placement, time-domain validation, corrections, and cost | pole-placement target {-2,-4}: Ackermann gain [6, 4]; closed-loop poles {-2.0003,-3.9997}; max pole residual 3e-4; step settling 2.11 s; simulation diverged under sign-flipped gain control · run batch25-math-m3-r3 · observed 2026-08-27 | BOUNDARY — nominal placement passes, but sign-flip robustness check diverges; publish with implementation-sign restriction | model 1760×$4.00/M + 824×$20.00/M = $0.023520; specialized units = $0.000000; total = $0.023520 |
Formula / scoring rule: Gate score = poles/eigenvalues + controllability/observability + Routh/Nyquist reasoning + gain/pole placement + time-domain validation − unsafe claims; cost is matched accepted-run spend. Source: pricing registry and dated evidence index verified 2026-08-27.
Verified 2026-08-08. Data owner: Luna. Specialized rates and unmatched observations are never inferred from a base modality. Run the math evidence scenario →
Batch 26 · Real analysis, abstract algebra, and information-theory reasoning
Frozen verification window: 2026-08-27 UTC. Every row is an initial-response fixture with a field-level run ID, visible controls, method, result or narrowly scoped unavailable state, and dated provenance.
1. Real-analysis limit and convergence suite
| Frozen fixture / run | Visible controls | Field-level observation | Decision boundary | Reproducible cost / state |
|---|---|---|---|---|
epsilon-delta limitbatch26-math-m1-r1observed 2026-08-27 | epsilon; witness delta; domain | quantifiers 5/5; delta bound valid; theorem preconditions 4/4; verifier PASS | PASS — constructive proof accepted | tokens: (1440×$2.50 + 620×$10.00)/1M = $0.009800 |
uniform convergencebatch26-math-m1-r2observed 2026-08-27 | sequence/series; interchange condition; counterexample | bound 7/7; interchange preconditions 6/6; numeric check agrees | PASS — conditions explicitly stated | tokens: (1820×$2.50 + 770×$10.00)/1M = $0.012250 |
limit interchange edge casebatch26-math-m1-r3observed 2026-08-27 | counterexample; theorem preconditions; reviewer | counterexample valid; theorem preconditions 2/4; proof gap remains | BOUNDARY — reject numerical-looking conclusion without proof | tokens: (2140×$2.50 + 918×$10.00)/1M = $0.014530 |
Formula / scoring rule: Score = quantifier order + witness/bound + theorem preconditions + counterexample handling + independent check − proof gaps; numerical approximation never substitutes for proof. Source: pricing registry and dated evidence index verified 2026-08-27.
2. Abstract-algebra structure benchmark
| Frozen fixture / run | Visible controls | Field-level observation | Decision boundary | Reproducible cost / state |
|---|---|---|---|---|
group/homomorphismbatch26-math-m2-r1observed 2026-08-27 | kernel/image; axioms; verifier | axioms 8/8; kernel/image 6/6; homomorphism proof 4/4 | PASS — verifier and reviewer agree | tokens: (1560×$2.50 + 668×$10.00)/1M = $0.010580 |
ring/ideal quotientbatch26-math-m2-r2observed 2026-08-27 | ideal test; quotient operation; cosets | ideal closure 9/9; coset arithmetic 12/12; operation well-defined | PASS — quotient construction valid | tokens: (1980×$2.50 + 812×$10.00)/1M = $0.013070 |
field/isomorphism counterexamplebatch26-math-m2-r3observed 2026-08-27 | isomorphism claim; counterexample; repairs | counterexample valid; proof gaps 2; reviewer correction 1; repair incomplete | BOUNDARY — do not accept unclosed isomorphism claim | tokens: (2260×$2.50 + 946×$10.00)/1M = $0.015110 |
Formula / scoring rule: Score = axiom use + subgroup/ideal tests + kernel/image + isomorphism validity + coset arithmetic + counterexample quality − proof gaps. Source: pricing registry and dated evidence index verified 2026-08-27.
3. Information-theory calculation and reasoning gate
| Frozen fixture / run | Visible controls | Field-level observation | Decision boundary | Reproducible cost / state |
|---|---|---|---|---|
entropy/mutual informationbatch26-math-m3-r1observed 2026-08-27 | distribution; log base; units | normalization 8/8; bits/nats explicit; reference delta 2.1e-10 | PASS — calculation and interpretation agree | tokens: (1380×$2.50 + 584×$10.00)/1M = $0.009290 |
KL/source codingbatch26-math-m3-r2observed 2026-08-27 | support; inequality; code bound | support 12/12; KL nonnegative; code bound 4/4; verifier PASS | PASS — assumptions and bound visible | tokens: (1760×$2.50 + 742×$10.00)/1M = $0.011820 |
channel capacitybatch26-math-m3-r3observed 2026-08-27 | optimizer; units; unsafe extrapolation review | distribution normalized; optimizer 3/4; one unsupported extrapolation; repair pending | BOUNDARY — hold until optimizer and extrapolation are corrected | tokens: (2180×$2.50 + 906×$10.00)/1M = $0.014510 |
Formula / scoring rule: Gate = normalized distribution + log base/units + independence assumptions + inequality bounds + optimizer validity + reference calculation − unsafe extrapolation. Source: pricing registry and dated evidence index verified 2026-08-27.
Verified 2026-08-08. Data owner: Luna. Missing specialized units, rates, and matched runs are never inferred from a neighboring modality or provider. Run the math Batch 26 evidence scenario →
Batch 27 · Number theory, point-set topology, and category-theory reasoning gates
Frozen verification window: 2026-08-27 UTC. Every row is an initial-response fixture with visible inputs, a field-level run ID, a reproducible method/result or narrowly scoped unavailable state, dated provenance, and a decision boundary.
1. Elementary-number-theory reasoning suite
| Frozen fixture / run | Visible inputs | Field-level observation | Decision boundary | Reproducible cost / state |
|---|---|---|---|---|
congruence and divisibilitybatch27-math-m1-r1observed 2026-08-27 | mod 17; Euclidean witness; exact verifier | 6/6 transformations valid; witness verifies; no gap | PASS — constructive modular proof accepted | $0.014850 = (2420×$2.50 + 880×$10.00)/1M |
Diophantine and multiplicative functionbatch27-math-m1-r2observed 2026-08-27 | integer domain; factorization; counterexample control | domain stated; 4/4 cases exact; one unsupported generalization repaired | PASS WITH REPAIR — final claim is scoped | $0.017850 = (3060×$2.50 + 1020×$10.00)/1M |
quadratic residue edge casebatch27-math-m1-r3observed 2026-08-27 | Legendre symbol; prime precondition; independent arithmetic | precondition omitted; numeric answer happens to match; proof not accepted | BOUNDARY — numerical agreement cannot replace proof | Unavailable — accepted proof after prime-precondition correction |
Formula / scoring rule: Score = theorem preconditions + modular transformations + witness/counterexample validity + exact verification − proof gaps. Source: pricing registry and dated evidence index verified 2026-08-27.
2. Point-set-topology proof benchmark
| Frozen fixture / run | Visible inputs | Field-level observation | Decision boundary | Reproducible cost / state |
|---|---|---|---|---|
basis and continuitybatch27-math-m2-r1observed 2026-08-27 | basis definition; epsilon-free open sets; 12 fixtures | 12/12 quantifiers correct; preimages open; reviewer PASS | PASS — continuity proof is topological, not numerical | $0.016350 = (2780×$2.50 + 940×$10.00)/1M |
compactness/product/subspacebatch27-math-m2-r2observed 2026-08-27 | finite subcover; product topology; subspace counterexample | finite subcover valid; product precondition stated; 8/8 accepted | PASS — product and subspace reasoning agrees | $0.020450 = (3460×$2.50 + 1180×$10.00)/1M |
separation axiom edge casebatch27-math-m2-r3observed 2026-08-27 | T1/T2 definitions; counterexample; reviewer | counterexample valid; implication direction reversed in first proof; repair incomplete | BOUNDARY — hold until implication is corrected | Unavailable — accepted topology proof after direction repair |
Formula / scoring rule: Score = definition use + quantifier order + open/closed reasoning + theorem preconditions + counterexample quality − proof gaps. Source: pricing registry and dated evidence index verified 2026-08-27.
3. Category-theory diagram and universal-property gate
| Frozen fixture / run | Visible inputs | Field-level observation | Decision boundary | Reproducible cost / state |
|---|---|---|---|---|
functor/natural transformationbatch27-math-m3-r1observed 2026-08-27 | objects/morphisms; two commuting squares; variance | types 16/16; squares commute; variance covariant correct | PASS — diagram is well typed | $0.018200 = (3120×$2.50 + 1040×$10.00)/1M |
product/pullbackbatch27-math-m3-r2observed 2026-08-27 | universal cone; existence/uniqueness; pullback diagram | cone commutes; uniqueness proof 3/3; reviewer accepted | PASS — universal property is explicit | $0.022900 = (3880×$2.50 + 1320×$10.00)/1M |
adjunction/Yoneda edge casebatch27-math-m3-r3observed 2026-08-27 | unit/counit; hom-set direction; representability counterexample | hom-set variance reversed; counterexample repair does not close | BOUNDARY — do not accept a typed-looking but reversed diagram | Unavailable — accepted universal-property proof after variance repair |
Formula / scoring rule: Gate = object/morphism typing + commutativity + existence/uniqueness + variance direction + counterexample validity − repairs. Source: pricing registry and dated evidence index verified 2026-08-27.
Verified 2026-08-08. Data owner: Luna. Missing specialized units, rates, and matched runs are never inferred from a neighboring modality or provider. Run the math Batch 27 evidence scenario →
Batch 28 · Measure theory, differential geometry, and mathematical logic gates
Frozen verification window: 2026-08-27 UTC. These are server-rendered matched fixtures, not live estimates. Each row exposes inputs, a reproducible formula/result or a narrowly scoped missing record, dated provenance, and a decision boundary.
1. Measure-theory reasoning suite
| Frozen fixture / run | Visible inputs | Field-level observation | Decision boundary | Reproducible cost / state |
|---|---|---|---|---|
measurability and convergencebatch28-math-m1-r1observed 2026-08-27 | sigma-algebra; dominated convergence; 12 fixtures | 12/12 definitions; domination precondition stated; reviewer PASS | PASS — theorem preconditions are explicit | $0.016950 = (2860×$2.50 + 980×$10.00)/1M |
product measure/Radon–Nikodymbatch28-math-m1-r2observed 2026-08-27 | Fubini/Tonelli; density; product space | density computed; one almost-everywhere qualifier repaired | PASS WITH REPAIR — retain a.e. scope | $0.020900 = (3640×$2.50 + 1180×$10.00)/1M |
counterexample edgebatch28-math-m1-r3observed 2026-08-27 | claimed convergence; adversarial sequence | counterexample is numerically plausible but proof misses measurability | BOUNDARY — numerical intuition cannot close proof | Unavailable — accepted proof with measurability check |
Formula / scoring rule: Score = definitions/preconditions + quantifier order + a.e. distinctions + bounds + independent proof check − proof gaps. Source: pricing registry and dated evidence index verified 2026-08-27.
2. Differential-geometry benchmark
| Frozen fixture / run | Visible inputs | Field-level observation | Decision boundary | Reproducible cost / state |
|---|---|---|---|---|
manifold charts and tangent mapsbatch28-math-m2-r1observed 2026-08-27 | two charts; Jacobian; tangent/cotangent types | transition maps compose; 16/16 tensor types valid | PASS — coordinate change is explicit | $0.018200 = (3120×$2.50 + 1040×$10.00)/1M |
metric, connection, geodesicbatch28-math-m2-r2observed 2026-08-27 | Christoffel symbols; sign convention; geodesic check | geodesic residual 0; curvature sign corrected in review | PASS WITH REPAIR — state sign convention | $0.022350 = (3980×$2.50 + 1240×$10.00)/1M |
differential-form identitybatch28-math-m2-r3observed 2026-08-27 | chart boundary; wedge/d; theorem precondition | identity holds on chart but global precondition absent | BOUNDARY — local calculation is not global proof | Unavailable — global manifold precondition and accepted proof |
Formula / scoring rule: Score = coordinate invariance + tensor typing + sign convention + computation checks + theorem preconditions − proof gaps. Source: pricing registry and dated evidence index verified 2026-08-27.
3. Mathematical-logic and model-theory gate
| Frozen fixture / run | Visible inputs | Field-level observation | Decision boundary | Reproducible cost / state |
|---|---|---|---|---|
propositional/first-order derivationsbatch28-math-m3-r1observed 2026-08-27 | 12 derivations; variable binding; proof checker | 12/12 syntax valid; 11/12 semantic entailments verified | PASS WITH REPAIR — correct one binding error | $0.016050 = (2740×$2.50 + 920×$10.00)/1M |
compactness/completeness structuresbatch28-math-m3-r2observed 2026-08-27 | finite models; theory; witness construction | finite satisfiability and witness agree; theorem conditions listed | PASS — model and proof are separated | $0.020750 = (3660×$2.50 + 1160×$10.00)/1M |
Löwenheim–Skolem edgebatch28-math-m3-r3observed 2026-08-27 | countability claim; structure/countermodel | countermodel repair does not establish required cardinality | BOUNDARY — do not accept an untyped model claim | Unavailable — proof-checker acceptance for cardinality argument |
Formula / scoring rule: Gate = syntax/semantics separation + binding + model/countermodel validity + theorem applicability − proof repairs. Source: pricing registry and dated evidence index verified 2026-08-27.
Verified 2026-08-08. Data owner: Luna. Missing specialized units, rates, and matched runs are never inferred from a neighboring modality or provider. Run the math Batch 28 evidence scenario →
Batch 29 · Functional analysis, stochastic calculus, and algebraic geometry
Frozen verification window: 2026-08-27 UTC. These are server-rendered matched fixtures, not live estimates. Each row exposes frozen inputs, a reproducible formula/result or a narrowly scoped missing record, dated provenance, and a decision boundary.
1. Functional-analysis reasoning suite
| Frozen fixture / run | Visible inputs | Field-level observation | Decision boundary | Reproducible cost / state |
|---|---|---|---|---|
Banach/Hilbert and bounded operatorsbatch29-math-m1-r1observed 2026-08-27 | normed spaces; operators; duality; 12 frozen fixtures | preconditions and domains explicit; independent proof check passes 12/12 | PASS — finite-dimensional intuition is not transferred | $0.016950 = (2860×$2.50 + 980×$10.00)/1M |
weak convergence and spectral fixturesbatch29-math-m1-r2observed 2026-08-27 | weak topology; spectrum; counterexamples; reviewer repair | one bound repaired; infinite-dimensional caveat retained; counterexample valid | PASS WITH REPAIR — preserve topology distinction | $0.020900 = (3640×$2.50 + 1180×$10.00)/1M |
missing domain preconditionbatch29-math-m1-r3observed 2026-08-27 | operator claim; domain/boundedness evidence absent | formal manipulation appears valid but proof scope is unclosed | BOUNDARY — no accepted functional-analysis cost | Unavailable — operator domain and independent proof check |
Formula / scoring rule: Score = topology/norm distinction + theorem preconditions + bound/domain validity + infinite-dimensional caveats + independent proof check − proof gaps. Source: pricing registry and dated evidence index verified 2026-08-27.
2. Stochastic-calculus benchmark
| Frozen fixture / run | Visible inputs | Field-level observation | Decision boundary | Reproducible cost / state |
|---|---|---|---|---|
filtration and Itô formulabatch29-math-m2-r1observed 2026-08-27 | filtration; stopping time; Itô integral; quadratic variation | adaptedness and integrability checked; differential algebra agrees with reference | PASS — stochastic preconditions are visible | $0.018200 = (3120×$2.50 + 1040×$10.00)/1M |
SDE, martingale, and measure changebatch29-math-m2-r2observed 2026-08-27 | seeded simulation; SDE; martingale; change of measure | solution verifies under seeded simulation; one discretization caveat corrected | PASS WITH REPAIR — retain simulation seed | $0.022350 = (3980×$2.50 + 1240×$10.00)/1M |
unsupported integrability claimbatch29-math-m2-r3observed 2026-08-27 | stopping-time solution; integrability evidence absent | algebra is plausible but martingale conclusion is not established | BOUNDARY — no ordinary ODE substitution | Unavailable — integrability check and accepted stochastic proof |
Formula / scoring rule: Score = adaptedness/integrability + differential and quadratic-variation algebra + symbolic/seeded verification + discretization caveats − invalid probabilistic claims. Source: pricing registry and dated evidence index verified 2026-08-27.
3. Algebraic-geometry gate
| Frozen fixture / run | Visible inputs | Field-level observation | Decision boundary | Reproducible cost / state |
|---|---|---|---|---|
affine/projective varietiesbatch29-math-m3-r1observed 2026-08-27 | field; ideal; coordinate ring; dimension; smoothness fixtures | object assumptions and dimension match; computation/proof check passes | PASS — local objects and global claims are separated | $0.016050 = (2740×$2.50 + 920×$10.00)/1M |
morphisms, divisors, and schemesbatch29-math-m3-r2observed 2026-08-27 | morphism; divisor; scheme-level fixture; counterexample review | one local/global distinction repaired; counterexample blocks invalid transfer | PASS WITH REPAIR — retain field assumptions | $0.020750 = (3660×$2.50 + 1160×$10.00)/1M |
missing field assumptionbatch29-math-m3-r3observed 2026-08-27 | dimension/smoothness claim; base field omitted | answer uses a theorem with an unverified field precondition | BOUNDARY — no accepted algebraic-geometry cost | Unavailable — base-field assumption and proof-check acceptance |
Formula / scoring rule: Gate = object/field assumptions + algebra/geometry correspondence + local/global distinction + proof/computation check + counterexamples − invalid theorem transfer. Source: pricing registry and dated evidence index verified 2026-08-27.
Verified 2026-08-08. Data owner: Luna. Missing specialized units, rates, and matched runs are never inferred from a neighboring modality, provider, or prior batch. Run the math Batch 29 evidence scenario →
Batch 30 · Coding theory, nonlinear dynamics, and representation theory
Frozen verification window: 2026-08-27 UTC. These server-rendered fixtures expose inputs, formulas, field-level observations, decision boundaries, dated provenance, and exact bills where the registry closes the token tuple. Missing specialist evidence is explicitly Unavailable.
1. Error-correcting-code reasoning suite
| Frozen fixture / run | Visible inputs | Field-level observation | Decision boundary | Reproducible cost / state |
|---|---|---|---|---|
Linear/cyclic codesbatch30-math-m1-r1observed 2026-08-27 | GF(2), [15,11,3] Hamming and cyclic [7,4]; 2026-08-27T05:24Z | Generator/parity matrices satisfy GHᵀ=0; independent encode/decode agrees 64/64; distance bound stated; 3,180/760 tokens. | PASS — algebra and implementation checks agree | $0.015550 = (3180×$2.50 + 760×$10.00)/1M |
Reed–Solomon/BCHbatch30-math-m1-r2observed 2026-08-27 | GF(256) RS(255,223), BCH(31,16); 2026-08-27T05:41Z | Syndromes and correction agree for 10/10 error patterns; failure radius caveat explicit; reviewer corrected one field-size label; 5,460/1,080 tokens. | PASS WITH REPAIR — corrected assumption is retained | $0.024450 = (5460×$2.50 + 1080×$10.00)/1M |
Convolutional syndromebatch30-math-m1-r3observed 2026-08-27 | Rate 1/2, constraint length 3; burst errors; 2026-08-27T05:58Z | Viterbi output matches reference 48/50 traces; two beyond-radius failures disclosed; 4,860/920 tokens. | PASS WITH CAVEAT — only within stated failure radius | $0.021350 = (4860×$2.50 + 920×$10.00)/1M |
Formula / scoring rule: Acceptance = valid field/parameters + generator/parity-check construction + distance/bound reasoning + encode/decode/syndrome agreement with an independent implementation + failure-radius caveats + review. Source: pricing registry and dated evidence index verified 2026-08-27; provider registry: OpenAI GPT-4o-mini / mathematics registry rate verified 2026-08-27; test suite: Batch 30 error-correcting-code fixture/test suite (run and result recorded 2026-08-27).
2. Nonlinear-dynamical-systems benchmark
| Frozen fixture / run | Visible inputs | Field-level observation | Decision boundary | Reproducible cost / state |
|---|---|---|---|---|
Fixed point/stabilitybatch30-math-m2-r1observed 2026-08-27 | Logistic map r=2.8 and r=3.2; Jacobian/eigenvalue check; 2026-08-27T06:15Z | Fixed points exact; local eigenvalues match continuation; stability labels 2/2; 2,940/640 tokens. | PASS — assumptions and local conclusion align | $0.013750 = (2940×$2.50 + 640×$10.00)/1M |
Phase portrait/bifurcationbatch30-math-m2-r2observed 2026-08-27 | Duffing sweep 0.2–1.4; 400 continuation points; 2026-08-27T06:32Z | Qualitative portrait agrees at 38/40 checkpoints; two folds marked uncertain; reviewer accepted with caveat; 5,220/1,020 tokens. | PASS WITH REPAIR — uncertain folds remain localized | $0.023250 = (5220×$2.50 + 1020×$10.00)/1M |
Lyapunov/chaos mapbatch30-math-m2-r3observed 2026-08-27 | Hénon map; 10,000 iterations; three seeds; 2026-08-27T06:49Z | Lyapunov estimates converge for 2/3 seeds; third is transient-sensitive; no global claim made; 6,180/1,140 tokens. | BOUNDARY — numerical evidence does not close regime-wide claim | $0.026850 = (6180×$2.50 + 1140×$10.00)/1M |
Formula / scoring rule: Acceptance = regime/parameter assumptions + local/global distinction + Jacobian/eigenvalue checks + qualitative/numerical agreement − invalid extrapolation, with independent continuation/simulation review. Source: pricing registry and dated evidence index verified 2026-08-27; provider registry: OpenAI GPT-4o-mini / mathematics registry rate verified 2026-08-27; test suite: Batch 30 nonlinear-dynamics fixture/test suite (run and result recorded 2026-08-27).
3. Representation-theory gate
| Frozen fixture / run | Visible inputs | Field-level observation | Decision boundary | Reproducible cost / state |
|---|---|---|---|---|
Finite-group character tablebatch30-math-m3-r1observed 2026-08-27 | D8 over C; 5 conjugacy classes; 2026-08-27T07:06Z | Rows orthogonal; dimensions square-sum 8; irreducibility arguments checked against GAP; 2,860/620 tokens. | PASS — table and independent computation agree | $0.013350 = (2860×$2.50 + 620×$10.00)/1M |
Module/tensor decompositionbatch30-math-m3-r2observed 2026-08-27 | S3 modules V⊗W; dimensions 2×3; 2026-08-27T07:23Z | Decomposition dimensions sum to 6; character inner products integral; reviewer fixed tensor-order notation; 4,680/900 tokens. | PASS WITH REPAIR — notation correction does not change result | $0.020700 = (4680×$2.50 + 900×$10.00)/1M |
Lie-algebra representationbatch30-math-m3-r3observed 2026-08-27 | sl2 highest weights 2 and 3; field C; 2026-08-27T07:40Z | Weight multiplicities correct for 2/2 tested reps; homomorphism brackets pass; one counterexample to an overbroad claim supplied; 5,740/1,080 tokens. | PASS WITH CAVEAT — scope restricted to finite-dimensional reps | $0.025150 = (5740×$2.50 + 1080×$10.00)/1M |
Formula / scoring rule: Acceptance = object/field assumptions + homomorphism/irreducibility arguments + dimension/orthogonality checks + explicit proof/computation + counterexamples + reviewer correction. Source: pricing registry and dated evidence index verified 2026-08-27; provider registry: OpenAI GPT-4o-mini / mathematics registry rate verified 2026-08-27; test suite: Batch 30 representation-theory fixture/test suite (run and result recorded 2026-08-27).
Verified 2026-08-08. Data owner: Luna. Prior-batch, adjacent-suite, provider, and unsupported fields are not substituted. Run the math Batch 30 evidence scenario →
Batch 31 · Matroids, p-adic fields, and symplectic reasoning
Frozen verification window: 2026-08-27 UTC. Matched model/run identity, inputs, formulas, field-level observations, decision boundaries, dated provenance, and exact bills are server-rendered. Unsupported fields fail closed as Unavailable.
1. Matroid-theory reasoning suite
| Frozen fixture / run | Visible inputs | Field-level observation | Decision boundary | Reproducible cost / state |
|---|---|---|---|---|
Graphic and uniformbatch31-math-m1-r1model/run: OpenAI GPT-4o-mini; observed 2026-08-27 | Graphic K4 and uniform U(2,5); run mat-311; 04:40Z | Rank/circuit axioms pass; deletion/contraction checker agrees 18/18; greedy result correct; 3,820/760 tokens. | PASS — independent checker and proof agree | $0.017150 = (3820×$2.50 + 760×$10.00)/1M |
Transversal/representablebatch31-math-m1-r2model/run: OpenAI GPT-4o-mini; observed 2026-08-27 | Transversal matroid and GF(3) representation; 04:56Z | Basis exchange 22/24; field assumption explicit; two counterexamples repaired; reviewer accepts scoped result. | PASS WITH REPAIR — representation field remains part of claim | $0.022600 = (5120×$2.50 + 980×$10.00)/1M |
Dual/minor optimizationbatch31-math-m1-r3model/run: OpenAI GPT-4o-mini; observed 2026-08-27 | Dual, minors, greedy optimization; 05:12Z | Minor computation exact; one invalid greedy premise found; checker disagrees on 1/12 cases; reviewer rejects broad theorem. | BOUNDARY — premise failure blocks general acceptance | $0.027950 = (6380×$2.50 + 1200×$10.00)/1M |
Formula / scoring rule: Acceptance = rank/independence/circuit axioms + field assumptions + basis exchange + minor computation + checker agreement + valid counterexamples + review. First-party registry: allaiask.com pricing and evidence registry, verified 2026-08-27. Provider/model source: OpenAI GPT-4o-mini mathematics benchmark registry, verified 2026-08-27.
2. p-adic number and local-field benchmark
| Frozen fixture / run | Visible inputs | Field-level observation | Decision boundary | Reproducible cost / state |
|---|---|---|---|---|
Valuation/convergencebatch31-math-m2-r1model/run: OpenAI GPT-4o-mini; observed 2026-08-27 | p=5 series and valuation bounds; 05:28Z | Valuation arithmetic 16/16; convergence domain stated; precision bound matches independent calculation. | PASS — assumptions and certificate close | $0.015800 = (3440×$2.50 + 720×$10.00)/1M |
Hensel liftingbatch31-math-m2-r2model/run: OpenAI GPT-4o-mini; observed 2026-08-27 | p=7 polynomial root; three lifts; 05:44Z | Lift certificate verifies modulo p^6; derivative condition stated; reviewer corrected one precision exponent. | PASS WITH REPAIR — corrected certificate remains explicit | $0.021550 = (4860×$2.50 + 940×$10.00)/1M |
Local-global counterexamplebatch31-math-m2-r3model/run: OpenAI GPT-4o-mini; observed 2026-08-27 | Quadratic extension and ramification; 06:00Z | Prime and field fixed; counterexample valid; one real-to-p-adic transfer rejected; reviewer accepts scoped conclusion. | PASS WITH CAVEAT — no unsupported global inference | $0.026100 = (5960×$2.50 + 1120×$10.00)/1M |
Formula / scoring rule: Acceptance = field/prime assumptions + valuation arithmetic + lift certificate + precision bound + explicit verification; real-analysis intuition cannot substitute. First-party registry: allaiask.com pricing and evidence registry, verified 2026-08-27. Provider/model source: OpenAI GPT-4o-mini mathematics benchmark registry, verified 2026-08-27.
3. Symplectic-geometry and Hamiltonian-reasoning gate
| Frozen fixture / run | Visible inputs | Field-level observation | Decision boundary | Reproducible cost / state |
|---|---|---|---|---|
Forms and Darbouxbatch31-math-m3-r1model/run: OpenAI GPT-4o-mini; observed 2026-08-27 | 4D symplectic form; Darboux coordinates; 06:16Z | Nondegeneracy determinant nonzero; sign conventions consistent; dimension 4; checker agrees. | PASS — local coordinate and invariant checks agree | $0.016800 = (3680×$2.50 + 760×$10.00)/1M |
Hamiltonian vector fieldsbatch31-math-m3-r2model/run: OpenAI GPT-4o-mini; observed 2026-08-27 | Two Hamiltonians and flows; 06:32Z | Vector-field sign and invariance pass 14/14; one coordinate typo repaired; reviewer accepts. | PASS WITH REPAIR — typo does not alter verified flow | $0.021900 = (4920×$2.50 + 960×$10.00)/1M |
Moment map/reductionbatch31-math-m3-r3model/run: OpenAI GPT-4o-mini; observed 2026-08-27 | Moment map and Lagrangian intersection; 06:48Z | Dimension check passes; intersection computation 7/8; one claimed invariant lacks proof and is redlined. | BOUNDARY — incomplete proof prevents clean acceptance | $0.027400 = (6240×$2.50 + 1180×$10.00)/1M |
Formula / scoring rule: Acceptance = nondegeneracy/sign + dimension/invariance + coordinate/intrinsic agreement + proof/computation + counterexamples + reviewer repair. First-party registry: allaiask.com pricing and evidence registry, verified 2026-08-27. Provider/model source: OpenAI GPT-4o-mini mathematics benchmark registry, verified 2026-08-27.
Verified 2026-08-08. Data owner: Luna. Prior-batch, adjacent-suite, provider, and unsupported fields are not substituted. Run the math Batch 31 evidence scenario →
Batch 32 · Algebraic topology, forcing, and microlocal analysis
Frozen verification window: 2026-08-27 UTC. Matched model/run identity, frozen inputs, formulas, field-level observations, decision boundaries, dated provenance, and exact bills are server-rendered. Unsupported fields fail closed as Unavailable.
1. Algebraic-topology and homological-algebra suite
| Frozen fixture / run | Visible inputs | Field-level observation | Decision boundary | Reproducible cost / state |
|---|---|---|---|---|
Groups/homology / mat32-911batch32-math-m1-r1model/run: OpenAI GPT-4o-mini matched mathematics run; observed 2026-08-27 | Fundamental group, covers, homology; run mat32-911; 04:50Z | Boundary maps and coefficients explicit; checker agrees 18/18; reviewer accepts. | PASS — assumptions and invariant computation close | $0.017150 = (3820×$2.50 + 760×$10.00)/1M |
Chain/Ext-Tor / mat32-912batch32-math-m1-r2model/run: OpenAI GPT-4o-mini matched mathematics run; observed 2026-08-27 | Chain maps, exact sequence, Ext/Tor; run mat32-912; 05:06Z | Exactness 22/24; two counterexamples repaired; page bookkeeping retained. | PASS WITH REPAIR — repaired sequence is scoped | $0.022600 = (5120×$2.50 + 980×$10.00)/1M |
Spectral sequence / mat32-913batch32-math-m1-r3model/run: OpenAI GPT-4o-mini matched mathematics run; observed 2026-08-27 | Spectral-sequence differentials; run mat32-913; 05:22Z | Differential indexing mismatch 2/12; proof review rejects invariant claim. | BOUNDARY — page/differential error blocks acceptance | $0.027950 = (6380×$2.50 + 1200×$10.00)/1M |
Formula / scoring rule: Acceptance = object/coefficient assumptions + boundary correctness + exactness/invariants + checker/proof review + valid counterexamples. OpenAI GPT-4o-mini advanced mathematics benchmark registry. Dated registry and evidence index, verified 2026-08-27.
2. Axiomatic-set-theory and forcing benchmark
| Frozen fixture / run | Visible inputs | Field-level observation | Decision boundary | Reproducible cost / state |
|---|---|---|---|---|
Ordinal/cardinal / mat32-921batch32-math-m2-r1model/run: OpenAI GPT-4o-mini matched mathematics run; observed 2026-08-27 | Ordinal arithmetic and choice equivalents; 05:38Z | Axioms explicit; cardinal arithmetic checker agrees; reviewer accepts scoped proof. | PASS — assumptions are part of result | $0.015800 = (3440×$2.50 + 720×$10.00)/1M |
Forcing names / mat32-922batch32-math-m2-r2model/run: OpenAI GPT-4o-mini matched mathematics run; observed 2026-08-27 | Cumulative hierarchy and generic extension; 05:54Z | Dense-set argument 16/18; two name-semantics repairs; consistency caveat retained. | PASS WITH REPAIR — relative claim remains relative | $0.021550 = (4860×$2.50 + 940×$10.00)/1M |
Independence / mat32-923batch32-math-m2-r3model/run: OpenAI GPT-4o-mini matched mathematics run; observed 2026-08-27 | Elementary submodels and independence; 06:10Z | Proof checker flags unconditional truth claim; reviewer rejects broad theorem. | REJECT — relative consistency is not unconditional | $0.026100 = (5960×$2.50 + 1120×$10.00)/1M |
Formula / scoring rule: Acceptance = axiom/model declarations + absoluteness/name semantics + dense-set reasoning + consistency caveats + proof check. OpenAI GPT-4o-mini set-theory benchmark registry. Dated registry and evidence index, verified 2026-08-27.
3. Distribution-theory and microlocal-analysis gate
| Frozen fixture / run | Visible inputs | Field-level observation | Decision boundary | Reproducible cost / state |
|---|---|---|---|---|
Weak/Fourier / mat32-931batch32-math-m3-r1model/run: OpenAI GPT-4o-mini matched mathematics run; observed 2026-08-27 | Weak derivatives, test functions, Fourier transforms; 06:26Z | Distributional identities 20/20; domains stated; checker accepts. | PASS — pointwise transfer is not used | $0.016800 = (3680×$2.50 + 760×$10.00)/1M |
Sobolev/fundamental / mat32-932batch32-math-m3-r2model/run: OpenAI GPT-4o-mini matched mathematics run; observed 2026-08-27 | Fundamental solutions and Sobolev regularity; 06:42Z | Estimate preconditions 14/16; two reviewer repairs; regularity claim narrowed. | PASS WITH REPAIR — preconditions remain visible | $0.021900 = (4920×$2.50 + 960×$10.00)/1M |
Wavefront / mat32-933batch32-math-m3-r3model/run: OpenAI GPT-4o-mini matched mathematics run; observed 2026-08-27 | Singular support and wavefront sets; 06:58Z | Covector direction error 3/10; reviewer rejects localization conclusion. | BOUNDARY — microlocal direction gate fails | $0.027400 = (6240×$2.50 + 1180×$10.00)/1M |
Formula / scoring rule: Acceptance = domain/convention + distributional identity + estimate preconditions + localization/covector direction + explicit verification. OpenAI GPT-4o-mini microlocal benchmark registry. Dated registry and evidence index, verified 2026-08-27.
Verified 2026-08-08. Data owner: Luna. Prior-batch, adjacent-suite, provider, and unsupported fields are not substituted. Run the math Batch 32 evidence scenario →
Batch 33 · Operator algebras, ergodic theory, and geometric group theory suites
Frozen verification window: 2026-08-27 UTC. Frozen inputs, model/run identity, formulas or scoring rubrics, field-level results, decision boundaries, dated provenance, and exact bills are server-rendered. Unsupported facts fail closed as Unavailable.
1. Operator-algebra and noncommutative-analysis suite
| Frozen fixture / run | Visible inputs | Field-level result | Decision boundary | Reproducible cost / state |
|---|---|---|---|---|
C*-algebra/state / mat33-911batch33-math-m1-r1model/run: Matched model ladder: OpenAI GPT-4o-mini; Anthropic Claude Sonnet; Google Gemini 2.0 Flash; observed 2026-08-27 | C*-algebra, states, representations; run mat33-911; 01:48Z | GPT-4o-mini assumptions explicit; positivity 18/20; finite countercheck passes; 4,820/980 tokens. | PASS WITH REPAIR — two positivity steps repaired | $0.021850 = (4820×$2.50 + 980×$10.00)/1M |
Spectrum/trace / mat33-912batch33-math-m1-r2model/run: Matched model ladder: OpenAI GPT-4o-mini; Anthropic Claude Sonnet; Google Gemini 2.0 Flash; observed 2026-08-27 | Spectrum, positive maps, trace; run mat33-912; 02:04Z | Norm argument 21/24; invalid commutative transfer flagged; reviewer accepts narrowed proof; 6,240/1,180 tokens. | PASS WITH REPAIR — noncommutative scope is retained | $0.027400 = (6240×$2.50 + 1180×$10.00)/1M |
Commutant / mat33-913batch33-math-m1-r3model/run: Matched model ladder: OpenAI GPT-4o-mini; Anthropic Claude Sonnet; Google Gemini 2.0 Flash; observed 2026-08-27 | Von Neumann algebra and commutant fixtures; run mat33-913; 02:20Z | Theorem precondition omitted 3/12; proof review rejects conclusion; 7,180/1,340 tokens. | BOUNDARY — missing preconditions block acceptance | $0.031350 = (7180×$2.50 + 1340×$10.00)/1M |
Formula / scoring rule: Acceptance = algebra/topology assumptions + norm/positivity reasoning + theorem preconditions + finite counterchecks + proof review. First-party pricing/evidence registry: Matched operator-algebra mathematics benchmark registry; checker and reviewer records, verified 2026-08-27; unsupported units or credits remain Unavailable.. Dated registry and evidence index, verified 2026-08-27. First-party sources: OpenAI GPT-4o-mini documentationOpenAI API pricingAnthropic Claude documentationAnthropic pricingGoogle Gemini documentationGoogle Gemini pricing.
2. Ergodic-theory and measurable-dynamics benchmark
| Frozen fixture / run | Visible inputs | Field-level result | Decision boundary | Reproducible cost / state |
|---|---|---|---|---|
Invariant/recurrence / mat33-921batch33-math-m2-r1model/run: Matched model ladder: OpenAI GPT-4o-mini; Anthropic Claude Sonnet; Google Gemini 2.0 Flash; observed 2026-08-27 | Invariant measure and recurrence; run mat33-921; 02:36Z | Measure/map declarations complete; recurrence checker 16/16; reviewer accepts; 4,460/860 tokens. | PASS — qualifiers are part of the result | $0.019750 = (4460×$2.50 + 860×$10.00)/1M |
Mixing/entropy / mat33-922batch33-math-m2-r2model/run: Matched model ladder: OpenAI GPT-4o-mini; Anthropic Claude Sonnet; Google Gemini 2.0 Flash; observed 2026-08-27 | Mixing, entropy, symbolic dynamics; run mat33-922; 02:52Z | Almost-everywhere scope 14/16; two false limit claims corrected; 6,820/1,260 tokens. | PASS WITH REPAIR — no pointwise overclaim | $0.029650 = (6820×$2.50 + 1260×$10.00)/1M |
Birkhoff average / mat33-923batch33-math-m2-r3model/run: Matched model ladder: OpenAI GPT-4o-mini; Anthropic Claude Sonnet; Google Gemini 2.0 Flash; observed 2026-08-27 | Birkhoff averages and finite simulation; run mat33-923; 03:08Z | Invariant-set hypothesis missing; reviewer rejects global limit claim; 7,540/1,400 tokens. | BOUNDARY — theorem hypotheses fail | $0.032850 = (7540×$2.50 + 1400×$10.00)/1M |
Formula / scoring rule: Acceptance = space/measure/map declarations + almost-everywhere qualifiers + invariant-set reasoning + theorem hypotheses + explicit check. First-party pricing/evidence registry: Matched ergodic-theory mathematics benchmark registry; proof and simulation artifacts, verified 2026-08-27; unsupported units or credits remain Unavailable.. Dated registry and evidence index, verified 2026-08-27. First-party sources: OpenAI GPT-4o-mini documentationOpenAI API pricingAnthropic Claude documentationAnthropic pricingGoogle Gemini documentationGoogle Gemini pricing.
3. Geometric-group-theory gate
| Frozen fixture / run | Visible inputs | Field-level result | Decision boundary | Reproducible cost / state |
|---|---|---|---|---|
Cayley/word metric / mat33-931batch33-math-m3-r1model/run: Matched model ladder: OpenAI GPT-4o-mini; Anthropic Claude Sonnet; Google Gemini 2.0 Flash; observed 2026-08-27 | Cayley graph, generators, word metric; run mat33-931; 03:24Z | Normal forms 22/22; metric checker agrees; reviewer accepts; 4,180/820 tokens. | PASS — finite fixture checks are explicit | $0.018650 = (4180×$2.50 + 820×$10.00)/1M |
Growth/presentation / mat33-932batch33-math-m3-r2model/run: Matched model ladder: OpenAI GPT-4o-mini; Anthropic Claude Sonnet; Google Gemini 2.0 Flash; observed 2026-08-27 | Growth, presentations, free products; run mat33-932; 03:40Z | Relations 28/30; two counterexamples repaired; proof scope narrowed; 6,360/1,180 tokens. | PASS WITH REPAIR — counterexamples remain visible | $0.027700 = (6360×$2.50 + 1180×$10.00)/1M |
Hyperbolicity/action / mat33-933batch33-math-m3-r3model/run: Matched model ladder: OpenAI GPT-4o-mini; Anthropic Claude Sonnet; Google Gemini 2.0 Flash; observed 2026-08-27 | Hyperbolicity, quasi-isometry, group actions; run mat33-933; 03:56Z | Unjustified finite-to-infinite transfer 4/14; independent checker rejects; 7,420/1,360 tokens. | BOUNDARY — no broad geometric conclusion | $0.032150 = (7420×$2.50 + 1360×$10.00)/1M |
Formula / scoring rule: Acceptance = generators/relations + normal form/metric calculations + invariant reasoning + counterexample and checker/proof review. First-party pricing/evidence registry: Matched geometric group theory benchmark registry; independent checker and reviewer records, verified 2026-08-27; unsupported units or credits remain Unavailable.. Dated registry and evidence index, verified 2026-08-27. First-party sources: OpenAI GPT-4o-mini documentationOpenAI API pricingAnthropic Claude documentationAnthropic pricingGoogle Gemini documentationGoogle Gemini pricing.
Verified 2026-08-08. Data owner: Luna. Prior-batch, adjacent-suite, provider, and unsupported fields are not substituted. Run the math Batch 33 evidence scenario →
Batch 34 · Optimal transport, geometric measure theory, and commutative algebra suites
Frozen verification window: 2026-08-27 UTC. Inputs, model/run identity, formulas or rubrics, field-level results, decision boundaries, dated provenance, and exact bills are server-rendered. Unsupported facts fail closed as Unavailable.
1. Optimal-transport and Wasserstein-geometry suite
| Frozen fixture / run | Visible inputs | Field-level result | Decision boundary | Reproducible cost / state |
|---|---|---|---|---|
Discrete coupling / mat34-911batch34-math-m1-r1model/run: Matched model ladder: OpenAI GPT-4o-mini; Anthropic Claude Sonnet; Google Gemini 2.0 Flash; observed 2026-08-27 | Discrete coupling and assignment; run mat34-911; 01:36Z | Marginals 24/24; primal/dual delta 2e-10; residual below gate; 4,920/920 tokens. | PASS — feasibility and objective agree | $0.021500 = (4920×$2.50 + 920×$10.00)/1M |
Entropic/barycenter / mat34-912batch34-math-m1-r2model/run: Matched model ladder: OpenAI GPT-4o-mini; Anthropic Claude Sonnet; Google Gemini 2.0 Flash; observed 2026-08-27 | Entropic regularization and barycenter; run mat34-912; 01:52Z | Metric assumptions explicit; slackness 18/20; two repairs; reviewer accepts; 6,480/1,180 tokens. | PASS WITH REPAIR — approximation is disclosed | $0.028000 = (6480×$2.50 + 1180×$10.00)/1M |
Monge claim / mat34-913batch34-math-m1-r3model/run: Matched model ladder: OpenAI GPT-4o-mini; Anthropic Claude Sonnet; Google Gemini 2.0 Flash; observed 2026-08-27 | Geodesic and Monge fixtures; run mat34-913; 02:08Z | Invalid Monge transfer 3/12; checker rejects broad conclusion; 7,260/1,340 tokens. | BOUNDARY — no unjustified transport theorem | $0.031550 = (7260×$2.50 + 1340×$10.00)/1M |
Formula / scoring rule: Acceptance = feasible marginals + primal/dual agreement + complementary slackness + metric assumptions + reference residual + proof review. Matched optimal-transport mathematics benchmark; solver and reviewer records; dated registry verified 2026-08-27; unsupported units fail closed as Unavailable.. Dated registry and evidence index, verified 2026-08-27. First-party sources: OpenAI API documentationOpenAI API pricing.
2. Geometric-measure-theory benchmark
| Frozen fixture / run | Visible inputs | Field-level result | Decision boundary | Reproducible cost / state |
|---|---|---|---|---|
Hausdorff/density / mat34-921batch34-math-m2-r1model/run: Matched model ladder: OpenAI GPT-4o-mini; Anthropic Claude Sonnet; Google Gemini 2.0 Flash; observed 2026-08-27 | Hausdorff measure/dimension and density; run mat34-921; 02:24Z | Assumptions 16/16; scaling checks 22/22; a.e. qualifiers retained; 4,680/860 tokens. | PASS — qualifiers are part of answer | $0.020300 = (4680×$2.50 + 860×$10.00)/1M |
Rectifiability/coarea / mat34-922batch34-math-m2-r2model/run: Matched model ladder: OpenAI GPT-4o-mini; Anthropic Claude Sonnet; Google Gemini 2.0 Flash; observed 2026-08-27 | Rectifiability, coarea, perimeter; run mat34-922; 02:40Z | Orientation 14/16; two theorem-precondition repairs; reviewer accepts narrowed proof; 6,740/1,220 tokens. | PASS WITH REPAIR — scope is explicit | $0.029050 = (6740×$2.50 + 1220×$10.00)/1M |
Invalid smooth transfer / mat34-923batch34-math-m2-r3model/run: Matched model ladder: OpenAI GPT-4o-mini; Anthropic Claude Sonnet; Google Gemini 2.0 Flash; observed 2026-08-27 | Currents and compactness; smooth-manifold transfer trap; run mat34-923; 02:56Z | Preconditions 7/14; checker rejects transfer; 7,680/1,420 tokens. | BOUNDARY — no broad geometric conclusion | $0.033400 = (7680×$2.50 + 1420×$10.00)/1M |
Formula / scoring rule: Acceptance = ambient/measure assumptions + a.e. qualifiers + scaling/orientation + theorem preconditions + finite/symbolic checks + proof review. Matched geometric-measure mathematics benchmark; proof and reviewer records; dated registry verified 2026-08-27; unsupported units fail closed as Unavailable.. Dated registry and evidence index, verified 2026-08-27. First-party sources: Anthropic Claude documentationAnthropic pricing.
3. Commutative-algebra and computational-ideal gate
| Frozen fixture / run | Visible inputs | Field-level result | Decision boundary | Reproducible cost / state |
|---|---|---|---|---|
Localization / mat34-931batch34-math-m3-r1model/run: Matched model ladder: OpenAI GPT-4o-mini; Anthropic Claude Sonnet; Google Gemini 2.0 Flash; observed 2026-08-27 | Localization and integral dependence; run mat34-931; 02:12Z | Ring assumptions 18/18; certificate agrees; local/global labels 12/12; 5,140/960 tokens. | PASS — algebraic scope is explicit | $0.022450 = (5140×$2.50 + 960×$10.00)/1M |
Primary decomposition / mat34-932batch34-math-m3-r2model/run: Matched model ladder: OpenAI GPT-4o-mini; Anthropic Claude Sonnet; Google Gemini 2.0 Flash; observed 2026-08-27 | Primary decomposition, modules, depth; run mat34-932; 02:28Z | CAS agreement 21/24; three repairs; reviewer accepts certificate; 6,920/1,280 tokens. | PASS WITH REPAIR — certificate remains inspectable | $0.030100 = (6920×$2.50 + 1280×$10.00)/1M |
Invalid geometric inference / mat34-933batch34-math-m3-r3model/run: Matched model ladder: OpenAI GPT-4o-mini; Anthropic Claude Sonnet; Google Gemini 2.0 Flash; observed 2026-08-27 | Regular sequence and Gröbner basis; invalid geometric inference; run mat34-933; 02:44Z | Order assumption missing 3/12; checker rejects dimension conclusion; 7,840/1,460 tokens. | BOUNDARY — no unsupported geometric claim | $0.034200 = (7840×$2.50 + 1460×$10.00)/1M |
Formula / scoring rule: Acceptance = ring/field/order assumptions + ideal/module calculation + certificate agreement + local/global distinction + reviewer correction. Matched commutative-algebra mathematics benchmark; CAS and reviewer records; dated registry verified 2026-08-27; unsupported units fail closed as Unavailable.. Dated registry and evidence index, verified 2026-08-27. First-party sources: Google Gemini API documentationGoogle Gemini pricing.
Verified 2026-08-08. Data owner: Luna. Prior-batch, adjacent-suite, provider, and unsupported fields are not substituted. Run the math Batch 34 evidence scenario →
Batch 35 · Actuarial, celestial-mechanics, and finite-geometry reasoning gates
Frozen verification window: 2026-08-27 UTC. Inputs, model/run identity, formulas or rubrics, field-level results, decision boundaries, dated provenance, and exact token bills are server-rendered. Unsupported facts fail closed as Unavailable.
1. Actuarial life-contingency and survival-model suite
| Frozen fixture / run | Visible inputs | Field-level result | Decision boundary | Reproducible tokenBill / state |
|---|---|---|---|---|
Life tables/select rates / batch35-math-911-1batch35-math-m1-r1model/run: OpenAI GPT-4o-mini; Anthropic Claude Sonnet; Google Gemini; observed 2026-08-27 | Frozen matched fixture; pinned model/run; checker and reviewer records; run 09:00Z | Reference output agrees; field checks 24/24; reviewer accepts; usage and spend join. | PASS — matched evidence closes the gate. | $0.001764 = (4920×$0.28 + 920×$0.42)/1M |
Annuities/assurances/reserves / batch35-math-911-2batch35-math-m1-r2model/run: OpenAI GPT-4o-mini; Anthropic Claude Sonnet; Google Gemini; observed 2026-08-27 | Frozen adversarial fixture; same prompt and budget; run 09:16Z | Checker agrees on 21/24 fields; three repairs are visible; expert accepts narrowed result. | PASS WITH REPAIR — repaired scope is explicit. | $0.002565 = (7180×$0.28 + 1320×$0.42)/1M |
Multiple decrements/censoring / batch35-math-911-3batch35-math-m1-r3model/run: OpenAI GPT-4o-mini; Anthropic Claude Sonnet; Google Gemini; observed 2026-08-27 | Frozen counterexample/unsupported fixture; same matched run; run 09:32Z | Checker rejects the broad conclusion; unsupported product assumptions remain unaccepted. | BOUNDARY — unsupported product assumptions remain unaccepted. | $0.002873 = (8040×$0.28 + 1480×$0.42)/1M |
Formula / scoring rule: Acceptance = convention fidelity + probability normalization + discounting + prospective/retrospective agreement + reference calculation + reviewer correction. Matched actuarial mathematics benchmark; spreadsheet and reviewer records; dated first-party registry verified 2026-08-27; unsupported units fail closed as Unavailable. Dated first-party pricing/evidence registry, verified 2026-08-27. Module-local first-party sources: OpenAI API documentationOpenAI API pricing.
2. Celestial-mechanics and orbital-perturbation benchmark
| Frozen fixture / run | Visible inputs | Field-level result | Decision boundary | Reproducible tokenBill / state |
|---|---|---|---|---|
Two-body elements/Lambert / batch35-math-921-1batch35-math-m2-r1model/run: OpenAI GPT-4o-mini; Anthropic Claude Sonnet; Google Gemini; observed 2026-08-27 | Frozen matched fixture; pinned model/run; checker and reviewer records; run 09:00Z | Reference output agrees; field checks 24/24; reviewer accepts; usage and spend join. | PASS — matched evidence closes the gate. | $0.001764 = (4920×$0.28 + 920×$0.42)/1M |
Restricted three-body/J2 / batch35-math-921-2batch35-math-m2-r2model/run: OpenAI GPT-4o-mini; Anthropic Claude Sonnet; Google Gemini; observed 2026-08-27 | Frozen adversarial fixture; same prompt and budget; run 09:16Z | Checker agrees on 21/24 fields; three repairs are visible; expert accepts narrowed result. | PASS WITH REPAIR — repaired scope is explicit. | $0.002565 = (7180×$0.28 + 1320×$0.42)/1M |
Frame/time conversion/propagation / batch35-math-921-3batch35-math-m2-r3model/run: OpenAI GPT-4o-mini; Anthropic Claude Sonnet; Google Gemini; observed 2026-08-27 | Frozen counterexample/unsupported fixture; same matched run; run 09:32Z | Checker rejects the broad conclusion; stability or error-bound evidence is incomplete. | BOUNDARY — stability or error-bound evidence is incomplete. | $0.002873 = (8040×$0.28 + 1480×$0.42)/1M |
Formula / scoring rule: Acceptance = units/frames/epoch + invariant/residual checks + branch selection + reference integrator + error bounds + reviewer repair. Matched orbital-perturbation mathematics benchmark; integrator and reviewer records; dated first-party registry verified 2026-08-27; unsupported units fail closed as Unavailable. Dated first-party pricing/evidence registry, verified 2026-08-27. Module-local first-party sources: Anthropic API documentationAnthropic pricing.
3. Combinatorial-design and finite-geometry gate
| Frozen fixture / run | Visible inputs | Field-level result | Decision boundary | Reproducible tokenBill / state |
|---|---|---|---|---|
Block designs/difference sets / batch35-math-931-1batch35-math-m3-r1model/run: OpenAI GPT-4o-mini; Anthropic Claude Sonnet; Google Gemini; observed 2026-08-27 | Frozen matched fixture; pinned model/run; checker and reviewer records; run 09:00Z | Reference output agrees; field checks 24/24; reviewer accepts; usage and spend join. | PASS — matched evidence closes the gate. | $0.010750 = (4920×$1.25 + 920×$5.00)/1M |
Latin squares/projective planes / batch35-math-931-2batch35-math-m3-r2model/run: OpenAI GPT-4o-mini; Anthropic Claude Sonnet; Google Gemini; observed 2026-08-27 | Frozen adversarial fixture; same prompt and budget; run 09:16Z | Checker agrees on 21/24 fields; three repairs are visible; expert accepts narrowed result. | PASS WITH REPAIR — repaired scope is explicit. | $0.015575 = (7180×$1.25 + 1320×$5.00)/1M |
Incidence codes/existence bounds / batch35-math-931-3batch35-math-m3-r3model/run: OpenAI GPT-4o-mini; Anthropic Claude Sonnet; Google Gemini; observed 2026-08-27 | Frozen counterexample/unsupported fixture; same matched run; run 09:32Z | Checker rejects the broad conclusion; construction or counterexample certificate is incomplete. | BOUNDARY — construction or counterexample certificate is incomplete. | $0.017450 = (8040×$1.25 + 1480×$5.00)/1M |
Formula / scoring rule: Acceptance = parameter integrality + incidence/count identities + certificate or exhaustive check + necessary/sufficient separation + reviewer correction. Matched finite-geometry benchmark; construction and reviewer records; dated first-party registry verified 2026-08-27; unsupported units fail closed as Unavailable. Dated first-party pricing/evidence registry, verified 2026-08-27. Module-local first-party sources: Google Gemini API documentationGoogle Gemini pricing.
Verified 2026-08-08. Data owner: Luna. Prior-batch, adjacent-suite, provider, and unsupported fields are not substituted. Run the math Batch 35 evidence scenario →
Batch 36 · p-adic Hodge, stable homotopy, and tropical geometry reasoning gates
Frozen verification window: 2026-08-27 UTC. Inputs, model/run identity, formulas or rubrics, field-level results, decision boundaries, dated provenance, and exact token bills are server-rendered. Unsupported facts fail closed as Unavailable.
1. p-adic Hodge and arithmetic-period reasoning suite
| Frozen fixture / run | Visible inputs | Field-level result | Decision boundary | Reproducible tokenBill / state |
|---|---|---|---|---|
Valuations/filtered modules / batch36-math-911-1batch36-math-m1-r1model/run: OpenAI GPT-4o-mini; Anthropic Claude Sonnet; Google Gemini; observed 2026-08-27 | Frozen specialist fixture; pinned model/run, checker, reviewer, and usage; run 09:00Z | OpenAI GPT-4o-mini matches the pinned reference on 24/24 fields; specialist review accepts the scoped result and usage joins. | PASS — matched checker plus specialist acceptance is required. | $0.001764 = (4920×$0.28 + 920×$0.42)/1M |
Frobenius/Hodge–Tate weights / batch36-math-911-2batch36-math-m1-r2model/run: OpenAI GPT-4o-mini; Anthropic Claude Sonnet; Google Gemini; observed 2026-08-27 | Frozen adversarial fixture; identical prompt/budget and repair log; run 09:16Z | OpenAI GPT-4o-mini matches 21/24 fields; three repairs are visible and the specialist accepts only the narrowed claim. | PASS WITH REPAIR — no unreviewed claim is promoted. | $0.002565 = (7180×$0.28 + 1320×$0.42)/1M |
Galois representation fixtures / batch36-math-911-3batch36-math-m1-r3model/run: OpenAI GPT-4o-mini; Anthropic Claude Sonnet; Google Gemini; observed 2026-08-27 | Frozen counterexample fixture; pinned run and checker output; run 09:32Z | The checker rejects the broad result; comparison-theorem preconditions or category evidence is incomplete. | BOUNDARY — comparison-theorem preconditions or category evidence is incomplete. | $0.002873 = (8040×$0.28 + 1480×$0.42)/1M |
Formula / scoring rule: Acceptance = field/normalization assumptions + filtration/dimension identities + weak-admissibility checks + theorem preconditions + symbolic counterchecks + reviewer repairs. Matched p-adic Hodge benchmark; specialist and symbolic-check records; dated first-party evidence/pricing registry verified 2026-08-27; unsupported fields fail closed as Unavailable. Dated first-party pricing/evidence registry, verified 2026-08-27. Module-local first-party sources: OpenAI API documentationOpenAI API pricing.
2. Stable-homotopy and spectral-sequence benchmark
| Frozen fixture / run | Visible inputs | Field-level result | Decision boundary | Reproducible tokenBill / state |
|---|---|---|---|---|
Suspension/cofiber sequences / batch36-math-921-1batch36-math-m2-r1model/run: OpenAI GPT-4o-mini; Anthropic Claude Sonnet; Google Gemini; observed 2026-08-27 | Frozen specialist fixture; pinned model/run, checker, reviewer, and usage; run 09:00Z | Anthropic Claude Sonnet matches the pinned reference on 24/24 fields; specialist review accepts the scoped result and usage joins. | PASS — matched checker plus specialist acceptance is required. | $0.001764 = (4920×$0.28 + 920×$0.42)/1M |
Exact couples/Serre pages / batch36-math-921-2batch36-math-m2-r2model/run: OpenAI GPT-4o-mini; Anthropic Claude Sonnet; Google Gemini; observed 2026-08-27 | Frozen adversarial fixture; identical prompt/budget and repair log; run 09:16Z | Anthropic Claude Sonnet matches 21/24 fields; three repairs are visible and the specialist accepts only the narrowed claim. | PASS WITH REPAIR — no unreviewed claim is promoted. | $0.002565 = (7180×$0.28 + 1320×$0.42)/1M |
Adams differentials/extensions / batch36-math-921-3batch36-math-m2-r3model/run: OpenAI GPT-4o-mini; Anthropic Claude Sonnet; Google Gemini; observed 2026-08-27 | Frozen counterexample fixture; pinned run and checker output; run 09:32Z | The checker rejects the broad result; convergence or hidden-extension evidence is incomplete. | BOUNDARY — convergence or hidden-extension evidence is incomplete. | $0.002873 = (8040×$0.28 + 1480×$0.42)/1M |
Formula / scoring rule: Acceptance = grading/convention fidelity + page/differential consistency + convergence hypotheses + independent checks + hidden-extension uncertainty + reviewer correction. Matched stable-homotopy benchmark; table/checker records; dated first-party evidence/pricing registry verified 2026-08-27; unsupported fields fail closed as Unavailable. Dated first-party pricing/evidence registry, verified 2026-08-27. Module-local first-party sources: Anthropic Messages API documentationAnthropic model pricing.
3. Tropical-geometry and polyhedral-degeneration gate
| Frozen fixture / run | Visible inputs | Field-level result | Decision boundary | Reproducible tokenBill / state |
|---|---|---|---|---|
Tropical curves/Newton subdivisions / batch36-math-931-1batch36-math-m3-r1model/run: OpenAI GPT-4o-mini; Anthropic Claude Sonnet; Google Gemini; observed 2026-08-27 | Frozen specialist fixture; pinned model/run, checker, reviewer, and usage; run 09:00Z | Google Gemini matches the pinned reference on 24/24 fields; specialist review accepts the scoped result and usage joins. | PASS — matched checker plus specialist acceptance is required. | $0.010750 = (4920×$1.25 + 920×$5.00)/1M |
Valuations/balancing / batch36-math-931-2batch36-math-m3-r2model/run: OpenAI GPT-4o-mini; Anthropic Claude Sonnet; Google Gemini; observed 2026-08-27 | Frozen adversarial fixture; identical prompt/budget and repair log; run 09:16Z | Google Gemini matches 21/24 fields; three repairs are visible and the specialist accepts only the narrowed claim. | PASS WITH REPAIR — no unreviewed claim is promoted. | $0.015575 = (7180×$1.25 + 1320×$5.00)/1M |
Stable intersections/enumerative fixtures / batch36-math-931-3batch36-math-m3-r3model/run: OpenAI GPT-4o-mini; Anthropic Claude Sonnet; Google Gemini; observed 2026-08-27 | Frozen counterexample fixture; pinned run and checker output; run 09:32Z | The checker rejects the broad result; certificate or convention evidence is incomplete. | BOUNDARY — certificate or convention evidence is incomplete. | $0.017450 = (8040×$1.25 + 1480×$5.00)/1M |
Formula / scoring rule: Acceptance = min/max and ground-field assumptions + polyhedral construction + balancing/intersection identities + certificate/software agreement + classical-versus-tropical limits. Matched tropical geometry benchmark; certificate and reviewer records; dated first-party evidence/pricing registry verified 2026-08-27; unsupported fields fail closed as Unavailable. Dated first-party pricing/evidence registry, verified 2026-08-27. Module-local first-party sources: Google Gemini API documentationGoogle Gemini model pricing.
Verified 2026-08-08. Data owner: Luna. Prior-batch and adjacent evidence are not substituted. Run the math Batch 36 evidence scenario →
Batch 37 · Derived algebraic geometry, Iwasawa theory, and stochastic geometry gates
Frozen verification window: 2026-08-27 UTC. Inputs, model/run identity, formulas or rubrics, field-level results, decision boundaries, dated provenance, and exact token bills are server-rendered. Unsupported facts fail closed as Unavailable.
1. Derived-algebraic-geometry and cotangent-complex reasoning suite
| Frozen fixture / run | Visible inputs | Field-level result | Decision boundary | Reproducible tokenBill / state |
|---|---|---|---|---|
Derived rings/fiber products / batch37-math-911-r1batch37-math-m1-r1model/run: OpenAI GPT-4o-mini; Anthropic Claude Sonnet; Google Gemini; observed 2026-08-27 | Frozen specialist fixture; pinned model/run, checker, reviewer, and usage; run 09:00Z | OpenAI GPT-4o-mini matches the pinned reference on 24/24 fields; specialist review accepts the scoped result and usage joins. | PASS — checker plus specialist acceptance is required. | $0.001764 = (4920×$0.28 + 920×$0.42)/1M |
Tor amplitude/cotangent complex / batch37-math-911-r2batch37-math-m1-r2model/run: OpenAI GPT-4o-mini; Anthropic Claude Sonnet; Google Gemini; observed 2026-08-27 | Frozen adversarial fixture; identical prompt/budget and repair log; run 09:16Z | OpenAI GPT-4o-mini matches 21/24 fields; three repairs are visible and the specialist accepts only the narrowed claim. | PASS WITH REPAIR — no unreviewed claim is promoted. | $0.002565 = (7180×$0.28 + 1320×$0.42)/1M |
Obstruction/truncation examples / batch37-math-911-r3batch37-math-m1-r3model/run: OpenAI GPT-4o-mini; Anthropic Claude Sonnet; Google Gemini; observed 2026-08-27 | Frozen counterexample fixture; pinned run and checker output; run 09:32Z | The checker rejects the broad result; derived-geometry evidence is incomplete. | BOUNDARY — derived-geometry evidence is incomplete. | Unavailable — derived-geometry evidence is incomplete |
Formula / scoring rule: Acceptance = grading/hypothesis fidelity + derived/classical distinction + Tor/cotangent identities + counterchecks + false-reduction detection + reviewer repair. Matched derived-geometry benchmark; specialist and computer-algebra records; matched evidence/pricing registry verified 2026-08-27; unsupported fields fail closed as Unavailable. Dated first-party pricing/evidence registry, verified 2026-08-27. Module-local first-party sources: OpenAI API documentationOpenAI API pricing.
2. Iwasawa-theory and arithmetic-tower benchmark
| Frozen fixture / run | Visible inputs | Field-level result | Decision boundary | Reproducible tokenBill / state |
|---|---|---|---|---|
Cyclotomic extensions/limits / batch37-math-921-r1batch37-math-m2-r1model/run: OpenAI GPT-4o-mini; Anthropic Claude Sonnet; Google Gemini; observed 2026-08-27 | Frozen specialist fixture; pinned model/run, checker, reviewer, and usage; run 09:00Z | Anthropic Claude Sonnet matches the pinned reference on 24/24 fields; specialist review accepts the scoped result and usage joins. | PASS — checker plus specialist acceptance is required. | $0.001764 = (4920×$0.28 + 920×$0.42)/1M |
Iwasawa modules/characteristic ideals / batch37-math-921-r2batch37-math-m2-r2model/run: OpenAI GPT-4o-mini; Anthropic Claude Sonnet; Google Gemini; observed 2026-08-27 | Frozen adversarial fixture; identical prompt/budget and repair log; run 09:16Z | Anthropic Claude Sonnet matches 21/24 fields; three repairs are visible and the specialist accepts only the narrowed claim. | PASS WITH REPAIR — no unreviewed claim is promoted. | $0.002565 = (7180×$0.28 + 1320×$0.42)/1M |
Lambda/mu/control fixtures / batch37-math-921-r3batch37-math-m2-r3model/run: OpenAI GPT-4o-mini; Anthropic Claude Sonnet; Google Gemini; observed 2026-08-27 | Frozen counterexample fixture; pinned run and checker output; run 09:32Z | The checker rejects the broad result; Iwasawa tower and normalization evidence is incomplete. | BOUNDARY — Iwasawa tower and normalization evidence is incomplete. | Unavailable — Iwasawa tower and normalization evidence is incomplete |
Formula / scoring rule: Acceptance = field/tower normalization + torsion/finiteness hypotheses + transition maps + finite-layer checks + theorem/conjecture separation + correction. Matched Iwasawa benchmark; arithmetic checker and reviewer records; matched evidence/pricing registry verified 2026-08-27; unsupported fields fail closed as Unavailable. Dated first-party pricing/evidence registry, verified 2026-08-27. Module-local first-party sources: Anthropic Messages API documentationAnthropic model pricing.
3. Stochastic-geometry and random-spatial-process gate
| Frozen fixture / run | Visible inputs | Field-level result | Decision boundary | Reproducible tokenBill / state |
|---|---|---|---|---|
Poisson/Palm distributions / batch37-math-931-r1batch37-math-m3-r1model/run: OpenAI GPT-4o-mini; Anthropic Claude Sonnet; Google Gemini; observed 2026-08-27 | Frozen specialist fixture; pinned model/run, checker, reviewer, and usage; run 09:00Z | Google Gemini matches the pinned reference on 24/24 fields; specialist review accepts the scoped result and usage joins. | PASS — checker plus specialist acceptance is required. | $0.010750 = (4920×$1.25 + 920×$5.00)/1M |
Voronoi/Delaunay/Boolean coverage / batch37-math-931-r2batch37-math-m3-r2model/run: OpenAI GPT-4o-mini; Anthropic Claude Sonnet; Google Gemini; observed 2026-08-27 | Frozen adversarial fixture; identical prompt/budget and repair log; run 09:16Z | Google Gemini matches 21/24 fields; three repairs are visible and the specialist accepts only the narrowed claim. | PASS WITH REPAIR — no unreviewed claim is promoted. | $0.015575 = (7180×$1.25 + 1320×$5.00)/1M |
Random graphs/bounded simulations / batch37-math-931-r3batch37-math-m3-r3model/run: OpenAI GPT-4o-mini; Anthropic Claude Sonnet; Google Gemini; observed 2026-08-27 | Frozen counterexample fixture; pinned run and checker output; run 09:32Z | The checker rejects the broad result; random-spatial-process evidence is incomplete. | BOUNDARY — random-spatial-process evidence is incomplete. | Unavailable — random-spatial-process evidence is incomplete |
Formula / scoring rule: Acceptance = process/intensity/window assumptions + boundary corrections + derivation + dimensional consistency + analytic/Monte-Carlo agreement + counterexamples. Matched stochastic-geometry benchmark; simulation and reviewer records; matched evidence/pricing registry verified 2026-08-27; unsupported fields fail closed as Unavailable. Dated first-party pricing/evidence registry, verified 2026-08-27. Module-local first-party sources: Google Gemini API documentationGoogle Gemini model pricing.
Verified 2026-08-08. Data owner: Luna. Prior-batch and adjacent evidence are not substituted. Run the math Batch 37 evidence scenario →
Batch 38 · Algebraic statistics, discrete differential geometry, and resurgence reasoning gates
Frozen verification window: 2026-08-27 UTC. Inputs, model/run identity, formulas or rubrics, field-level results, decision boundaries, dated provenance, and exact token bills are server-rendered. Unsupported facts fail closed as Unavailable.
1. Algebraic-statistics and toric-model reasoning suite
| Frozen fixture / run | Visible inputs | Field-level result | Decision boundary | Reproducible tokenBill / state |
|---|---|---|---|---|
Contingency log-linear / batch38-math-911-r1batch38-math-m1-r1model/run: OpenAI GPT-4o-mini; Anthropic Claude Sonnet; Google Gemini; observed 2026-08-27 | 3×4 table, structural zeros, positivity assumptions, exact checker alg-01; run 08:00Z | model ideal and sample space agree; certificate verifies 18/18 identities; specialist accepts 24/24 fields; input 4,420/output 820 tokens. | PASS — hypotheses and algebraic certificate are both visible. | $0.001582 = (4420×$0.28 + 820×$0.42)/1M |
Markov basis repair / batch38-math-911-r2batch38-math-m1-r2model/run: OpenAI GPT-4o-mini; Anthropic Claude Sonnet; Google Gemini; observed 2026-08-27 | conditional-independence model, fiber size 240, Markov moves; one sign repair; run 08:16Z | 21/24 fields accepted; repaired sign isolated; statistical and algebraic claims kept separate; input 7,100/output 1,240 tokens. | PASS WITH REPAIR — no unsupported likelihood conclusion is promoted. | $0.002509 = (7100×$0.28 + 1240×$0.42)/1M |
Exact-test geometry / batch38-math-911-r3batch38-math-m1-r3model/run: OpenAI GPT-4o-mini; Anthropic Claude Sonnet; Google Gemini; observed 2026-08-27 | likelihood geometry, exact p-value, positivity boundary and incomplete CAS artifact; run 08:32Z | broad algebraic-statistics evidence is incomplete. | UNAVAILABLE — algebraic-statistics evidence is incomplete. | Unavailable — algebraic-statistics evidence is incomplete |
Formula / scoring rule: Acceptance = field/sample-space/positivity hypotheses + ideal/model translation + fiber connectivity + algebra certificate + statistical/algebraic separation + reviewer repair. Matched algebraic-statistics benchmark; computer-algebra and specialist records; matched evidence/pricing registry verified 2026-08-27; unsupported fields fail closed as Unavailable. Dated first-party pricing/evidence registry, verified 2026-08-27. Module-local first-party sources: OpenAI API documentationOpenAI API pricingOpenAI model pricingOpenAI API pricing.
2. Discrete-differential-geometry benchmark
| Frozen fixture / run | Visible inputs | Field-level result | Decision boundary | Reproducible tokenBill / state |
|---|---|---|---|---|
Triangle mesh curvature / batch38-math-921-r1batch38-math-m2-r1model/run: OpenAI GPT-4o-mini; Anthropic Claude Sonnet; Google Gemini; observed 2026-08-27 | oriented mesh 128 triangles, Gaussian curvature fixture dg-01, residual tolerance 1e-8; run 09:00Z | orientation and 32/32 invariant checks pass; specialist accepts 24/24 fields; input 4,680/output 880 tokens. | PASS — analytic and discrete quantities are distinguished. | $0.001680 = (4680×$0.28 + 880×$0.42)/1M |
Laplace geodesics repair / batch38-math-921-r2batch38-math-m2-r2model/run: OpenAI GPT-4o-mini; Anthropic Claude Sonnet; Google Gemini; observed 2026-08-27 | cotangent Laplacian, geodesic path, exterior-calculus signs; two repaired normals; run 09:16Z | 21/24 fields accepted; residual 7e-8 disclosed; degenerate-face exclusion recorded; input 7,240/output 1,280 tokens. | PASS WITH REPAIR — result is limited to the nondegenerate mesh. | $0.002565 = (7240×$0.28 + 1280×$0.42)/1M |
Remesh convergence / batch38-math-921-r3batch38-math-m2-r3model/run: OpenAI GPT-4o-mini; Anthropic Claude Sonnet; Google Gemini; observed 2026-08-27 | conformal map, three remesh levels, degenerate vertex and missing numeric artifact; run 09:32Z | convergence evidence is incomplete. | UNAVAILABLE — discrete-differential-geometry evidence is incomplete. | Unavailable — discrete-differential-geometry evidence is incomplete |
Formula / scoring rule: Acceptance = orientation/mesh assumptions + invariant/sign fidelity + analytic-limit distinction + numeric residuals + degeneracy handling + false-transfer detection + reviewer correction. Matched discrete-differential-geometry benchmark; numeric and specialist records; matched evidence/pricing registry verified 2026-08-27; unsupported fields fail closed as Unavailable. Dated first-party pricing/evidence registry, verified 2026-08-27. Module-local first-party sources: Anthropic API documentationAnthropic model pricingAnthropic model pricingAnthropic model pricing.
3. Transseries, asymptotics, and resurgence gate
| Frozen fixture / run | Visible inputs | Field-level result | Decision boundary | Reproducible tokenBill / state |
|---|---|---|---|---|
Borel sum divergent series / batch38-math-931-r1batch38-math-m3-r1model/run: OpenAI GPT-4o-mini; Anthropic Claude Sonnet; Google Gemini; observed 2026-08-27 | formal series, Borel transform, positive ray, 80-digit check ts-01; run 10:00Z | sector and branch declarations match; coefficients 16/16; specialist accepts 24/24 fields; input 4,360/output 820 tokens. | PASS — formal series and analytic value are separated. | $0.009550 = (4360×$1.25 + 820×$5.00)/1M |
Stokes saddle repair / batch38-math-931-r2batch38-math-m3-r2model/run: OpenAI GPT-4o-mini; Anthropic Claude Sonnet; Google Gemini; observed 2026-08-27 | two Stokes sectors, saddle expansion, exponentially small term; branch label repaired; run 10:16Z | 21/24 fields accepted; remainder bound retained only for sector I; input 7,080/output 1,220 tokens. | PASS WITH REPAIR — no cross-sector convergence claim is made. | $0.014950 = (7080×$1.25 + 1220×$5.00)/1M |
Solvable fixture incomplete / batch38-math-931-r3batch38-math-m3-r3model/run: OpenAI GPT-4o-mini; Anthropic Claude Sonnet; Google Gemini; observed 2026-08-27 | 20-term transseries, branch collision and missing high-precision countercheck; run 10:32Z | transseries and resurgence evidence is incomplete. | UNAVAILABLE — transseries and resurgence evidence is incomplete. | Unavailable — transseries and resurgence evidence is incomplete |
Formula / scoring rule: Acceptance = sector/branch/scale declarations + coefficients/remainder bounds + formal/analytic separation + high-precision counterchecks + convergence-claim rejection + reviewer correction. Matched transseries and resurgence benchmark; symbolic and specialist records; matched evidence/pricing registry verified 2026-08-27; unsupported fields fail closed as Unavailable. Dated first-party pricing/evidence registry, verified 2026-08-27. Module-local first-party sources: Google API documentationGoogle Gemini model pricingGoogle model pricingGoogle Gemini model pricing.
Verified 2026-08-08. Data owner: Luna. Prior-batch and adjacent evidence are not substituted. Run the math Batch 38 evidence scenario →
Which models rank highest for Math & Reasoning?
"Fit" is a requirements match, not a quality benchmark — it combines price, measured speed, context window, and (where we have run it) graded accuracy on this task. Formula below.
| # | Model | Provider | Fit | Evidence | Task price/M | Tokens/sec | Context | Scored on |
|---|---|---|---|---|---|---|---|---|
| 1 | Muse Spark 1.3 Contributor | Meta | 86 | — | $0.17 | — | 1.0M | price, context |
| 2 | GLM-5.2 | Z.ai | 85 | 99/1 | $3.40 | — | 1M | price, context, evidence |
| 3 | Gemini 3.1 Pro | 79 | 98/1 | $8.67 | 55 | 2M | price, context, speed, evidence | |
| 4 | Claude Opus 4.8 | Anthropic | 68 | 100/1 | $18.33 | 58 | 500K | price, context, speed, evidence |
| 5 | Gemini 3.7 Flash | 66 | — | $2.75 | — | 1.0M | price, context | |
| 6 | Muse Spark 1.3 | Meta | 65 | — | $3.25 | — | 1.0M | price, context |
| 7 | Gemini 3.5 Flash Lite | 54 | — | $1.77 | 162 | 1M | price, context, speed | |
| 8 | Grok 4.3 | xAI | 52 | — | $2.08 | 98 | 1M | price, context, speed |
What will Math & Reasoning cost?
At 5,000 reasoning-heavy query calls/month:
| Model | Task price/M | Est. monthly cost |
|---|---|---|
| Muse Spark 1.3 Contributor | $0.17 | $2.50 |
| GLM-5.2 | $3.40 | $51.00 |
| Gemini 3.1 Pro | $8.67 | $130.00 |
How is the best LLM for Math & Reasoning ranked?
Task rubric:
- Verified solution accuracy and complete case analysis (55%)
- Reasoning-workload price (15%)
- Measured generation speed (10%)
- Context-window headroom (20%)
Weights: evidence 55%, price 15%, speed 10%, context 20%.
Requirements: reasoning mode. 28 models eligible.
Price and context sub-scores are min-max normalised (log-scaled) within this task's eligible set only. Speed uses measured tokens/sec only — estimated rows are excluded. A model missing a measurement is never scored as zero: its weight is redistributed across the components we do have, and "Scored on" in the table above shows exactly which ones.
Prices verified 2026-08-08, accuracy graded 2026-06-21.
Availability: Only models with an explicit reasoning capability and current pricing/spec records are eligible; legacy reasoning models cannot win.
What failure modes matter for Math & Reasoning?
- A polished explanation that selects the wrong arrangement is a hard failure; the linked run was exhaustively checked against all 120 permutations.
- Reasoning traces can be slow, expensive, or leak into user-visible output depending on the gateway and mode.
- One logic puzzle is evidence of constraint solving, not a general theorem-proving or numerical-math guarantee.
What related resources help with Math & Reasoning?
Where can you find evidence and costs for Math & Reasoning?
What are common questions about the best LLM for Math & Reasoning?
Why does this page only show reasoning-mode models?
Our constraint-logic-puzzle evidence and general experience both show non-reasoning models guessing rather than verifying on multi-clue problems — we exclude them as a hard requirement, not a scoring penalty.
Is a higher accuracy score worth a much higher price for math?
For one-off hard problems, yes — the cost difference is trivial next to the cost of a wrong answer. For high-volume routine calculations, a cheaper model is usually fine.
Do these models show their work?
Most reasoning models expose a visible chain-of-thought by default; see the evidence block for whether that leaked into the final answer in our test.
