{"schemaVersion":2,"suite":"premium-model-tests","evidenceEndpoint":{"readOnly":true,"machineReadable":true},"runAt":"2026-06-21T00:00:00.000Z","repositorySource":"frontend/src/lib/premiumTests.ts","tests":[{"slug":"constraint-logic-puzzle","title":"Solving a Constraint Logic Puzzle","shortTitle":"Logic Puzzle","category":"Complex Reasoning","icon":"∴","summary":"A five-house constraint-satisfaction puzzle with five interlocking clues and a single valid solution. The model must reason through both cases, eliminate the dead end, and report the exact arrangement and count.","criteria":["Finds the one valid arrangement: White, Green, Red, Blue, Yellow (houses 1–5).","States the correct final count of 1.","Shows sound case analysis on both yellow-end positions and uses every clue to eliminate the dead end.","Gives a clear final answer with the count on its own line."],"groundTruth":"Exactly one arrangement satisfies all five clues — House 1: White, 2: Green, 3: Red, 4: Blue, 5: Yellow — so the count is 1 (verified by exhaustive search over all 120 permutations).","metaTitle":"Best Premium AI for Hard Logic & Reasoning — GPT-5.4 Pro vs Opus vs Gemini vs GLM","metaDescription":"We gave the same five-clue constraint puzzle to the top flagship models — GPT-5.4 Pro, Claude Opus 4.8, Gemini 3.1 Pro and GLM 5.2 (Max) — and measured real speed, cost and reasoning quality. See which premium model wins.","keywords":["best AI for reasoning","GPT-5.4 Pro vs Claude Opus","Gemini 3.1 Pro reasoning","GLM 5.2 max benchmark","premium LLM comparison 2026","best AI for logic puzzles","frontier model reasoning test"],"prompt":"Solve this logic puzzle by reasoning it through, then give a final answer.\n\nFive houses stand in a row, numbered 1 to 5 from left to right. Each house is painted a different color, one of: red, blue, green, yellow, white. The following clues all hold:\n1. The green house is immediately to the right of the white house.\n2. The red house is somewhere to the left of the blue house (not necessarily adjacent).\n3. The yellow house is at one of the two ends of the row.\n4. There are exactly two houses between the green house and the yellow house.\n5. The blue house is not at either end.\n\nHow many distinct color arrangements satisfy ALL of these clues? List every valid arrangement as house 1 through house 5, and then state the final count on its own line.","results":[{"modelId":"gpt-5.4-pro","modelName":"GPT-5.4 Pro","provider":"OpenAI","outputPricePerM":180,"output":"Let the houses be positions 1–5.\n\n- Since the **green house is immediately to the right of the white house**, the pair must be one of:\n  - (1,2) = white, green\n  - (2,3) = white, green\n  - (3,4) = white, green\n  - (4,5) = white, green\n\n- The **yellow house is at an end** (position 1 or 5), and there are **exactly two houses between green and yellow**, so green must be 3 positions away from yellow.\n\n## Case 1: Yellow at 1\nThen green must be at 4.  \nSo white must be at 3.\n\nArrangement so far:\n1. yellow\n2. ?\n3. white\n4. green\n5. ?\n\nThe remaining colors are red and blue.  \nSince **red is left of blue**, we must have:\n2. red\n5. blue\n\nBut **blue cannot be at either end**, and position 5 is an end.  \nSo this case is impossible.\n\n## Case 2: Yellow at 5\nThen green must be at 2.  \nSo white must be at 1.\n\nArrangement so far:\n1. white\n2. green\n3. ?\n4. ?\n5. yellow\n\nThe remaining colors are red and blue.  \nSince **red is left of blue**, we must have:\n3. red\n4. blue\n\nBlue is not at an end, so this works.\n\n## Valid arrangement\n1. white, 2. green, 3. red, 4. blue, 5. yellow\n\n1","status":"completed","finishReason":"","latencyMs":37818,"tokensIn":178,"tokensOut":822,"tokensPerSecond":21.7,"costUsd":0.1533,"accuracy":100,"accuracyNote":"Correct unique arrangement and count (1) with tight, rigorous case analysis on both yellow positions, final count on its own line. Now that the live API reports real usage for this model, its true cost is shown — by far the priciest run here.","receiptVersion":1,"promptHash":"119b0025774a31bd409ae9f1f387f96eaea0c8eb3f793dc0186496fecc3df336","rawOutputHash":"beea172063b8f2d3e67aa379ad19d2339907b45b4525c9897ca741db8b2e46b5","graderVersion":"claude-agent-grader-v1","solverVersion":"constraint-solver-v1-120-permutations","harnessVersion":null,"validationState":"recorded","validationFlags":["prompt-hash=sha256","raw-output-hash=sha256","status=completed","accuracy=range-valid","latency=recorded","tokens=recorded","cost=recorded","grader-score=recorded","reasoning-leak=absent"],"correctionVersion":"correction-v1"},{"modelId":"claude-opus-4-8","modelName":"Claude Opus 4.8","provider":"Anthropic","outputPricePerM":25,"output":"<think>\nHouses 1-5. Colors: red, blue, green, yellow, white.\n\nClue 1: green = white+1 (white immediately left of green).\nClue 2: red < blue.\nClue 3: yellow at position 1 or 5.\nClue 4: |green - yellow| = 3.\nClue 5: blue not at 1 or 5.\n\nLet me enumerate by yellow position.\n\nCase yellow=1:\nClue 4: green = 1±3 = 4 (since -2 invalid). So green=4.\nClue 1: white=3.\nRemaining: red, blue for positions 2,5.\nClue 5: blue not at 5, so blue=2, red=5. But clue 2: red<blue → 5<2 false. Invalid.\n\nSo no valid with yellow=1.\n\nCase yellow=5:\nClue 4: green = 5±3 = 2 or 8. green=2.\nClue 1: white=1.\nRemaining: red, blue for positions 3,4.\nClue 5: blue not at 1,5, ok both. \nClue 2: red<blue. Positions 3,4: red=3, blue=4. Valid.\nArrangement: 1 white, 2 green, 3 red, 4 blue, 5 yellow.\n\nCheck all clues: green(2)=white(1)+1 ✓. red(3)<blue(4) ✓. yellow=5 end ✓. |green-yellow|=|2-5|=3 ✓. blue=4 not end ✓.\n\nOnly one arrangement.\n</think>\n\n## Reasoning\n\n**Clue constraints:**\n- Green = White + 1 (adjacent)\n- Red < Blue\n- Yellow at position 1 or 5\n- |Green − Yellow| = 3\n- Blue at position 2, 3, or 4\n\n**Case: Yellow = 1**\n- Clue 4: Green = 4 (only valid option). So White = 3.\n- Remaining positions 2 and 5 get red and blue.\n- Clue 5 forbids blue at 5, so Blue = 2, Red = 5.\n- But Clue 2 needs Red < Blue → 5 < 2, false. ❌ No solution.\n\n**Case: Yellow = 5**\n- Clue 4: Green = 2 (only valid option). So White = 1.\n- Remaining positions 3 and 4 get red and blue.\n- Clue 2 (Red < Blue): Red = 3, Blue = 4.\n- Check Clue 5: Blue = 4, not an end. ✓\n\nThis gives:\n**House 1: White, House 2: Green, House 3: Red, House 4: Blue, House 5: Yellow**\n\nVerify all clues: Green(2)=White(1)+1 ✓; Red(3)<Blue(4) ✓; Yellow at end ✓; |2−5|=3 ✓; Blue not at end ✓.\n\n## Valid Arrangement(s)\n1. White, Green, Red, Blue, Yellow\n\n**1**","status":"completed","finishReason":"","latencyMs":10119,"tokensIn":246,"tokensOut":985,"tokensPerSecond":97.3,"costUsd":0.025855,"accuracy":100,"accuracyNote":"Correct arrangement and count via tight, fully-verified case analysis on both yellow positions, with a final five-clue check. The leading <think> block is Opus 4.8's reasoning channel surfaced by our gateway, not dumped into the answer — the final solution itself is clean and correct.","receiptVersion":1,"promptHash":"119b0025774a31bd409ae9f1f387f96eaea0c8eb3f793dc0186496fecc3df336","rawOutputHash":"cac6fe506285c705adbd84cb5a5d0e1560dbde69fe6ce791e76167c1ebd38fe5","graderVersion":"claude-agent-grader-v1","solverVersion":"constraint-solver-v1-120-permutations","harnessVersion":null,"validationState":"recorded","validationFlags":["prompt-hash=sha256","raw-output-hash=sha256","status=completed","accuracy=range-valid","latency=recorded","tokens=recorded","cost=recorded","grader-score=recorded","reasoning-leak=detected"],"correctionVersion":"correction-v1"},{"modelId":"gemini-3.1-pro","modelName":"Gemini 3.1 Pro","provider":"Google","outputPricePerM":12,"output":"To find the valid color arrangements, we can deduce the positions of the houses step by step (numbered 1 to 5 from left to right):\n\n1. **Analyze Yellow and Green's positions:**\nClue 3 states the yellow house is at one of the two ends (Position 1 or Position 5). \nClue 4 states there are exactly two houses between the green house and the yellow house. \n- *Case A:* If the yellow house is at Position 1, the green house must be at Position 4.\n- *Case B:* If the yellow house is at Position 5, the green house must be at Position 2.\n\n2. **Analyze White's position:**\nClue 1 states the green house is immediately to the right of the white house. This means the white house is exactly one spot to the left of the green house.\n- *Case A:* If green is at 4, white must be at 3. (Arrangement so far: 1:Yellow, 2:Empty, 3:White, 4:Green, 5:Empty)\n- *Case B:* If green is at 2, white must be at 1. (Arrangement so far: 1:White, 2:Green, 3:Empty, 4:Empty, 5:Yellow)\n\n3. **Analyze Red and Blue's positions:**\nThe remaining colors to place are red and blue. Clue 2 states the red house is somewhere to the left of the blue house.\n- *Case A:* The empty spots are 2 and 5. For red to be to the left of blue, red must be at 2 and blue must be at 5. However, Clue 5 states the blue house cannot be at either end. Since Position 5 is an end, **Case A is invalid.**\n- *Case B:* The empty spots are 3 and 4. For red to be to the left of blue, red must be at 3 and blue must be at 4. Let's check Clue 5: Blue is at Position 4, which is not an end. This works perfectly!\n\nThis leaves us with exactly one valid arrangement that satisfies all the clues.\n\n**Valid Arrangement:**\nHouse 1: White, House 2: Green, House 3: Red, House 4: Blue, House 5: Yellow\n\n1","status":"completed","finishReason":"","latencyMs":22828,"tokensIn":180,"tokensOut":518,"tokensPerSecond":22.7,"costUsd":0.006576,"accuracy":98,"accuracyNote":"Correct arrangement and count via clean step-by-step deduction. Slightly lighter on explicit verification than Opus, but fully sound and the cheapest correct answer here.","receiptVersion":1,"promptHash":"119b0025774a31bd409ae9f1f387f96eaea0c8eb3f793dc0186496fecc3df336","rawOutputHash":"598a199b193a2c86c727861f860be1aafa3c6b395b594c7a2e95c1863f4051dd","graderVersion":"claude-agent-grader-v1","solverVersion":"constraint-solver-v1-120-permutations","harnessVersion":null,"validationState":"recorded","validationFlags":["prompt-hash=sha256","raw-output-hash=sha256","status=completed","accuracy=range-valid","latency=recorded","tokens=recorded","cost=recorded","grader-score=recorded","reasoning-leak=absent"],"correctionVersion":"correction-v1"},{"modelId":"glm-5.2","modelName":"GLM 5.2 (Max)","provider":"Z.ai","outputPricePerM":4.4,"output":"<think>\nThe problem asks for the number of distinct color arrangements of 5 houses in a row, numbered 1 to 5 from left to right.\nThe colors are red, blue, green, yellow, white.\nClues:\n1. Green is immediately to the right of White. (W, G)\n2. Red is somewhere to the left of Blue.\n3. Yellow is at one of the two ends (house 1 or house 5).\n4. There are exactly two houses between Green and Yellow.\n5. Blue is not at either end (so Blue is in house 2, 3, or 4).\n\nLet's analyze the positions of Yellow and the W-G block.\nYellow is at 1 or 5.\nCase 1: Yellow is at 1.\nSince there are exactly two houses between Green and Yellow, Green must be at 4.\nIf Green is at 4, White must be at 3 (since Green is immediately to the right of White).\nSo we have:\n1: Yellow\n2: ?\n3: White\n4: Green\n5: ?\nThe remaining colors are Red and Blue.\nThey must go in houses 2 and 5.\nClue 5 says Blue is not at either end. So Blue cannot be at 5.\nThus, Blue must be at 2, and Red must be at 5.\nBut Clue 2 says Red is somewhere to the left of Blue.\nIf Blue is at 2 and Red is at 5, Red is to the right of Blue. This violates Clue 2.\nSo Case 1 yields no valid arrangements.\n\nCase 2: Yellow is at 5.\nSince there are exactly two houses between Green and Yellow, Green must be at 2.\nIf Green is at 2, White must be at 1 (since Green is immediately to the right of White).\nSo we have:\n1: White\n2: Green\n3: ?\n4: ?\n5: Yellow\nThe remaining colors are Red and Blue.\nThey must go in houses 3 and 4.\nClue 5 says Blue is not at either end. Houses 3 and 4 are not ends, so this is satisfied.\nClue 2 says Red is to the left of Blue.\nSo Red must be at 3, and Blue must be at 4.\nLet's check all clues for this arrangement:\n1: White\n2: Green\n3: Red\n4: Blue\n5: Yellow\n\nClue 1: Green is immediately to the right of White. (W at 1, G at 2) - Yes.\nClue 2: Red is to the left of Blue. (R at 3, B at 4) - Yes.\nClue 3: Yellow is at one of the two ends. (Y at 5) - Yes.\nClue 4: Exactly two houses between Green and Yellow. (G at 2, Y at 5, houses 3 and 4 are between) - Yes.\nClue 5: Blue is not at either end. (B at 4) - Yes.\n\nAre there any other possibilities?\nLet's double check.\nCould Yellow be at 1 and Green at 4? We checked this, it leads to W at 3, Y at 1. Remaining houses 2, 5 for R, B. B cannot be at 5, so B at 2, R at 5. But R must be left of B. Contradiction.\nCould Yellow be at 5 and Green at 2? We checked this, it leads to W at 1, Y at 5. Remaining houses 3, 4 for R, B. B cannot be at 5 (already satisfied). R must be left of B, so R at 3, B at 4. This works.\n\nWait, are there any other interpretations of \"exactly two houses between\"?\nIf Yellow is at 1, Green is at 4. Houses 2 and 3 are between. That's exactly two houses.\nIf Yellow is at 5, Green is at 2. Houses 3 and 4 are between. That's exactly two houses.\nAre there any other positions for Yellow? No, clue 3 says Yellow is at one of the two ends.\nAre there any other positions for Green? No, because the distance between 1 and 4 is 2 houses, and between 2 and 5 is 2 houses.\nSo Green must be at 2 or 4.\nIf Green is at 4, White is at 3.\nIf Green is at 2, White is at 1.\n\nLet's quickly review the steps.\nColors: R, B, G, Y, W.\nPositions: 1, 2, 3, 4, 5.\nY is at 1 or 5.\nIf Y=1, G=4. W=3. Remaining: 2, 5 for R, B.\nB cannot be at 5 (end), so B=2, R=5.\nBut R must be left of B. 5 is not left of 2. Invalid.\n\nIf Y=5, G=2. W=1. Remaining: 3, 4 for R, B.\nB cannot be at ends (3 and 4 are not ends, so both valid).\nR must be left of B, so R=3, B=4. Valid.\n\nIs there any ambiguity in \"The green house is immediately to the right of the white house\"?\nNo, it means W is at x, G is at x+1.\nIs there any ambiguity in \"exactly two houses between\"?\nNo, |pos(G) - pos(Y)| = 3.\nIs there any ambiguity in \"The red house is somewhere to the left of the blue house\"?\nNo, pos(R) < pos(B).\n\nLet's check if there's any other way.\nWhat if Yellow is at 1, Green is at 4, White is at 3.\nHouses: 1: Y, 2: ?, 3: W, 4: G, 5: ?\nColors left: R, B.\nIf B=2, R=5. R is not left of B.\nIf B=5, R=2. B is at an end, which violates clue 5.\nSo no solution for Y=1.\n\nWhat if Yellow is at 5, Green is at 2, White is at 1.\nHouses: 1: W, 2: G, 3: ?, 4: ?, 5: Y.\nColors left: R, B.\nIf R=3, B=4. R is left of B. B is not at an end. Valid.\nIf R=4, B=3. R is not left of B. Invalid.\n\nSo there is exactly 1 valid arrangement.\nThe arrangement is:\nHouse 1: White\nHouse 2: Green\nHouse 3: Red\nHouse 4: Blue\nHouse 5: Yellow\n\nLet me re-read the prompt to make sure I didn't miss anything.\n\"Five houses stand in a row, numbered 1 to 5 from left to right.\"\n\"Each house is painted a different color, one of: red, blue, green, yellow, white.\"\n\"1. The green house is immediately to the right of the white house.\" -> W, G\n\"2. The red house is somewhere to the left of the blue house (not necessarily adjacent).\" -> R < B\n\"3. The yellow house is at one of the two ends of the row.\" -> Y=1 or Y=5\n\"4. There are exactly two houses between the green house and the yellow house.\" -> |G - Y| = 3\n\"5. The blue house is not at either end.\" -> B != 1, B != 5\n\nEverything seems perfectly straightforward.\nLet's write down the solution clearly.\n- State the possible positions for Yellow based on Clue 3 (1 or 5).\n- Use Clue 4 to find the corresponding positions for Green (4 or 2).\n- Use Clue 1 to find the corresponding positions for White (3 or 1).\n- Evaluate the two cases:\n  - Case 1: Y=1, G=4, W=3. Remaining positions for R and B are 2 and 5. Clue 5 says B cannot be 5, so B=2, R=5. But Clue 2 says R must be left of B, which is violated. So this case is invalid.\n  - Case 2: Y=5, G=2, W=1. Remaining positions for R and B are 3 and 4. Clue 2 says R must be left of B, so R=3, B=4. Clue 5 is satisfied since B is not at an end. This case is valid.\n- Conclude there is exactly 1 valid arrangement.\n- List the arrangement as house 1 through house 5.\n- State the final count on its own line.\n\nThe arrangement:\nHouse 1: White\nHouse 2: Green\nHouse 3: Red\nHouse 4: Blue\nHouse 5: Yellow\n\nFinal count: 1\n\nLet me double check the \"exactly two houses between\" part.\nIf Y is at 1, and G is at 4.\nHouses between 1 and 4 are 2 and 3. That's exactly two houses.\nIf Y is at 5, and G is at 2.\nHouses between 2 and 5 are 3 and 4. That's exactly two houses.\nYes, this is correct.\n\nWhat if \"exactly two houses between\" means the distance is 2?\nNo, \"two houses between\" means there are two houses separating them. For example, between house 1 and house 4, there are houses 2 and 3. That's two houses. If it meant distance is 2, it would be \"one house between\". So |G - Y| = 3 is correct.\n\nLet's do a quick sanity check.\nColors: R, B, G, Y, W.\nArrangement: W, G, R, B, Y.\n1. Green immediately right of White: W at 1, G at 2. Yes.\n2. Red left of Blue: R at 3, B at 4. Yes.\n3. Yellow at an end: Y at 5. Yes.\n4. Exactly two houses between Green and Yellow: G at 2, Y at 5. Houses 3 and 4 are between. Yes.\n5. Blue not at either end: B at 4. Yes.\n\nEverything is consistent. The logic is solid. No other interpretations seem likely.\nI will structure the output to show the reasoning, list the valid arrangement, and then state the final count.\n</think>\nTo solve this logic puzzle, we can deduce the positions of the houses step-by-step using the given clues:\n\n**Step 1: Determine the possible positions for the Yellow and Green houses.**\n* Clue 3 states that the yellow house is at one of the two ends (house 1 or house 5).\n* Clue 4 states that there are exactly two houses between the green house and the yellow house. \nThis gives us two possible scenarios:\n* **Scenario A:** Yellow is in house 1, which means Green must be in house 4.\n* **Scenario B:** Yellow is in house 5, which means Green must be in house 2.\n\n**Step 2: Place the White house.**\n* Clue 1 states that the green house is immediately to the right of the white house (meaning they are a pair: White, Green).\n* In **Scenario A** (Green in 4), White must be in house 3. \n* In **Scenario B** (Green in 2), White must be in house 1.\n\n**Step 3: Place the Red and Blue houses.**\n* Clue 5 states that the blue house is not at either end, meaning it cannot be in house 1 or house 5.\n\nLet's test **Scenario A**:\n* Houses: 1: Yellow, 2: ?, 3: White, 4: Green, 5: ?\n* The remaining colors for houses 2 and 5 are Red and Blue. \n* Since Blue cannot be at an end (house 5), Blue must be in house 2, leaving Red in house 5.\n* However, Clue 2 says the red house is to the left of the blue house. Here, Red (5) is to the right of Blue (2), which violates the clue. \n* Therefore, Scenario A is invalid.\n\nLet's test **Scenario B**:\n* Houses: 1: White, 2: Green, 3: ?, 4: ?, 5: Yellow\n* The remaining colors for houses 3 and 4 are Red and Blue. \n* Clue 2 says Red is to the left of Blue. Therefore, Red must be in house 3 and Blue must be in house 4.\n* Let's check Clue 5: Blue is in house 4, which is not an end. This fits perfectly.\n\nThe only valid arrangement is:\nHouse 1: White\nHouse 2: Green\nHouse 3: Red\nHouse 4: Blue\nHouse 5: Yellow\n\n1","status":"completed","finishReason":"","latencyMs":38370,"tokensIn":185,"tokensOut":2827,"tokensPerSecond":73.7,"costUsd":0.0126978,"accuracy":99,"accuracyNote":"Reaches the correct arrangement and count with sound, fully-verified case analysis. The long <think> section is GLM's reasoning surfaced by our gateway from its separate reasoning channel (not dumped into the answer), so it isn't penalised — the final solution itself is clean and correct.","receiptVersion":1,"promptHash":"119b0025774a31bd409ae9f1f387f96eaea0c8eb3f793dc0186496fecc3df336","rawOutputHash":"2dfceaafb7a800a36299028ff1928a564e4158c4228e5d77bf4c32a676288310","graderVersion":"claude-agent-grader-v1","solverVersion":"constraint-solver-v1-120-permutations","harnessVersion":null,"validationState":"recorded","validationFlags":["prompt-hash=sha256","raw-output-hash=sha256","status=completed","accuracy=range-valid","latency=recorded","tokens=recorded","cost=recorded","grader-score=recorded","reasoning-leak=detected"],"correctionVersion":"correction-v1"}],"receiptVersion":1,"promptHash":"119b0025774a31bd409ae9f1f387f96eaea0c8eb3f793dc0186496fecc3df336","graderVersion":"claude-agent-grader-v1","solverVersion":"constraint-solver-v1-120-permutations","harnessVersion":null,"validationState":"recorded","validationFlags":["suite-family=premium","prompt-hash=sha256","raw-output-hashes=per-result-sha256","grader-version=declared","validation=deterministic","solver-version=declared"],"correctionVersion":"correction-v1"},{"slug":"median-two-sorted-arrays","title":"Median of Two Sorted Arrays in O(log n)","shortTitle":"Hard Algorithm","category":"Advanced Coding","icon":"〈/〉","summary":"A classic hard algorithm: compute the median of two sorted lists in O(log(min(m,n))) time. A merge is explicitly disallowed, so the model must implement the tricky binary-search partition correctly — including empty-list and even/odd edge cases — and return code only.","criteria":["Returns the correct median across thousands of randomized inputs and all edge cases.","Achieves true O(log(min(m,n))) via a binary-search partition — no merging or sorting the combined input.","Handles an empty input list and always returns a float.","Returns only the code, with no explanation or leaked reasoning."],"groundTruth":"Every solution was executed against a reference implementation over 5,000+ random list pairs plus fixed edge cases (empty lists, duplicates, negatives, odd/even totals). All four produced the correct median with the required O(log) partition; the differentiator is instruction-following and clean output.","metaTitle":"Best Premium AI for Hard Coding — GPT-5.4 Pro vs Opus 4.8 vs Gemini vs GLM 5.2","metaDescription":"We asked the top flagship models for an O(log n) median-of-two-sorted-arrays implementation and ran every answer through a 5,000-case correctness harness. Compare real speed, cost and code quality.","keywords":["best AI for coding","GPT-5.4 Pro coding benchmark","Claude Opus 4.8 code","Gemini 3.1 Pro coding","GLM 5.2 max coding","premium LLM coding comparison","median of two sorted arrays AI"],"prompt":"Write a Python function `median_of_two_sorted(a, b)` that returns the median of two already-sorted lists of numbers `a` and `b`.\n\nRequirements:\n- It must run in O(log(min(len(a), len(b)))) time. Do NOT merge the lists or sort the combined input (an O(m+n) solution is not acceptable).\n- The two input lists are each individually sorted in non-decreasing order.\n- Handle the case where one list is empty.\n- For an even total number of elements, return the average of the two middle values; for an odd total, return the single middle value. Always return a float.\n- Return ONLY the code inside a single Python code block, with no explanation.","results":[{"modelId":"gpt-5.4-pro","modelName":"GPT-5.4 Pro","provider":"OpenAI","outputPricePerM":180,"output":"```python\nfrom math import inf\nfrom typing import Sequence\n\n\ndef median_of_two_sorted(a: Sequence[float], b: Sequence[float]) -> float:\n    if len(a) > len(b):\n        a, b = b, a\n\n    n, m = len(a), len(b)\n    if n == 0 and m == 0:\n        raise ValueError(\"median is undefined for two empty lists\")\n\n    total = n + m\n    half = (total + 1) // 2\n\n    left, right = 0, n\n    while left <= right:\n        i = (left + right) // 2\n        j = half - i\n\n        left_a = -inf if i == 0 else a[i - 1]\n        right_a = inf if i == n else a[i]\n\n        left_b = -inf if j == 0 else b[j - 1]\n        right_b = inf if j == m else b[j]\n\n        if left_a <= right_b and left_b <= right_a:\n            if total % 2 == 1:\n                return float(max(left_a, left_b))\n            return float((max(left_a, left_b) + min(right_a, right_b)) / 2.0)\n\n        if left_a > right_b:\n            right = i - 1\n        else:\n            left = i + 1\n\n    raise ValueError(\"input lists must be sorted in non-decreasing order\")\n```","status":"completed","finishReason":"","latencyMs":206116,"tokensIn":152,"tokensOut":548,"tokensPerSecond":2.7,"costUsd":0.1032,"accuracy":100,"accuracyNote":"Passes all 5,000+ randomized cases and every edge case; genuine O(log(min(m,n))) partition search, returns a float, code-only, and explicitly raises on two empty lists. Correct — but the slowest run by far (over three minutes), and now that real usage is reported, comfortably the most expensive.","receiptVersion":1,"promptHash":"abc8c17072e616feae256681c64aa944e31b145375338fb5269d4eb831708a57","rawOutputHash":"a2aefdcfc47376d961f5c43a54d8fe0f7226dd2a8e0334b1127cbd755f2f3945","graderVersion":"claude-agent-grader-v1","solverVersion":null,"harnessVersion":"median-harness-v1-seed-2026-5000-cases","validationState":"recorded","validationFlags":["prompt-hash=sha256","raw-output-hash=sha256","status=completed","accuracy=range-valid","latency=recorded","tokens=recorded","cost=recorded","grader-score=recorded","reasoning-leak=absent","code-fence=present"],"correctionVersion":"correction-v1"},{"modelId":"claude-opus-4-8","modelName":"Claude Opus 4.8","provider":"Anthropic","outputPricePerM":25,"output":"```python\ndef median_of_two_sorted(a, b):\n    # Ensure a is the smaller list\n    if len(a) > len(b):\n        a, b = b, a\n\n    m, n = len(a), len(b)\n    total = m + n\n    half = (total + 1) // 2\n\n    lo, hi = 0, m\n    while lo <= hi:\n        i = (lo + hi) // 2  # elements taken from a\n        j = half - i        # elements taken from b\n\n        a_left = a[i - 1] if i > 0 else float('-inf')\n        a_right = a[i] if i < m else float('inf')\n        b_left = b[j - 1] if j > 0 else float('-inf')\n        b_right = b[j] if j < n else float('inf')\n\n        if a_left <= b_right and b_left <= a_right:\n            if total % 2 == 1:\n                return float(max(a_left, b_left))\n            else:\n                return (max(a_left, b_left) + min(a_right, b_right)) / 2.0\n        elif a_left > b_right:\n            hi = i - 1\n        else:\n            lo = i + 1\n\n    raise ValueError(\"Input lists are not sorted or invalid\")\n```","status":"completed","finishReason":"","latencyMs":3790,"tokensIn":220,"tokensOut":382,"tokensPerSecond":100.8,"costUsd":0.010650000000000001,"accuracy":100,"accuracyNote":"Passes all 5,000+ randomized cases and every edge case; genuine O(log(min(m,n))) partition, float return, code-only. The fastest correct solution in this task.","receiptVersion":1,"promptHash":"abc8c17072e616feae256681c64aa944e31b145375338fb5269d4eb831708a57","rawOutputHash":"2ba7b9c31f2a871ff6037dee27a26cfa4aba1d36f417e2b01ac65473b802a3a6","graderVersion":"claude-agent-grader-v1","solverVersion":null,"harnessVersion":"median-harness-v1-seed-2026-5000-cases","validationState":"recorded","validationFlags":["prompt-hash=sha256","raw-output-hash=sha256","status=completed","accuracy=range-valid","latency=recorded","tokens=recorded","cost=recorded","grader-score=recorded","reasoning-leak=absent","code-fence=present"],"correctionVersion":"correction-v1"},{"modelId":"gemini-3.1-pro","modelName":"Gemini 3.1 Pro","provider":"Google","outputPricePerM":12,"output":"```python\ndef median_of_two_sorted(a, b):\n    if len(a) > len(b):\n        a, b = b, a\n        \n    m, n = len(a), len(b)\n    low, high = 0, m\n    \n    while low <= high:\n        partitionA = (low + high) // 2\n        partitionB = (m + n + 1) // 2 - partitionA\n        \n        maxLeftA = float('-inf') if partitionA == 0 else a[partitionA - 1]\n        minRightA = float('inf') if partitionA == m else a[partitionA]\n        \n        maxLeftB = float('-inf') if partitionB == 0 else b[partitionB - 1]\n        minRightB = float('inf') if partitionB == n else b[partitionB]\n        \n        if maxLeftA <= minRightB and maxLeftB <= minRightA:\n            if (m + n) % 2 == 1:\n                return float(max(maxLeftA, maxLeftB))\n            else:\n                return (max(maxLeftA, maxLeftB) + min(minRightA, minRightB)) / 2.0\n        elif maxLeftA > minRightB:\n            high = partitionA - 1\n        else:\n            low = partitionA + 1\n```","status":"completed","finishReason":"","latencyMs":17413,"tokensIn":164,"tokensOut":320,"tokensPerSecond":18.4,"costUsd":0.004168,"accuracy":99,"accuracyNote":"Passes all tests with a clean, minimal O(log) partition and float return, code-only. Docked one point only because two empty lists yield NaN rather than an explicit guard (not required by the prompt).","receiptVersion":1,"promptHash":"abc8c17072e616feae256681c64aa944e31b145375338fb5269d4eb831708a57","rawOutputHash":"c87151f5b92f95381d9b0bf0ec89b041834f164eda008b3f0a2baa885fc37617","graderVersion":"claude-agent-grader-v1","solverVersion":null,"harnessVersion":"median-harness-v1-seed-2026-5000-cases","validationState":"recorded","validationFlags":["prompt-hash=sha256","raw-output-hash=sha256","status=completed","accuracy=range-valid","latency=recorded","tokens=recorded","cost=recorded","grader-score=recorded","reasoning-leak=absent","code-fence=present"],"correctionVersion":"correction-v1"},{"modelId":"glm-5.2","modelName":"GLM 5.2 (Max)","provider":"Z.ai","outputPricePerM":4.4,"output":"<think>\nWe need to write a Python function `median_of_two_sorted(a, b)` that returns the median of two sorted lists in O(log(min(len(a), len(b)))) time. The function should handle empty lists, return float, and for even total return average of two middle values, for odd return the single middle value.\n\nWe must implement the classic binary search on the smaller array. The algorithm: ensure a is the smaller array. Let m = len(a), n = len(b). We want to partition a and b such that left half contains elements <= right half. We binary search on partition index i in a (0 to m). Then j = (m + n + 1) // 2 - i. We need to ensure that a[i-1] <= b[j] and b[j-1] <= a[i] (with boundary checks). If condition holds, we found the correct partition. Then if total length is odd, median is max of left half. If even, median is (max(left) + min(right)) / 2.0.\n\nEdge cases: empty arrays. If a is empty, median is just median of b. If b is empty, median of a. But the algorithm should handle it if we set a as the smaller array. If a is empty, m=0, then i=0, j = (0+n+1)//2 = (n+1)//2. Then we need to check conditions: a[i-1] doesn't exist, so we treat as -inf. b[j-1] <= a[i] (a[i] doesn't exist, treat as inf). So condition holds. Then left max is b[j-1] (if j>0) else -inf. Right min is b[j] (if j<n) else inf. Then compute median. This works.\n\nWe must return float. For odd total, return float(median). For even, return average as float.\n\nImplementation details: Use integer indices. Use float('inf') and float('-inf') for boundaries.\n\nLet's write the code.\n\n```python\ndef median_of_two_sorted(a, b):\n    # Ensure a is the smaller array\n    if len(a) > len(b):\n        a, b = b, a\n    m, n = len(a), len(b)\n    # Binary search on a\n    low, high = 0, m\n    while low <= high:\n        i = (low + high) // 2\n        j = (m + n + 1) // 2 - i\n        # Get left and right elements with boundaries\n        a_left = a[i-1] if i > 0 else float('-inf')\n        a_right = a[i] if i < m else float('inf')\n        b_left = b[j-1] if j > 0 else float('-inf')\n        b_right = b[j] if j < n else float('inf')\n        \n        if a_left <= b_right and b_left <= a_right:\n            # Found correct partition\n            if (m + n) % 2 == 1:\n                return float(max(a_left, b_left))\n            else:\n                return (max(a_left, b_left) + min(a_right, b_right)) / 2.0\n        elif a_left > b_right:\n            # i is too large, need to decrease i\n            high = i - 1\n        else:\n            # b_left > a_right, need to increase i\n            low = i + 1\n    # Should never reach here if inputs are sorted\n    raise ValueError(\"Input lists are not sorted or invalid.\")\n```\n\nCheck: For odd total, (m+n+1)//2 gives the size of left half (larger half). For even, left half size = (m+n)//2. The formula j = (m+n+1)//2 - i works for both: if total odd, left half has one more element, so j = (total+1)//2 - i. If total even, (total+1)//2 = total//2 (since total even, total+1 odd, integer division gives total//2). So j = total//2 - i. That's correct.\n\nTest with examples:\na = [1, 3], b = [2] -> m=2, n=1. a is larger? Actually len(a)=2, len(b)=1, so we swap: a=[2], b=[1,3]. m=1, n=2. low=0, high=1. i=0: j=(1+2+1)//2 - 0 = 4//2=2. a_left=-inf, a_right=2, b_left=b[1]=3, b_right=inf. Check: a_left <= b_right (-inf <= inf) true. b_left <= a_right? 3 <= 2 false. So b_left > a_right, need to increase i. low=1. i=1: j=2-1=1. a_left=a[0]=2, a_right=inf, b_left=b[0]=1, b_right=b[1]=3. Check: 2<=3 true, 1<=inf true. Found. total=3 odd. max(2,1)=2. return 2.0. Correct.\n\na = [1, 2], b = [3, 4] -> m=2, n=2. low=0, high=2. i=1: j=(4+1)//2 -1 = 2-1=1. a_left=1, a_right=2, b_left=3, b_right=4. Check: 1<=4 true, 3<=2 false. b_left > a_right, increase i. low=2. i=2: j=2-2=0. a_left=2, a_right=inf, b_left=-inf, b_right=3. Check: 2<=3 true, -inf<=inf true. Found. total=4 even. max(2, -inf)=2, min(inf, 3)=3. avg=2.5. Correct.\n\nEmpty a: a=[], b=[1,2,3]. m=0, n=3. low=0, high=0. i=0: j=(0+3+1)//2 -0 = 4//2=2. a_left=-inf, a_right=inf, b_left=b[1]=2, b_right=b[2]=3. Check: -inf<=3 true, 2<=inf true. Found. total=3 odd. max(-inf,2)=2. return 2.0. Correct.\n\nEmpty both? Not specified but if both empty, m=0,n=0. low=0, high=0. i=0: j=(0+0+1)//2 -0 = 0. a_left=-inf, a_right=inf, b_left=-inf, b_right=inf. Check: -inf<=inf true, -inf<=inf true. Found. total=0 even? (0%2==0). max(-inf,-inf)=-inf, min(inf,inf)=inf. avg = (-inf+inf)/2 = nan. But problem likely assumes at least one element? Usually median of empty is undefined. We could handle by returning 0.0 or raising error. But requirements say \"Handle the case where one list is empty.\" It doesn't mention both empty. We can leave as is, or add a check: if m==0 and n==0: return 0.0. But to be safe, we can just let it return nan, but better to handle. Since it's not specified, we can assume at least one list non-empty. But to be robust, we can add: if m == 0 and n == 0: return 0.0. However, the problem says \"Handle the case where one list is empty.\" So both empty might not be tested. I'll include a check for both empty returning 0.0 or maybe raise ValueError. But the instruction says \"Return ONLY the code inside a single Python code block, with no explanation.\" So we just provide the function. I'll add a guard: if not a and not b: return 0.0. But that might be unnecessary. Let's keep it simple.\n\nAlso, we need to ensure we return float. In Python, division / returns float. For odd case, we return float(max(...)). That's fine.\n\nTime complexity: O(log(min(m,n))) because we binary search on the smaller array. Space O(1).\n\nLet's write the final code block.\n</think>\n```python\ndef median_of_two_sorted(a, b):\n    # Ensure a is the smaller array for O(log(min(m, n))) complexity\n    if len(a) > len(b):\n        a, b = b, a\n    m, n = len(a), len(b)\n    \n    # Edge case: both empty (not required, but safe)\n    if m == 0 and n == 0:\n        return 0.0\n        \n    low, high = 0, m\n    while low <= high:\n        i = (low + high) // 2\n        j = (m + n + 1) // 2 - i\n        \n        a_left = a[i - 1] if i > 0 else float('-inf')\n        a_right = a[i] if i < m else float('inf')\n        b_left = b[j - 1] if j > 0 else float('-inf')\n        b_right = b[j] if j < n else float('inf')\n        \n        if a_left <= b_right and b_left <= a_right:\n            if (m + n) % 2 == 1:\n                return float(max(a_left, b_left))\n            else:\n                return (max(a_left, b_left) + min(a_right, b_right)) / 2.0\n        elif a_left > b_right:\n            high = i - 1\n        else:\n            low = i + 1\n            \n    raise ValueError(\"Input lists are not sorted or invalid.\")\n```","status":"completed","finishReason":"","latencyMs":47713,"tokensIn":158,"tokensOut":2148,"tokensPerSecond":45,"costUsd":0.009672400000000001,"accuracy":100,"accuracyNote":"Passes all 5,000+ randomized cases and every edge case with a genuine O(log) partition and float return. The <think> block ahead of the code is GLM's reasoning channel surfaced by our gateway, not reasoning dumped into the answer — GLM's actual content is the clean code block — so it scores level with the other correct solutions, as the cheapest of them.","receiptVersion":1,"promptHash":"abc8c17072e616feae256681c64aa944e31b145375338fb5269d4eb831708a57","rawOutputHash":"1e909c8417f150d1aac0eb4b7d4c3219ed4678847a6c0d18a24005722a5b6ae3","graderVersion":"claude-agent-grader-v1","solverVersion":null,"harnessVersion":"median-harness-v1-seed-2026-5000-cases","validationState":"recorded","validationFlags":["prompt-hash=sha256","raw-output-hash=sha256","status=completed","accuracy=range-valid","latency=recorded","tokens=recorded","cost=recorded","grader-score=recorded","reasoning-leak=detected","code-fence=present"],"correctionVersion":"correction-v1"}],"receiptVersion":1,"promptHash":"abc8c17072e616feae256681c64aa944e31b145375338fb5269d4eb831708a57","graderVersion":"claude-agent-grader-v1","solverVersion":null,"harnessVersion":"median-harness-v1-seed-2026-5000-cases","validationState":"recorded","validationFlags":["suite-family=premium","prompt-hash=sha256","raw-output-hashes=per-result-sha256","grader-version=declared","validation=deterministic","harness-version=declared"],"correctionVersion":"correction-v1"}]}