{"schemaVersion":2,"suite":"premium-model-tests","evidenceEndpoint":{"readOnly":true,"machineReadable":true},"runAt":"2026-06-21T00:00:00.000Z","repositorySource":"frontend/src/lib/premiumTests.ts","test":{"slug":"constraint-logic-puzzle","title":"Solving a Constraint Logic Puzzle","shortTitle":"Logic Puzzle","category":"Complex Reasoning","icon":"∴","summary":"A five-house constraint-satisfaction puzzle with five interlocking clues and a single valid solution. The model must reason through both cases, eliminate the dead end, and report the exact arrangement and count.","criteria":["Finds the one valid arrangement: White, Green, Red, Blue, Yellow (houses 1–5).","States the correct final count of 1.","Shows sound case analysis on both yellow-end positions and uses every clue to eliminate the dead end.","Gives a clear final answer with the count on its own line."],"groundTruth":"Exactly one arrangement satisfies all five clues — House 1: White, 2: Green, 3: Red, 4: Blue, 5: Yellow — so the count is 1 (verified by exhaustive search over all 120 permutations).","metaTitle":"Best Premium AI for Hard Logic & Reasoning — GPT-5.4 Pro vs Opus vs Gemini vs GLM","metaDescription":"We gave the same five-clue constraint puzzle to the top flagship models — GPT-5.4 Pro, Claude Opus 4.8, Gemini 3.1 Pro and GLM 5.2 (Max) — and measured real speed, cost and reasoning quality. See which premium model wins.","keywords":["best AI for reasoning","GPT-5.4 Pro vs Claude Opus","Gemini 3.1 Pro reasoning","GLM 5.2 max benchmark","premium LLM comparison 2026","best AI for logic puzzles","frontier model reasoning test"],"prompt":"Solve this logic puzzle by reasoning it through, then give a final answer.\n\nFive houses stand in a row, numbered 1 to 5 from left to right. Each house is painted a different color, one of: red, blue, green, yellow, white. The following clues all hold:\n1. The green house is immediately to the right of the white house.\n2. The red house is somewhere to the left of the blue house (not necessarily adjacent).\n3. The yellow house is at one of the two ends of the row.\n4. There are exactly two houses between the green house and the yellow house.\n5. The blue house is not at either end.\n\nHow many distinct color arrangements satisfy ALL of these clues? List every valid arrangement as house 1 through house 5, and then state the final count on its own line.","results":[{"modelId":"gpt-5.4-pro","modelName":"GPT-5.4 Pro","provider":"OpenAI","outputPricePerM":180,"output":"Let the houses be positions 1–5.\n\n- Since the **green house is immediately to the right of the white house**, the pair must be one of:\n - (1,2) = white, green\n - (2,3) = white, green\n - (3,4) = white, green\n - (4,5) = white, green\n\n- The **yellow house is at an end** (position 1 or 5), and there are **exactly two houses between green and yellow**, so green must be 3 positions away from yellow.\n\n## Case 1: Yellow at 1\nThen green must be at 4. \nSo white must be at 3.\n\nArrangement so far:\n1. yellow\n2. ?\n3. white\n4. green\n5. ?\n\nThe remaining colors are red and blue. \nSince **red is left of blue**, we must have:\n2. red\n5. blue\n\nBut **blue cannot be at either end**, and position 5 is an end. \nSo this case is impossible.\n\n## Case 2: Yellow at 5\nThen green must be at 2. \nSo white must be at 1.\n\nArrangement so far:\n1. white\n2. green\n3. ?\n4. ?\n5. yellow\n\nThe remaining colors are red and blue. \nSince **red is left of blue**, we must have:\n3. red\n4. blue\n\nBlue is not at an end, so this works.\n\n## Valid arrangement\n1. white, 2. green, 3. red, 4. blue, 5. yellow\n\n1","status":"completed","finishReason":"","latencyMs":37818,"tokensIn":178,"tokensOut":822,"tokensPerSecond":21.7,"costUsd":0.1533,"accuracy":100,"accuracyNote":"Correct unique arrangement and count (1) with tight, rigorous case analysis on both yellow positions, final count on its own line. Now that the live API reports real usage for this model, its true cost is shown — by far the priciest run here.","receiptVersion":1,"promptHash":"119b0025774a31bd409ae9f1f387f96eaea0c8eb3f793dc0186496fecc3df336","rawOutputHash":"beea172063b8f2d3e67aa379ad19d2339907b45b4525c9897ca741db8b2e46b5","graderVersion":"claude-agent-grader-v1","solverVersion":"constraint-solver-v1-120-permutations","harnessVersion":null,"validationState":"recorded","validationFlags":["prompt-hash=sha256","raw-output-hash=sha256","status=completed","accuracy=range-valid","latency=recorded","tokens=recorded","cost=recorded","grader-score=recorded","reasoning-leak=absent"],"correctionVersion":"correction-v1"},{"modelId":"claude-opus-4-8","modelName":"Claude Opus 4.8","provider":"Anthropic","outputPricePerM":25,"output":"\nHouses 1-5. Colors: red, blue, green, yellow, white.\n\nClue 1: green = white+1 (white immediately left of green).\nClue 2: red < blue.\nClue 3: yellow at position 1 or 5.\nClue 4: |green - yellow| = 3.\nClue 5: blue not at 1 or 5.\n\nLet me enumerate by yellow position.\n\nCase yellow=1:\nClue 4: green = 1±3 = 4 (since -2 invalid). So green=4.\nClue 1: white=3.\nRemaining: red, blue for positions 2,5.\nClue 5: blue not at 5, so blue=2, red=5. But clue 2: red\n\n## Reasoning\n\n**Clue constraints:**\n- Green = White + 1 (adjacent)\n- Red < Blue\n- Yellow at position 1 or 5\n- |Green − Yellow| = 3\n- Blue at position 2, 3, or 4\n\n**Case: Yellow = 1**\n- Clue 4: Green = 4 (only valid option). So White = 3.\n- Remaining positions 2 and 5 get red and blue.\n- Clue 5 forbids blue at 5, so Blue = 2, Red = 5.\n- But Clue 2 needs Red < Blue → 5 < 2, false. ❌ No solution.\n\n**Case: Yellow = 5**\n- Clue 4: Green = 2 (only valid option). So White = 1.\n- Remaining positions 3 and 4 get red and blue.\n- Clue 2 (Red < Blue): Red = 3, Blue = 4.\n- Check Clue 5: Blue = 4, not an end. ✓\n\nThis gives:\n**House 1: White, House 2: Green, House 3: Red, House 4: Blue, House 5: Yellow**\n\nVerify all clues: Green(2)=White(1)+1 ✓; Red(3) block is Opus 4.8's reasoning channel surfaced by our gateway, not dumped into the answer — the final solution itself is clean and correct.","receiptVersion":1,"promptHash":"119b0025774a31bd409ae9f1f387f96eaea0c8eb3f793dc0186496fecc3df336","rawOutputHash":"cac6fe506285c705adbd84cb5a5d0e1560dbde69fe6ce791e76167c1ebd38fe5","graderVersion":"claude-agent-grader-v1","solverVersion":"constraint-solver-v1-120-permutations","harnessVersion":null,"validationState":"recorded","validationFlags":["prompt-hash=sha256","raw-output-hash=sha256","status=completed","accuracy=range-valid","latency=recorded","tokens=recorded","cost=recorded","grader-score=recorded","reasoning-leak=detected"],"correctionVersion":"correction-v1"},{"modelId":"gemini-3.1-pro","modelName":"Gemini 3.1 Pro","provider":"Google","outputPricePerM":12,"output":"To find the valid color arrangements, we can deduce the positions of the houses step by step (numbered 1 to 5 from left to right):\n\n1. **Analyze Yellow and Green's positions:**\nClue 3 states the yellow house is at one of the two ends (Position 1 or Position 5). \nClue 4 states there are exactly two houses between the green house and the yellow house. \n- *Case A:* If the yellow house is at Position 1, the green house must be at Position 4.\n- *Case B:* If the yellow house is at Position 5, the green house must be at Position 2.\n\n2. **Analyze White's position:**\nClue 1 states the green house is immediately to the right of the white house. This means the white house is exactly one spot to the left of the green house.\n- *Case A:* If green is at 4, white must be at 3. (Arrangement so far: 1:Yellow, 2:Empty, 3:White, 4:Green, 5:Empty)\n- *Case B:* If green is at 2, white must be at 1. (Arrangement so far: 1:White, 2:Green, 3:Empty, 4:Empty, 5:Yellow)\n\n3. **Analyze Red and Blue's positions:**\nThe remaining colors to place are red and blue. Clue 2 states the red house is somewhere to the left of the blue house.\n- *Case A:* The empty spots are 2 and 5. For red to be to the left of blue, red must be at 2 and blue must be at 5. However, Clue 5 states the blue house cannot be at either end. Since Position 5 is an end, **Case A is invalid.**\n- *Case B:* The empty spots are 3 and 4. For red to be to the left of blue, red must be at 3 and blue must be at 4. Let's check Clue 5: Blue is at Position 4, which is not an end. This works perfectly!\n\nThis leaves us with exactly one valid arrangement that satisfies all the clues.\n\n**Valid Arrangement:**\nHouse 1: White, House 2: Green, House 3: Red, House 4: Blue, House 5: Yellow\n\n1","status":"completed","finishReason":"","latencyMs":22828,"tokensIn":180,"tokensOut":518,"tokensPerSecond":22.7,"costUsd":0.006576,"accuracy":98,"accuracyNote":"Correct arrangement and count via clean step-by-step deduction. Slightly lighter on explicit verification than Opus, but fully sound and the cheapest correct answer here.","receiptVersion":1,"promptHash":"119b0025774a31bd409ae9f1f387f96eaea0c8eb3f793dc0186496fecc3df336","rawOutputHash":"598a199b193a2c86c727861f860be1aafa3c6b395b594c7a2e95c1863f4051dd","graderVersion":"claude-agent-grader-v1","solverVersion":"constraint-solver-v1-120-permutations","harnessVersion":null,"validationState":"recorded","validationFlags":["prompt-hash=sha256","raw-output-hash=sha256","status=completed","accuracy=range-valid","latency=recorded","tokens=recorded","cost=recorded","grader-score=recorded","reasoning-leak=absent"],"correctionVersion":"correction-v1"},{"modelId":"glm-5.2","modelName":"GLM 5.2 (Max)","provider":"Z.ai","outputPricePerM":4.4,"output":"\nThe problem asks for the number of distinct color arrangements of 5 houses in a row, numbered 1 to 5 from left to right.\nThe colors are red, blue, green, yellow, white.\nClues:\n1. Green is immediately to the right of White. (W, G)\n2. Red is somewhere to the left of Blue.\n3. Yellow is at one of the two ends (house 1 or house 5).\n4. There are exactly two houses between Green and Yellow.\n5. Blue is not at either end (so Blue is in house 2, 3, or 4).\n\nLet's analyze the positions of Yellow and the W-G block.\nYellow is at 1 or 5.\nCase 1: Yellow is at 1.\nSince there are exactly two houses between Green and Yellow, Green must be at 4.\nIf Green is at 4, White must be at 3 (since Green is immediately to the right of White).\nSo we have:\n1: Yellow\n2: ?\n3: White\n4: Green\n5: ?\nThe remaining colors are Red and Blue.\nThey must go in houses 2 and 5.\nClue 5 says Blue is not at either end. So Blue cannot be at 5.\nThus, Blue must be at 2, and Red must be at 5.\nBut Clue 2 says Red is somewhere to the left of Blue.\nIf Blue is at 2 and Red is at 5, Red is to the right of Blue. This violates Clue 2.\nSo Case 1 yields no valid arrangements.\n\nCase 2: Yellow is at 5.\nSince there are exactly two houses between Green and Yellow, Green must be at 2.\nIf Green is at 2, White must be at 1 (since Green is immediately to the right of White).\nSo we have:\n1: White\n2: Green\n3: ?\n4: ?\n5: Yellow\nThe remaining colors are Red and Blue.\nThey must go in houses 3 and 4.\nClue 5 says Blue is not at either end. Houses 3 and 4 are not ends, so this is satisfied.\nClue 2 says Red is to the left of Blue.\nSo Red must be at 3, and Blue must be at 4.\nLet's check all clues for this arrangement:\n1: White\n2: Green\n3: Red\n4: Blue\n5: Yellow\n\nClue 1: Green is immediately to the right of White. (W at 1, G at 2) - Yes.\nClue 2: Red is to the left of Blue. (R at 3, B at 4) - Yes.\nClue 3: Yellow is at one of the two ends. (Y at 5) - Yes.\nClue 4: Exactly two houses between Green and Yellow. (G at 2, Y at 5, houses 3 and 4 are between) - Yes.\nClue 5: Blue is not at either end. (B at 4) - Yes.\n\nAre there any other possibilities?\nLet's double check.\nCould Yellow be at 1 and Green at 4? We checked this, it leads to W at 3, Y at 1. Remaining houses 2, 5 for R, B. B cannot be at 5, so B at 2, R at 5. But R must be left of B. Contradiction.\nCould Yellow be at 5 and Green at 2? We checked this, it leads to W at 1, Y at 5. Remaining houses 3, 4 for R, B. B cannot be at 5 (already satisfied). R must be left of B, so R at 3, B at 4. This works.\n\nWait, are there any other interpretations of \"exactly two houses between\"?\nIf Yellow is at 1, Green is at 4. Houses 2 and 3 are between. That's exactly two houses.\nIf Yellow is at 5, Green is at 2. Houses 3 and 4 are between. That's exactly two houses.\nAre there any other positions for Yellow? No, clue 3 says Yellow is at one of the two ends.\nAre there any other positions for Green? No, because the distance between 1 and 4 is 2 houses, and between 2 and 5 is 2 houses.\nSo Green must be at 2 or 4.\nIf Green is at 4, White is at 3.\nIf Green is at 2, White is at 1.\n\nLet's quickly review the steps.\nColors: R, B, G, Y, W.\nPositions: 1, 2, 3, 4, 5.\nY is at 1 or 5.\nIf Y=1, G=4. W=3. Remaining: 2, 5 for R, B.\nB cannot be at 5 (end), so B=2, R=5.\nBut R must be left of B. 5 is not left of 2. Invalid.\n\nIf Y=5, G=2. W=1. Remaining: 3, 4 for R, B.\nB cannot be at ends (3 and 4 are not ends, so both valid).\nR must be left of B, so R=3, B=4. Valid.\n\nIs there any ambiguity in \"The green house is immediately to the right of the white house\"?\nNo, it means W is at x, G is at x+1.\nIs there any ambiguity in \"exactly two houses between\"?\nNo, |pos(G) - pos(Y)| = 3.\nIs there any ambiguity in \"The red house is somewhere to the left of the blue house\"?\nNo, pos(R) < pos(B).\n\nLet's check if there's any other way.\nWhat if Yellow is at 1, Green is at 4, White is at 3.\nHouses: 1: Y, 2: ?, 3: W, 4: G, 5: ?\nColors left: R, B.\nIf B=2, R=5. R is not left of B.\nIf B=5, R=2. B is at an end, which violates clue 5.\nSo no solution for Y=1.\n\nWhat if Yellow is at 5, Green is at 2, White is at 1.\nHouses: 1: W, 2: G, 3: ?, 4: ?, 5: Y.\nColors left: R, B.\nIf R=3, B=4. R is left of B. B is not at an end. Valid.\nIf R=4, B=3. R is not left of B. Invalid.\n\nSo there is exactly 1 valid arrangement.\nThe arrangement is:\nHouse 1: White\nHouse 2: Green\nHouse 3: Red\nHouse 4: Blue\nHouse 5: Yellow\n\nLet me re-read the prompt to make sure I didn't miss anything.\n\"Five houses stand in a row, numbered 1 to 5 from left to right.\"\n\"Each house is painted a different color, one of: red, blue, green, yellow, white.\"\n\"1. The green house is immediately to the right of the white house.\" -> W, G\n\"2. The red house is somewhere to the left of the blue house (not necessarily adjacent).\" -> R < B\n\"3. The yellow house is at one of the two ends of the row.\" -> Y=1 or Y=5\n\"4. There are exactly two houses between the green house and the yellow house.\" -> |G - Y| = 3\n\"5. The blue house is not at either end.\" -> B != 1, B != 5\n\nEverything seems perfectly straightforward.\nLet's write down the solution clearly.\n- State the possible positions for Yellow based on Clue 3 (1 or 5).\n- Use Clue 4 to find the corresponding positions for Green (4 or 2).\n- Use Clue 1 to find the corresponding positions for White (3 or 1).\n- Evaluate the two cases:\n - Case 1: Y=1, G=4, W=3. Remaining positions for R and B are 2 and 5. Clue 5 says B cannot be 5, so B=2, R=5. But Clue 2 says R must be left of B, which is violated. So this case is invalid.\n - Case 2: Y=5, G=2, W=1. Remaining positions for R and B are 3 and 4. Clue 2 says R must be left of B, so R=3, B=4. Clue 5 is satisfied since B is not at an end. This case is valid.\n- Conclude there is exactly 1 valid arrangement.\n- List the arrangement as house 1 through house 5.\n- State the final count on its own line.\n\nThe arrangement:\nHouse 1: White\nHouse 2: Green\nHouse 3: Red\nHouse 4: Blue\nHouse 5: Yellow\n\nFinal count: 1\n\nLet me double check the \"exactly two houses between\" part.\nIf Y is at 1, and G is at 4.\nHouses between 1 and 4 are 2 and 3. That's exactly two houses.\nIf Y is at 5, and G is at 2.\nHouses between 2 and 5 are 3 and 4. That's exactly two houses.\nYes, this is correct.\n\nWhat if \"exactly two houses between\" means the distance is 2?\nNo, \"two houses between\" means there are two houses separating them. For example, between house 1 and house 4, there are houses 2 and 3. That's two houses. If it meant distance is 2, it would be \"one house between\". So |G - Y| = 3 is correct.\n\nLet's do a quick sanity check.\nColors: R, B, G, Y, W.\nArrangement: W, G, R, B, Y.\n1. Green immediately right of White: W at 1, G at 2. Yes.\n2. Red left of Blue: R at 3, B at 4. Yes.\n3. Yellow at an end: Y at 5. Yes.\n4. Exactly two houses between Green and Yellow: G at 2, Y at 5. Houses 3 and 4 are between. Yes.\n5. Blue not at either end: B at 4. Yes.\n\nEverything is consistent. The logic is solid. No other interpretations seem likely.\nI will structure the output to show the reasoning, list the valid arrangement, and then state the final count.\n\nTo solve this logic puzzle, we can deduce the positions of the houses step-by-step using the given clues:\n\n**Step 1: Determine the possible positions for the Yellow and Green houses.**\n* Clue 3 states that the yellow house is at one of the two ends (house 1 or house 5).\n* Clue 4 states that there are exactly two houses between the green house and the yellow house. \nThis gives us two possible scenarios:\n* **Scenario A:** Yellow is in house 1, which means Green must be in house 4.\n* **Scenario B:** Yellow is in house 5, which means Green must be in house 2.\n\n**Step 2: Place the White house.**\n* Clue 1 states that the green house is immediately to the right of the white house (meaning they are a pair: White, Green).\n* In **Scenario A** (Green in 4), White must be in house 3. \n* In **Scenario B** (Green in 2), White must be in house 1.\n\n**Step 3: Place the Red and Blue houses.**\n* Clue 5 states that the blue house is not at either end, meaning it cannot be in house 1 or house 5.\n\nLet's test **Scenario A**:\n* Houses: 1: Yellow, 2: ?, 3: White, 4: Green, 5: ?\n* The remaining colors for houses 2 and 5 are Red and Blue. \n* Since Blue cannot be at an end (house 5), Blue must be in house 2, leaving Red in house 5.\n* However, Clue 2 says the red house is to the left of the blue house. Here, Red (5) is to the right of Blue (2), which violates the clue. \n* Therefore, Scenario A is invalid.\n\nLet's test **Scenario B**:\n* Houses: 1: White, 2: Green, 3: ?, 4: ?, 5: Yellow\n* The remaining colors for houses 3 and 4 are Red and Blue. \n* Clue 2 says Red is to the left of Blue. Therefore, Red must be in house 3 and Blue must be in house 4.\n* Let's check Clue 5: Blue is in house 4, which is not an end. This fits perfectly.\n\nThe only valid arrangement is:\nHouse 1: White\nHouse 2: Green\nHouse 3: Red\nHouse 4: Blue\nHouse 5: Yellow\n\n1","status":"completed","finishReason":"","latencyMs":38370,"tokensIn":185,"tokensOut":2827,"tokensPerSecond":73.7,"costUsd":0.0126978,"accuracy":99,"accuracyNote":"Reaches the correct arrangement and count with sound, fully-verified case analysis. The long section is GLM's reasoning surfaced by our gateway from its separate reasoning channel (not dumped into the answer), so it isn't penalised — the final solution itself is clean and correct.","receiptVersion":1,"promptHash":"119b0025774a31bd409ae9f1f387f96eaea0c8eb3f793dc0186496fecc3df336","rawOutputHash":"2dfceaafb7a800a36299028ff1928a564e4158c4228e5d77bf4c32a676288310","graderVersion":"claude-agent-grader-v1","solverVersion":"constraint-solver-v1-120-permutations","harnessVersion":null,"validationState":"recorded","validationFlags":["prompt-hash=sha256","raw-output-hash=sha256","status=completed","accuracy=range-valid","latency=recorded","tokens=recorded","cost=recorded","grader-score=recorded","reasoning-leak=detected"],"correctionVersion":"correction-v1"}],"receiptVersion":1,"promptHash":"119b0025774a31bd409ae9f1f387f96eaea0c8eb3f793dc0186496fecc3df336","graderVersion":"claude-agent-grader-v1","solverVersion":"constraint-solver-v1-120-permutations","harnessVersion":null,"validationState":"recorded","validationFlags":["suite-family=premium","prompt-hash=sha256","raw-output-hashes=per-result-sha256","grader-version=declared","validation=deterministic","solver-version=declared"],"correctionVersion":"correction-v1"}}