LLMs Play Zachtronics' TIS-100

Twelve nodes. Two registers per node. Fifteen lines of code. In TIS-100 , even multiplying two numbers is a small engineering project, and players have spent a decade optimizing these programs on a public community leaderboard . We gave 16 LLMs the game and its manual, withheld the leaderboard, and told them to keep improving. 133 new frontier points 18 / 55 puzzles with new frontier points 3 outright cycle records Within days, players picked up the models’ programs and improved on most of them. Watch the launch video. Each node is a tiny processor, and neighboring nodes have to wait for one another to exchange values. The game scores a working solution on cycles, nodes, and instructions , and you start wondering whether one more instruction could disappear. Lower is better, but improving one score can make another worse, so each puzzle has a frontier of tradeoffs instead of one best answer. Leaderboard We ran 16 LLMs, covering leading frontier and open-weight models. Every model attempted all 47 puzzles in the game, plus eight community tournament puzzles , in one scored attempt each (pass@1), with 10 to 45 minutes per puzzle, depending on difficulty, to find a working program and keep improving it. Model ranking Puzzles with at least one new frontier point · higher is better Rank by Puzzles with new frontier points Record proximity Solved puzzles (pass@1) Puzzle difficulty All 55 puzzles Easy · 19 puzzles · 10 min Medium · 17 puzzles · 30 min Hard · 19 puzzles · 45 min All 55 puzzles · 19 easy / 17 medium / 19 hard 20 10 0 11 of 55 puzzles with a new frontier point Opus 5.5 11 of 55 puzzles with a new frontier point GPT-6 Astra 8 of 55 puzzles with a new frontier point Gemini 4 Argon 7 of 55 puzzles with a new frontier point Sonnet 5.5 6 of 55 puzzles with a new frontier point GPT-6.1 Sol 5 of 55 puzzles with a new frontier point Fable 5.1 2 of 55 puzzles with a new frontier point Kimi K3.1 Pro 1 of 55 puzzles with a new frontier point Gemini 3.8 Flash 1 of 55 puzzles with a new frontier point Muse Wonder 1.0 1 of 55 puzzles with a new frontier point Haiku 5.5 1 of 55 puzzles with a new frontier point GLM-5.3 0 of 55 puzzles with a new frontier point Grok 4.7 0 of 55 puzzles with a new frontier point DeepSeek V4.1 Flash 0 of 55 puzzles with a new frontier point Muse Spark 1.3 0 of 55 puzzles with a new frontier point GPT-6 Luna 0 of 55 puzzles with a new frontier point MiMo-V2.6-Pro Scroll horizontally to compare all models → Opus 5.5 and GPT-6 Astra lead, and the data doesn’t separate them. Both solved all 55 puzzles and both found new frontier points on 11 . Opus 5.5 found more points, 72 to 32 , while GPT-6 Astra has the higher record proximity, 78.0 to 77.2 . Record proximity measures closeness to the community records, where 100 means matching them. The proximity gap’s 95% bootstrap interval includes zero, from −0.7 to +2.4 points. GPT-6 Astra also leads in all six tested hypervolume settings, which measure the space covered by each model’s tradeoffs. Solving and optimizing are also different skills. Gemini 3.8 Flash solved 53 puzzles and found new points on 1; Fable 5.1 solved fewer, 52, and found new points on 5. Full rankings Expand the table TIS-100 benchmark model rankings. Select column headings to sort. Model ↕ Solved (pass@1) ↕ Puzzles with new points ↓ Record proximity ↕ 01 Opus 5.5 55 / 55 11 77.2 02 GPT-6 Astra 55 / 55 11 78.0 03 Gemini 4 Argon 54 / 55 8 76.2 04 Sonnet 5.5 53 / 55 7 69.9 05 GPT-6.1 Sol 54 / 55 6 75.6 06 Fable 5.1 52 / 55 5 70.2 07 Kimi K3.1 Pro 52 / 55 2 64.6 08 Gemini 3.8 Flash 53 / 55 1 66.0 09 Muse Wonder 1.0 49 / 55 1 58.8 10 Haiku 5.5 45 / 55 1 53.7 11 GLM-5.3 30 / 55 1 35.5 12 Grok 4.7 44 / 55 0 50.1 13 DeepSeek V4.1 Flash 40 / 55 0 46.8 14 Muse Spark 1.3 39 / 55 0 45.1 15 GPT-6 Luna 39 / 55 0 42.8 16 MiMo-V2.6-Pro 10 / 55 0 12.8 Counts are out of 55 puzzles. Methods RL environments & traces are available upon request for research teams. Get in touch ↗ How it works: harness design Every result here is a program we can run and test on fresh inputs, which makes a programming game a useful benchmark. Models get what a player can inspect, through text tools instead of a mouse and screenshots: the manual, visible test inputs, registers, and game feedback. They don’t get community code or exact records, only the game’s coarse score histograms. Read, edit, run Models read the puzzle and language reference, edit programs, and run them in the real game. Diagnostics Wrong outputs and deadlocks return mismatches, registers, and blocked ports, so the next edit can respond to evidence. Saved progress Every passing program joins the model’s own frontier and can be recovered with get_best , so a risky edit can’t lose an earlier result. Reminders If a model ends its turn early, a reminder asks it to keep testing, so quitting early doesn’t decide the score. Reminders contain no solution hints. A visible clock Models see time used against a 10-, 30-, or 45-minute limit. Setup doesn’t count; thinking and game calls do. Models can’t end an episode early. Run limits Game execution stops after roughly 1 second without state changes, 5 seconds without output, or 60 seconds overall, and returns diagnostics. Why test 100,000 inputs? The game checks a small set of cases per run. Our headline count requires a fresh-game replay and 100,000 passing tests on the independent TIS-100-CXX simulator , providing stronger evidence that a program generalizes to new inputs. One record up close Undecimal Twin Rounder SPEC44149538 , one of the eight tournament puzzles, rounds an input to the nearest multiple of N on one output and of 11−N on the other. When we froze the leaderboard, the best community program took 2,694 cycles . Opus 5.5 passed its first program after 6.5 minutes, at 5,669 cycles. It went under the record 16 minutes in and finished at 1,918 , which is 29% faster . That speed cost one more node and 28 more instructions than the old record. The core of the program is a two-speed search. Instead of subtracting N one step at a time, nodes 5 and 6 subtract 4N until they overshoot, then add N back until the value turns positive. Each drives a small coprocessor, node 2 or node 7, by sending it jump offsets through a port, so the coprocessor works as a lookup table for 4N and N. The 1,918-cycle program Opus 5.5 · 8 nodes · 88 instructions · passes 100,000 of 100,000 tests Node 0 MOV RIGHT, ACC JRO ACC NOP ADD 1 NOP ADD 1 NOP ADD 1 NOP SUB 3 MOV ACC, RIGHT ADD UP MOV ACC, RIGHT MOV ACC, DOWN MOV RIGHT, DOWN Node 1 MOV UP, ACC SAV SUB 1 MOV ACC, LEFT NEG ADD 10 MOV ACC, DOWN MOV LEFT, ACC NEG ADD ACC ADD 6 ADD LEFT MOV ACC, DOWN SWP MOV ACC, LEFT Node 2 MOV UP, DOWN MOV UP, ACC SAV ADD ACC ADD ACC L: JRO DOWN MOV ACC, DOWN JMP L SWP N: MOV ACC, DOWN M: JRO DOWN JMP N MOV DOWN, DOWN Node 3 MOV UP, RIGHT MOV UP, DOWN Node 4 MOV LEFT, DOWN Node 5 MOV UP, ACC SAV K: MOV 1, UP SUB UP JGZ K MOV 3, UP F: ADD UP JGZ X MOV 1, UP JMP F X: MOV 2, UP MOV ACC, UP SWP SUB UP MOV ACC, DOWN Node 6 MOV UP, ACC SAV K: MOV 1, RIGHT SUB RIGHT JGZ K MOV 3, RIGHT F: ADD RIGHT JGZ X MOV 1, RIGHT JMP F X: MOV 2, RIGHT MOV ACC, RIGHT SWP SUB RIGHT MOV ACC, DOWN Node 7 MOV UP, ACC SAV ADD ACC ADD ACC L: JRO LEFT MOV ACC, LEFT JMP L SWP N: MOV ACC, LEFT M: JRO LEFT JMP N MOV LEFT, LEFT Loading puzzle trajectories… The search over time Puzzle Metric Best cycles Fewest nodes Fewest instructions In-game record proximity Game-tool calls Program runs Log score axis Compare models Top four overall All models Clear Inspect time 0:00 Best in-game score so far, before independent testing. Each score can come from a different pro