← back to research
beyond-the-illusion.md

Beyond the Illusion

16 June 2025·5 min read

This post originally credited the rebuttal to Apple's reasoning paper to a website, and welded it to an unrelated DeepMind system. Correcting the record: what 'The Illusion of Thinking' measured, which of the three criticisms held up, and the two eval-harness bugs underneath the whole argument.

aillmsevaluationreasoning-modelsmlops

This post originally credited the rebuttal to Apple's reasoning paper to "AlphaXiv" — alphaXiv is a discussion layer over arXiv built by Stanford students, not an author — and then tied that rebuttal to Google DeepMind's AlphaEvolve, which had nothing to do with it. Neither claim survives thirty seconds of checking. The record:

What the Apple paper measured

"The Illusion of Thinking" (arXiv:2506.06941, 7 June 2025) is by Parshin Shojaee, Iman Mirzadeh, Keivan Alizadeh, Maxwell Horton, Samy Bengio and Mehrdad Farajtabar. Their objection to standard maths and coding benchmarks was contamination plus silence about the reasoning trace, so they built four puzzles with a dial on complexity — Tower of Hanoi, Checker Jumping, River Crossing, Blocks World — and ran Claude 3.7 Sonnet with and without thinking, DeepSeek-R1 and V3, and o3-mini across it.

Three findings. Accuracy collapses to zero past a complexity threshold. Reasoning effort climbs with complexity and then falls as that threshold approaches, with token budget still unspent. And there are three regimes: non-thinking models ahead on easy instances, thinking models ahead in the middle, both at zero at the top.

That third result is the most useful thing in the paper and it got almost no airtime, because the headline was the collapse.

The rebuttal, and who signed it

Three days later: arXiv:2506.09250, 10 June 2025. The document's own title is "The Illusion of the Illusion of Thinking"; the arXiv metadata title is the drier "Comment on The Illusion of Thinking". Version 1 listed two authors — "C. Opus, Anthropic" and "A. Lawsen, Open Philanthropy". C. Opus is Claude Opus 4. Version 2, six days later, removed it under arXiv's authorship policy. Both versions are still posted.

The criticisms are all about the harness.

Hanoi output length. An optimal solution for N disks is 2^N−1 moves. At roughly five tokens a move, a 64,000–100,000 token budget runs dry around N=7 or 8 — where the accuracy curve falls off. Models said so in their outputs before truncating.

The grader. It compared move lists, so a model that stopped against a ceiling scored the same as one that reasoned badly. Nothing distinguished them.

River Crossing. Instances with more than five pairs and a boat holding three have no solution. Models were marked wrong for pointing that out — Lawsen's comparison is penalising a SAT solver for returning unsatisfiable on an unsatisfiable formula.

Then the alternative: ask for a Lua function that prints the solution rather than the move list. Hanoi at N=15, under 5,000 tokens, high accuracy on models previously scored at zero. The paper calls those experiments preliminary and they are — emitting three lines of recursion is a different job from holding 32,767 moves in order without drifting. It shows the model knows the algorithm, and says nothing about executing one at length.

What the replication found

Rethinking the Illusion of Thinking (Dellibarda Varela, Romero-Sorozabal, Rocon and Cebrian, 1 July 2025) re-ran both puzzles and the result splits. Hanoi failures were not purely output constraints; something does give out around eight disks. Restricted to solvable instances, River Crossing models cleared configurations with more than 100 pairs.

One benchmark's collapse was largely real. The other was a bug in the problem set.

Apple's camera-ready concedes the second half. Version 3 (20 November 2025, NeurIPS 2025) adds an appendix titled "Response to Main Criticisms". On tokens they hold their ground: collapse starts at N=7–8, roughly 100–200 moves, inside the context window, with the first wrong move landing well before the output ceiling. On River Crossing they accept that the dynamics shift for N≥6, where optimal boat capacity becomes four.

AlphaEvolve is a different system

AlphaEvolve (DeepMind, 14 May 2025) is a coding agent: Gemini models propose edits to a program, an automated evaluator scores each candidate, survivors get mutated again. What evolves is source code. No population of models, no bred architectures, no mutated weights — the original version of this post described a system that does not exist and built three paragraphs of MLOps advice on it. Its published results are concrete: 48 scalar multiplications for a 4×4 complex matrix product against the 49 in Strassen's 1969 algorithm; 0.7% of Google's fleet-wide compute recovered through a better Borg scheduling heuristic; a 23% faster matmul kernel in Gemini's own training stack, worth about 1% of total training time.

The load-bearing component is the evaluator. AlphaEvolve only works where a machine can score a candidate automatically and cheaply, which is why the wins look like kernels and packing bounds.

The part that transfers

Two of the three criticisms that stuck to a front-page paper were harness bugs, and both are cheap to make.

Scoring a truncated completion as an incorrect answer: any grader that diffs output against a reference and ignores the API's stop reason does this, and it fails in the direction that looks like a finding — the model appears to degrade as the task gets long, because length is when the ceiling arrives. Keep max_tokens in its own bucket.

Shipping a generated test set nobody proved solvable: generation and solution are separate code paths, and usually only one has tests. Something has to solve every procedurally built instance before it counts.

Neither is a sophisticated mistake, which is the reason it's worth writing down. Apple's appendix now confines River Crossing analysis to fewer than six pairs.

Papers:

← back to research