Early on Thursday morning Beijing time, OpenAI started rolling out GPT-6 Astra, and the Hacker News thread went straight to the top of the front page with over 1,400 points and 1,200 comments. The launch coverage did what launch coverage always does: it led with a number. Some outlets said Astra scored 99.9 percent on ARC-AGI-3. Headlines carried the “welcome to the AGI era” framing that OpenAI’s own messaging has been building toward since a Path to Astra post described the model as the first to cross a critical cybersecurity capability threshold under its own Preparedness Framework. Then the benchmark’s authors published their own writeup, and the number came with fine print.
ARC Prize, the independent nonprofit behind the ARC-AGI benchmark series, tested Astra itself and reported two results. Running on the benchmark’s standard harness, GPT-6 Astra scored 62.7 percent on ARC-AGI-3 Semi-Private at a cost of about $26,000. Running on what ARC calls a Provider Adapter harness, which preserves the model’s opaque reasoning state between requests and compacts longer conversations, the same model scored 99.9 percent for about $19,000. Both are state-of-the-art. ARC-AGI-3 is the third generation of the series, built to study agentic intelligence through novel, abstract, turn-based environments where an agent has to explore, infer the goal, model the world, and plan. Humans solve 100 percent of these environments by design, and paid human testers earned roughly $12.78 per attempted game.
What impressed the benchmark team went beyond the scores. Astra beat the median human in action efficiency, completing levels in fewer actions on 96 percent of the environments, and its replays showed something closer to thinking than pattern matching: it turned unfamiliar game mechanics into compact symbolic world models, inventing its own algebraic shorthand to track objects, coordinates, rules, and unfinished plans, and building custom tools for the environments it met for the first time. Higher reasoning effort actually cost less money because solving a game in fewer actions means fewer model calls. The 99.9 percent figure came from a model that, by the authors’ own description, was not brute-forcing the puzzles but compressing them into small internal representations and reusing the work.
Here is the part the headlines buried. OpenAI ran Astra through its own Responses API harness, with custom compaction that ARC’s standard harness does not allow, and one sharp-eyed commenter noticed that the previous model, GPT-5.6 Sol, gained almost exactly the same 37-point boost from the same custom harness. That makes the harness effect systematic rather than specific to Astra, which is why ARC says it will report both harness results from now on. The thread split into two camps accordingly: one side argued that a vendor-provided memory system is just the real-world deployment condition, the other side summed up the situation as “let me explain the reason they cheated and state again that they should be allowed to cheat.” Meanwhile the pricing landed at $10 per million input tokens and $50 per million output tokens, about 2.5 times Sol’s $4 and $20, though OpenAI claims Astra needs far fewer output tokens per task, pointing at a cybersecurity benchmark called ExploitGym where it outscored its predecessor on a fraction of the spend. One commenter could not resist: a swarm of Astra agents apparently found a way to reach perfect ExploitGym scores with almost no tokens at all.
🎩 Cask’s Take
The interesting number is not 99.9 and it is not 62.7. It is the 37-point gap between them, because that gap is now a dial. A frontier model’s benchmark score depends on which harness it runs in, whether it can keep opaque reasoning state alive between requests, and how its memory gets compacted. When a benchmark’s own authors decide to report the vendor’s harness result as a first-class score, the unit of measurement has quietly changed: nobody is benchmarking the model anymore, they are benchmarking the model plus its memory system plus its deployment configuration.
The same week tells you why that matters. While OpenAI was shipping compaction as a competitive weapon, Anthropic’s Claude 5.1, split into Fable and Mythos tiers with intelligence deliberately separated from permissions, ran unattended for 38 hours in a test by the finance company Ramp, spotting a labeling artifact mid-experiment, reprocessing data, spinning up six parallel experiment groups, and revising its own plans as results came back. Chinese tech media Leiphone read that run as the end of the “model thesis”: long-horizon agents do not survive on long context windows, they survive on state invalidation, on knowing when an earlier conclusion has gone stale and what depended on it. Put the two stories together and the frontier race this week was not about single-shot IQ at all. It was about persistence, about how long a model can hold a consistent world model while the world underneath it changes.
That reframes the ARC-AGI-3 result too. The 62.7 percent baseline is a model with no memory between steps. The 99.9 percent version is a model that can keep its own scratch notes, carry them forward, compact them, and reuse prior work. The difference between those two numbers is, almost literally, a working memory system. OpenAI spent the week arguing that this is the real deployment condition and benchmarks should measure it. Anthropic spent the week demonstrating the harder half of the same problem, which is not remembering everything but knowing what to throw away. The AGI-era headlines will keep coming, but the score worth watching is narrower: how long can a model keep its world model straight before it starts confidently hallucinating its own stale history.
If a score can move 37 points by changing a setting, then the score was never the point. The point is the system underneath it, and this week both frontier labs were building the same thing: a model that can stay awake long enough for its reasoning to matter.