The Collection

Gary Marcus wrote this one in a hurry, he says, before a long flight without wifi, and asks readers to forgive the typos. The argument survives the rush. OpenAI has put out what he calls a “massive new math drop”, and his complaint is not with the result. “The real news here isn’t the result,” he writes; “it’s not what we were told.” The report, as he quotes it, leans on phrases such as “same procedure” and “using an unreleased model”. His verdict: “This would never pass peer review.”

He lists what is missing. The procedure. The architecture: were the proofs written in one shot and then checked by Lean, the formal proof checker, or produced in a loop of tries and corrections? The failure rate. The training, post-training and data augmentation. Without those, he argues, nobody can say how far the result reaches beyond mathematics.

No failure rate means no idea what one success is worth.

He leaves both doors open. The system “could be a legitimate step toward AGI”, or “a clever leveraging of Lean and synthetic data in a verifiable domain with no generality whatsoever.” From the first report, he says, you cannot tell which. His sharpest line is aimed at the audience, which he describes as a cheering section that applauds without asking basic scientific questions. The letter promises a second part with Terence Tao’s view. That part did not come through in the copy this paper received, so Tao is not quoted here.

Senior people from OpenAI, Anthropic, Google and Meta testified under oath before the full New York City Council. Former lab staff, including Daniel Kokotajlo, Alex Turner and Jacob Coxon, testified too. AI Governance Lead’s summary sorts the answers into two piles: where the companies were direct, and where they stayed vague. The vague pile holds the things that matter most for holding anyone to account.

Asked for the probability of a catastrophic outcome, the company representatives could not or would not give one. OpenAI’s representative said even a 1 to 20% chance would be “not remotely acceptable”, and still offered no framework for measuring it. Asked whether they would release a model that failed internal or external safety tests, the answers were, in the letter’s word, non-committal. Liability for harm caused by AI was left unclear. The former insiders described a move-fast culture and called extinction-level risk more likely than not on the current path. That is their claim, and this paper prints it as theirs.

“Not remotely acceptable” is a feeling. A threshold has a number on it.

New York is floating roughly ten bills: outside validation before a model can be sold or deployed in the city, kill switches, liability, whistleblower rewards and shutdown powers. How many will pass is open. Most of the letter’s detail on each point sits behind a paywall this paper has not read. Its closing advice stands without it: inside a company, treat what a provider promises as one input, and set your own numbers, your own rule for when not to deploy, and your own escalation path.

The study is METR’s, published in July 2025: sixteen open-source developers, 246 real tasks. Before starting, the developers forecast that AI would cut their time by 24%. Timed, the work took 19% longer. That result has been quoted to exhaustion, Roan Brasil Monteiro writes, and “fewer people opened the CSV”. He did. He re-ran the public data using the measures companies actually use, and worked out how large a pilot has to be before it can say anything.

His headline gives the answer in two numbers. A 30-task pilot concludes nothing 89% of the time. Seeing a 10% gain takes about 1,400 tasks. Most corporate trials sit far closer to the first figure than the second, which means most of the “AI made us faster” slides in circulation are reporting noise, in either direction.

Thirty tasks is an anecdote with a spreadsheet attached.

The rest of the piece, including the protocol he says he would run, is member-only on Medium, and this paper has not read it. It does not guess at the method behind the 89%. What can be filed is the gap the study opened in the first place. The people doing the work were confident and wrong about their own speed, and only the stopwatch caught it. A forecast is not working. A timed sample, big enough, is.

Google Playground lets adults in the United States make games by describing them, with no coding. The creator changes characters, rules and physics by talking, then shares the result privately, through an unlisted link, or in Google’s public gallery, where ratings and play decide what gets found. Published games are screened, and Google keeps the power to remove them. The terms leave the creator legally responsible for what goes out. Google supplies the model, the hosting, the distribution and the enforcement; the person typing the prompts carries the liability. Unity Spark, a richer editor that is still to come, will publish through Playground too.

The letter’s point is that this is what easy software looks like after the demo, and that companies are next. A normal application arrives with a project, a repository, a budget and a named owner. A generated one may start as an employee describing a problem, get connected to company data, and spread by link until it matters, all before the company’s list of applications knows it exists.

The app is useful, popular and important before anyone has written down what it is.

Worse for anyone asking to see the working: its real specification is spread across the model, the instructions, the retrieved data, the tools, the permissions and the vendor’s updates. Reading the code, if there is code, will not explain it. The letter rejects a review for every experiment, because people would just move to personal accounts. It asks instead for a release line: the point where a private experiment has enough reach, data or authority to need an owner. Where exactly to draw it sits behind the paywall, unread here.