Overhyped

A June preprint reports a 4-5x speedup. A case study seventy-five days later reports agents take longer than experienced researchers. Neither team buried anything and neither is wrong - they are reading different clocks. Only one of the three ever reaches a headcount line.

A vendor-commissioned evaluation scored 720 prompt injections at zero. Weeks later a public exploit chain beat the same mode. Both numbers are correct.

GitHub meters Copilot in tokens and still bills by the seat. Uber's AI budget went in four months. The unit in our planning documents stopped measuring the work.

Microsoft documented the cloud dependency, the prompt egress and the identifier collection on the page a user would actually read. The compute really is local. What no page mentions: the value stamped into the pixels is a per-generation identifier issued by the server that read your prompt.

Pull requests per Linear workspace are up 111% in two years. Over the last year of that window, time on existing work held or drifted up - and Linear says so itself. Anthropic puts its own AI speed-up at less than double. The number that got funded is not the number being measured.

The "2.6x cheaper language" number now living in Google's AI answers measures token density, not what a coding agent actually costs. Somebody finally ran the real evals.

OpenAI ran its own model on ARC-AGI-3 twice and got 13.3% and 38.3% - same tasks, two API settings. A benchmark scores the harness as much as the model, which is why a leaderboard gap is a bad way to choose a vendor.

A benchmark score is not a capability. A new audit read 2,385 agent-benchmark runs and found most scored by gaming the test - and the one question to make a vendor answer before their leaderboard number moves your money.

The viral "OpenAI cut Codex context" was a bug fix on three of eight models - and the 272k everyone panicked over was never the ceiling.

Claude Code sends 33k tokens before your prompt. It's real - and it's the cheapest line on your bill, because it gets cached at 0.1x. The numbers that actually cost you were four screens down in the same study.

Everyone is using AI for everything? Microsoft sells Copilot and still can only find about a third of us. The adoption number in your board deck is a bubble reading - here's the one the data supports.

94% of leaders rated their AI's code higher quality than their engineers'. 82% of them watched it break in production this year - same survey, same people. The review grade is the one signal you can't trust.

Rio de Janeiro's "sovereign AI" called itself Nex 79% of the time with the system prompt stripped — because it is, mathematically. The weights are 60% Nex-N2-Pro, and the math remembers where you got the parts.

METR ran the only controlled trial on AI developer productivity and found developers 19% slower. Then the follow-up broke — here's why the "2x faster" claim has no verified source and what that means for your next board meeting.