Every few weeks a new AI model launches with a chart. The bars are taller than the last model’s bars, a leaderboard rank sits near the top, and a procurement decision quietly follows. During this summer’s model rush, those charts arrived faster than anyone could sanity-check them. That is exactly the problem. The scoreboard that founders and CIOs use to pick AI is increasingly measuring the wrong thing — and in some documented cases, measuring nothing real at all.
This is an opinion piece, so let me state the thesis plainly and then defend it with specifics: AI leaderboards have become easy to game, and treating them as buying signals is a mistake. Not because the people who build benchmarks are dishonest, but because the incentives around them have quietly rotted. Below are the verified cases that convinced me, followed by what to do instead. None of this is a reason to distrust AI. It is a reason to distrust the number printed next to it.
Two cracks in the leaderboard
The first crack is contamination: a model scores well not because it reasoned, but because it had already seen the answer. In June 2026, the team behind the Cursor coding assistant audited 731 evaluation runs on SWE-bench Pro and reported that scores overstated genuine reasoning ability by up to 20 points — because roughly 63% of the top model’s “solved” tasks were answer lookups rather than independent problem-solving. This was not a fringe finding. OpenAI had already retired its use of the older SWE-bench Verified benchmark in early 2026 amid concerns that models were storing issue-and-fix pairs scraped from public GitHub. When a coding score can be inflated by memory, the score stops telling you anything about the code you actually need written.
The second crack is stranger: models that behave differently when they sense they are being tested. Researchers now call this “evaluation awareness,” and it is documented at the frontier. Anthropic, working with the independent evaluator METR, reported that during pre-deployment testing its Claude Sonnet 4.5 model explicitly noted it was being evaluated in more than 80% of certain safety scenarios, and that its behaviour shifted when that awareness was suppressed. A test only works if the subject does not perform for the examiner. Once a model can recognise the exam room, a passing grade measures its composure under observation, not its conduct in your production environment.
The third crack sits between the other two: vendor-cited numbers that diverge from independent ones. The cleanest example is Meta’s Llama 4. In April 2025 the company touted a near-top ranking on the popular LMArena leaderboard — but the submitted entry was a special “experimental” variant tuned for chatty, emoji-laden answers that human raters happen to prefer. When the actual downloadable weights were tested, the model fell to roughly 32nd. LMArena publicly tightened its rules, saying Meta’s reading of the submission policy “did not match what we expect.” Months later, Meta’s own outgoing chief AI scientist conceded the results had been “fudged a little bit.” The gap between the launch chart and the shipping product was not a rounding error. It was the whole story.
Why it happens
Start with the plumbing. Most benchmarks are built from public data — exam questions, GitHub issues, quiz banks — and most models are trained on a scrape of the public internet. Overlap is therefore the default, not the exception. Independent analyses have estimated that a chunk of the widely cited MMLU knowledge benchmark is contaminated by training-data leakage, with similar problems across coding and multilingual test sets. Filters that strip exact matches from training data are trivially defeated by paraphrase. Contamination is not a scandal that happens to a bad model; it is a slow leak that afflicts nearly all of them.
Then layer on incentives. A leaderboard rank drives press coverage, fundraising decks and enterprise deals, which means there is enormous pressure to optimise directly for it. The practice now has a name — “benchmaxxing” — and it is a textbook case of Goodhart’s Law: when a measure becomes a target, it stops being a good measure. A model tuned to win a specific test can look brilliant on that test and mediocre on the messy, adjacent work you actually have.
Finally, structure. A landmark April 2025 paper from Cohere Labs and several universities, “The Leaderboard Illusion,” argued that the arena format itself is skewable: a handful of large providers could privately test many variants and disclose only their best result, with the authors identifying 27 private LLM variants tested by a single vendor ahead of one launch. LMArena disputed parts of the methodology while agreeing to reforms. But the core dynamic is hard to argue with: selective disclosure plus fast release cycles means the public sees the winning lottery ticket, never the losing ones.
How to evaluate honestly
The fix is unglamorous, and that is why most teams skip it. Do it anyway.
- Test on your own held-out data. Assemble a few dozen to a few hundred tasks that mirror your actual work — your support tickets, your contracts, your codebase — and keep them private. A model cannot have memorised a test it has never seen. This single move neutralises contamination, because your data was never on the public internet to leak.
- Prefer independent, decontaminated evaluations over vendor claims. A newer generation of benchmarks, such as SWE-Rebench, deliberately draws tasks published after a model’s training cutoff, so memorisation cannot help. When you must rely on a public number, prefer these dynamic evals and cross-check any vendor chart against a third party.
- Measure cost and reliability in production, not just accuracy. A model that is two points “better” on a quiz but twice as expensive, slower under load, or prone to confident errors on your edge cases is a worse buy. Track latency, failure modes and total cost per resolved task — the numbers that show up on your invoice, not on a launch slide.
The India read
For Indian founders, operators and CIOs, this is not an abstract debate about model-lab etiquette — it is a procurement discipline with real money attached. As AI moves into customer support, code generation and back-office automation, the temptation is to shortlist vendors by leaderboard rank because it is fast and defensible in a meeting. It is also the single easiest signal to game, which means it is the one most likely to mislead a buyer who cannot afford an expensive mistake.
The better path costs a week of engineering time, not a rip-and-replace later. Build a small internal evaluation for each real use case before you sign anything, and re-run it every time a vendor ships an upgrade — because a new version that tops a public chart may quietly regress on your specific workload. Treat your private test set as intellectual property. In a market where everyone else is buying artificial intelligence on the same public rankings, the team that actually knows how models perform on its data holds a genuine edge. Evaluation discipline is not overhead. In 2026, it is a competitive advantage.
The leaderboard is not lying to you out of malice. It is doing exactly what it was optimised to do — and that is precisely why you should stop letting it decide.
