DeepMind's $200K 'Measuring AGI' Kaggle hackathon awarded a $25K grand prize to a submission the data-science community is calling blatant AI-generated slop

Google DeepMind's AGI Benchmark Contest Awarded $25K to a Submission the Community Calls AI-Generated

Google DeepMind launched a $200,000 Kaggle hackathon to build better tests for artificial general intelligence. It awarded a $25,000 grand prize to a submission that data scientists are now calling AI-generated filler. And the contest judges handed over the money.

That failure is the story, not the hackathon drama. A competition whose explicit purpose was to rigorously evaluate AI cognitive abilities could not tell whether its own benchmark submissions were written by humans or generated by machines. If the pipelines designed to measure AI are already contaminated by AI output, every leaderboard they produce inherits that ambiguity.

The setup is straightforward. In March 2026, Google DeepMind published "Measuring Progress Toward AGI: A Cognitive Taxonomy," a paper identifying 10 cognitive abilities the researchers hypothesize matter for general intelligence, spanning perception, memory, reasoning, metacognition, problem solving, and social cognition 1. To turn that framework into usable evaluation tools, DeepMind partnered with Kaggle on a hackathon with a total prize pool of $200,000. Four grand-prize winners would each receive $25,000, with additional $10,000 awards for top performers across five specialized tracks 1.

The competition targeted five cognitive abilities where the evaluation gap was widest: learning, metacognition, attention, executive functions, and social cognition 1. Participants used Kaggle's Community Benchmarks platform to build and test their evaluations against frontier AI models 1. Submissions opened March 17 and closed April 16, with results announced June 1 1.

DeepMind's proposed evaluation protocol is three-staged: evaluate AI systems across cognitive tasks using held-out test sets, collect human baselines from a demographically representative sample, and map each system's performance relative to the human distribution 1. The ambition is genuine. The architecture is careful. But the protocol assumes a clean boundary between human-authored tests and machine output. That boundary did not hold.

When the winners were named, the data-science community responded quickly. A discussion thread on the Kaggle competition page and a Hacker News post both flagged at least one grand-prize submission as exhibiting hallmarks of LLM-generated text rather than original human work 23. Community members in both threads identified patterns they associated with machine-generated output, arguing the submission read like LLM filler rather than a thoughtfully designed cognitive benchmark 23.

This is where the episode stops being about one contest and becomes a structural problem. AGI benchmarks are only as trustworthy as the human-designed tasks inside them. If evaluators cannot tell whether a benchmark was written by a person or produced by an LLM, the measurement chain inherits that ambiguity at every level. A model tested on AI-generated benchmarks is being evaluated against a yardstick that might itself be the product of an AI system.

Detection tools exist, which makes the oversight harder to excuse. An independent analysis demonstrated that LLM-generated text carries strong statistical patterns detectable by classical machine learning methods, achieving roughly 85% single-sentence accuracy using a scikit-learn SVM pipeline 4. The same analysis notes that mainstream LLM output exhibits word-choice patterns consistent enough that even a Naive Bayes classifier can separate AI text from human writing 4. If detection is feasible but nobody applied it before awarding $25,000, the gap is procedural, not technological.

DeepMind's own framework acknowledges contamination as a concern. The paper proposes using held-out test sets specifically to prevent data contamination 1. But that safeguard targets AI models memorizing their training data. It does not address the inverse problem: AI-generated content infiltrating the human side of the evaluation pipeline. The framework guards the test-taker against seeing the answers. It has nothing to say about who wrote the questions.

The implications ripple outward. Every benchmark leaderboard assumes its tasks were designed by people who understand what specific cognitive abilities look like when tested. If that assumption breaks, the rankings stop measuring what they claim to measure. They start measuring how well a model mirrors the statistical fingerprints of whatever system generated the test. You are not evaluating intelligence. You are evaluating recursive imitation.

The hardest problem in measuring AGI may not be designing the right tests. It may be proving the tests were written by a human.