One in three new arXiv papers now reads as AI-written, rigorous measurement study finds

One in Three New arXiv Papers Reads as AI-Written, Says a Study Unusually Honest About Its Own Blind Spots

Roughly a third of recent papers on arXiv read as machine-generated, according to a measurement study that scored 12,750 submissions. Computer science leads at 65%. Mathematics sits near zero. What makes those numbers credible rather than alarmist is how the study was built: around the one objection that guts most AI-detection claims.

Most "N% of content is now AI" headlines rest on detectors that also flag genuine human writing. If a tool marks 40% of new papers as machine-written but also flags 20% of papers from before ChatGPT existed, the real story is the 20% nobody mentioned. The team behind unslop, which publishes its academic-writing detector for free use, designed this study to neutralize that problem.

A false-positive anchor

The researchers set their flag threshold using papers submitted in 2021 and 2022, before ChatGPT, as ground-truth human text. They tuned it so just 0.4% of those pre-LLM papers register as machine-written 1. The pre-ChatGPT years then function as a built-in control: if the post-2022 rise were an artifact of the detector itself, the control period would flag just as high.

It does not. The trajectory is unmistakable: holding steady at 0.4% across the 2021 to 2022 control window, climbing within months of ChatGPT's arrival, then rising in two waves to roughly 32% in the most recent complete quarter, with a peak near 39% in early 2026 1. At its calibrated threshold, the tool correctly passes 99.6% of pre-LLM scientific writing as human and catches 85% of AI-generated academic text 1.

Two design choices sharpen the measurement. The team pulled version-1 PDFs for every paper, so a manuscript revised in 2026 cannot leak modern phrasing into a 2023 slot 1. They also scored full body text rather than abstracts, because abstracts understate the signal: the same paper can score under 20% on its abstract and over 70% on its body 1.

The field spread

The headline third conceals enormous variation across disciplines. Over the 12 months to July 2026, the flagged share by field: computer science 65%, quantitative biology 56%, electrical engineering systems 51%, economics and finance 47%, applied physics 34%, statistics 31%, condensed matter 24%, high-energy physics 14%, astrophysics 11%, and mathematics 0.7%, each figure carrying a bootstrap 95% confidence interval 1. The fields that rose most after ChatGPT are not the ones with the highest pre-LLM control rates 1. That separation rules out a baseline artifact bleeding upward into the results.

Where the detector goes deliberately blind

The mathematics result, at 0.7%, looks like one field holding the line. The study's authors do not make that claim, and their candor here is the most instructive part of the entire report.

Math papers are dominated by notation and theorem-proof structure. Once equations and references are stripped out, what remains is sparse and unfamiliar to the detector's training distribution 1. A mathematics paper drafted with heavy model assistance could score low simply because its text falls outside what the detector recognizes.

The 0.7% figure is therefore consistent with two radically different explanations: mathematicians adopt AI less, or the detector cannot see AI in that register. The data cannot distinguish between them 1.

This confound cuts both ways. The fields where the detector operates most confidently are the prose-heavy ones, which are also the fields that rise most. So the aggregate trend holds. But in low-scoring fields, the rankings should be read as a lower bound on actual adoption, not a precise count 1.

Two further limitations round out the picture. The detector is more sensitive to some language models than others, and the team cannot evaluate it against the private mix of tools and prompts that authors actually use 1. Incomplete coverage means the reported prevalence is a floor, not a ceiling. And a flag estimates whether text reads as machine-written at a calibrated probability. It cannot separate lightly edited AI output from fully generated text, and a single score is never sufficient to accuse a specific author of misconduct 1.

So what

In computer science, quantitative biology, and electrical engineering, a majority of recent arXiv submissions now carry the statistical fingerprint of machine writing. Whether that reflects heavy AI-assisted editing or wholesale generation, this study cannot resolve at the level of an individual paper.

The 0.4% anchor, the confidence intervals, the candid accounting of where the instrument loses traction: these are the features that make the 32% figure mean something, even if what it means is less certain than the shock headline implies.

The gap between what we can measure and what is actually happening in the scientific record is real. The study also makes clear that closing it entirely is not yet possible, and the authors know it.