OpenAI publishes new measurements of AI agents doing everyday wet-lab science

OpenAI's newest benchmark for everyday wet-lab science scores its top agent, GPT-6 Astra, at 66.7 percent of available credit, and that score is only the second-most interesting number in the report. The most interesting is a cloning exercise in which Astra correctly identified the visual groups on a plate and still picked the colonies that would have purified the wrong plasmid. 1 2

The sequence shows where the program is heading. The first had GPT-5 optimize a real molecular cloning protocol to 79 times its starting efficiency, work published with Red Queen Bio as a showcase of novel lab technique. 3 The second, LifeSciBench, arrived June 17: an expert-authored benchmark of 750 research tasks across seven biological domains, built with 173 scientists, on which even the strongest models cleared only about one task in three. 4 5 The third is deliberately mundane, 100 tasks of everyday bench reading built with biotech company Ligo. 1 Each step down in glamour has come with a step up in honest accounting, and the accounting is the story.

What was actually measured: 100 real experiments, 18,500 graded attempts

The tasks draw on evidence from real experiments instead of polished figures: gel images, colony plates, microscopy, sequencing files, instrument exports, and videos of bench work, across five areas of wet-lab science from protein methods and imaging to molecular cloning. 1 OpenAI evaluated seven of its models in 37 combinations of model and reasoning effort, five attempts at every task, for the 18,500 graded results the report counts. The arithmetic reconciles: 37 settings times 100 tasks times 5 attempts. 1 Scoring works by partial credit. Expert-written rubrics award points for conclusions, measurements, reasoning, and for recognizing when the evidence is insufficient to support a conclusion, so a partly-right answer scores differently from a right one. 1 Every task passed review by a second Ligo scientist, an external PhD-level wet-lab scientist, and a final internal reviewer. 1 AI Herald reports the benchmark is publicly available, which means outside evaluators can score any vendor's agent on the same tasks. 6

The headline number is a credit share, not a success rate

At xhigh, the highest reasoning effort, Astra earned 66.7 percent of available credit, with OpenAI's reported 95 percent confidence interval spanning 60.3 to 73.1. GPT-5.6 Sol earned 43.8 percent, spanning 37.6 to 50.0. 1 Two computed reads follow. The generation gap is solid: the bottom of Astra's range sits 10.3 points above the top of Sol's, so the ordering survives the uncertainty. And the frontier model leaves a third of the available credit unearned at maximum effort. Neither figure is the share of tasks completed correctly, a distinction the reporting is careful to make. 1 The documented wins are narrower and more useful to know: catching a control plate whose label names the wrong antibiotic, flagging dye interference in a cell-death readout, and recognizing when an image cannot settle a question. 1 AI Herald's read of the results adds that performance varied significantly by task type, with quantitative and protocol-based tasks proving harder than the agents' general reasoning record would suggest. 6

Three failure families, one pattern

The published failure examples sort into three families. 1

• Perception misses. On a colony-counting task Astra missed visible colonies, failed to separate close neighbors, and marked empty agar, landing roughly 12 percent above the benchmark's reference count. 1
• Control reversals. In a cloning task Astra assumed a larger DNA product indicated success and overrode the comparison the controls supported. It saw the groups correctly and got their meaning backwards. 1
• Evidence discipline. Astra described compartments inside cells as darker overall while discounting lighter examples elsewhere in the same image, even though it had noticed that focus shifted their appearance. 1

Set those failures beside the wins and one pattern covers both. Where the model is asked to notice that evidence is insufficient, it now often gets the call right. Where it must act on evidence it has already seen correctly, it can still conclude the opposite of what that evidence licenses. Bench competence, on this showing, is not a knowledge gap. It is a wiring problem between seeing and concluding.

Bench-ready or demo-ware

AI Herald's framing is the right procurement lens: the benchmark is a floor rather than a ceiling, because standardized tasks cannot capture a given lab's instrument quirks, protocol variations, reagent lot differences, and institutional knowledge. 6 For lab directors, the measured profile supports a specific division of labor: agents for triage and first-pass reading, humans for the judgment calls, because the documented failure mode is the confident wrong conclusion sitting downstream of a correct observation. 1 For anyone buying lab automation, the question to put to vendors follows directly: quote the credit share and the task success rate as separate numbers, and show the failure profile alongside the average. A vendor quoting only a benchmark percentage is selling the demo framing these measurements exist to replace.

The competence BioBasicsBench records is still partial: ready to read the evidence, not yet ready to say what it proves.