OpenAI’s new lab benchmark shows progress and costly gaps in AI judgment

OpenAI and Ligo tested agents on real experimental evidence. Higher scores still left basic laboratory mistakes.
OpenAI introduced BioBasicsBench on November 6, an evaluation developed with enzyme-design company Ligo that puts AI agents through 100 routine wet-lab analysis tasks. The questions draw on researchers’ own experimental data, including images and instrument readings. OpenAI
At the highest reasoning setting shared by all seven tested models, GPT-6 Astra earned 66.7% of available credit, compared with 43.8% for GPT-5.6 Sol. Those numbers are average rubric scores, not the share of experiments completed successfully. The comparison covers OpenAI’s models, so it does not establish a lead over other developers. OpenAI
The examples show why a better score needs careful interpretation. Astra caught an antibiotic mismatch on a control plate that an older model accepted. In another task, however, it identified visual patterns correctly and then assigned them the wrong experimental meaning, recommending the wrong colonies. R&D World also highlighted a case where Astra overcounted colonies by about 12%. R&D World
The test gives partial credit for separate elements of an answer. Scientists reviewed each task internally and externally, and each model-and-setting combination received five attempts per task. That makes the evaluation useful for locating weaknesses, while keeping the result distinct from an independent test of a deployed laboratory assistant. OpenAI
For a laboratory considering these tools, the practical question is which errors a scientist would have to catch. A persuasive explanation can increase the review burden if its final recommendation quietly contradicts the evidence. An average score alone cannot tell a team how much supervision a particular workflow needs.
BioBasicsBench offers a concrete set of problems to investigate before handing agents more responsibility. A useful next step would be independent testing on other laboratories’ data, with attention to error severity and how reliably models flag uncertainty. That would help turn benchmark progress into a clearer case for everyday use.
Sources and citations
Revision history
Published .