OpenAI’s newest LLM, o3, is facing scrutiny after independent tests found it solved a far fewer number of tough math problems than the company first claimed. When OpenAI unveiled o3 in December, executives said the model could answer “just over a fourth” of the problems in FrontierMath,

Historically, each new generation of OpenAI's models has delivered incremental improvements in factual accuracy, with hallucination rates dropping as the technology matured. However, internal testing and

By OpenAI 's own testing, its newest reasoning models, o3 and o4 -mini, hallucinate significantly higher than o1.