
OpenAI gets caught vibe graphing
Jay Peters
created: Aug. 7, 2025, 9:33 p.m. | updated: Aug. 8, 2025, 1:35 p.m.
During its big GPT-5 livestream on Thursday, OpenAI showed off a few charts that made the model seem quite impressive — but if you look closely, some graphs were a little bit off.
In one, ironically showing how well GPT-5 does in “deception evals across models,” the scale is all over the place.
OpenAI appears to have accurate numbers for this chart in its GPT-5 blog post, however, where GPT-5’s deception rate is labeled as 16.5 percent.
With this chart, OpenAI showed onstage that one of GPT-5’s scores is lower than o3’s but is shown with a bigger bar.
An OpenAI marketing staffer also apologized, saying, “We fixed the chart in the blog guys, apologies for the unintentional chart crime.”OpenAI didn’t immediately respond to a request for comment.
1 week, 3 days ago: The Verge