A team of researchers at the AI evaluation company Andon Labs put a large language model in charge of controlling a robot vacuum. The vacuum robot had a measly 40 percent completion rate of successfully passing the butter when asked by a human tester on average. “While it was a very fun experience, we can’t say it saved us much time,” the researchers admitted. “Although LLMs have repeatedly surpassed humans in evaluations requiring analytical intelligence, we find humans still outperform LLMs on Butter-Bench,” the company wrote. More on robot AIs: Chinese Unleashing AI-Powered Robot Dinosaurs