Image missing.
The "First AI Software Engineer" Is Bungling the Vast Majority of Tasks It's Asked to Do

Victor Tangermann

created: Jan. 24, 2025, 4:19 p.m. | updated: March 19, 2025, 5:36 p.m.

<p>Researchers have found that AI tech company Cognition's Devin, which it claims to be the "first AI software engineer," is astonishingly bad at its job. In a recent analysis, a team of machine learning data scientists behind the independent AI research and development lab Answer.AI spent a month with the AI assistant, concluding that despite almost a year of hype, it "rarely worked." "Out of 20 tasks we attempted, we saw 14 failures, three inconclusive results, and just three successes," the researchers found. "More concerning was our inability to predict which tasks would succeed," they wrote. "Even tasks similar to […]</p>

5 months, 2 weeks ago: Futurism