September 17, 2025
Estimated reading time: 2 minutes
A new Queen’s University study throws up warning signs for those relying on AI output without careful checking.
A new analysis of 82,845 chat logs shows that popular coding assistants often reply at great length – typically 14 times longer than the human prompt – and much of the code they produce contains basic errors.
Researchers from Queen’s University in Canada looked at the real-world conversations from the WildChat corpus of developer conversations with ChatGPT, containing 368,506 code snippets in more than 20 programming languages. Nearly seven in ten conversations involved some form of back and forth, often because users shifted goals mid-thread or had to clarify missing details.
Join LeadDev.com for free to access this content
September 17, 2025
