As some of the world\u2019s largest tech firms look to AI to write code, new research shows that relying too much on AI can impede skill acquisition in junior developers.<\/b>
\nAI may be able to help complete some types of work, like coding, faster. But it may come at the cost of actually getting better at what you’re doing and mastering new skills in the process, new research shows.
AI firm Anthropic, which develops the ChatGPT-competitor Claude, conducted the study on 52 junior software engineers. The participants were given a series of Python-based coding tasks, preceded by a short warm-up, and then quizzed on the skills they learned afterward. The whole process lasted about one hour and 15 minutes.
The researchers found that the group given AI assistance finished the tasks two minutes faster than the non-AI group, but generally performed far worse on the quiz afterward. The AI group averaged 50% on the post-task quiz, compared to 67% in the non-AI coding group. The largest gap in scores between the two groups was on debugging questions, where programmers were asked how to fix faulty code.
\u201cCognitive effort\u2014and even getting painfully stuck\u2014is likely important for fostering mastery,\u201d the researchers said. \u201cThis is also a lesson that applies to how individuals choose to work with AI, and which tools they use.\u201d
The study also found it wasn\u2019t just the fact that programmers used AI that impacted skill acquisition\u2014it was how they used it. Researchers identified several common patterns that cropped up among high and low performing groups on the post-task test.
The worst-performing participants had either entirely delegated all their coding to AI or started by attempting to code manually before calling in AI\u2019s aid. Users who employed AI to debug their code directly, rather than asking questions about where it went wrong, also generally performed poorly on the later test.
Meanwhile, programmers who asked AI questions about why the generated code worked\u2014and followed up with additional questions\u2014performed far better. Users who took a hybrid code-explanation approach, asking the AI to explain code as it generated it, performed better still. Users who asked only «conceptual» questions\u2014requesting explanations of concepts and issues rather than having the AI do the work directly\u2014performed by far the best on the test afterward.
The findings come as companies like Google and Microsoft set ambitious targets for incorporating AI into their code output, with Meta saying it plans for over 50% of its code to be written by AI. Even cutting edge NASA missions aren\u2019t immune from AI written code. In December, commands which were generated by Anthropic Claude, with human supervision, were sent to NASA\u2019s Perseverance Rover on Mars.
And while coders in this study were broadly faster when using AI, whether AI-assisted coding is actually quicker overall remains up for debate. A study from the AI research nonprofit Metr earlier this year found that AI actually slowed down the programmers it tested, as time spent prompting the AI equaled, or exceeded, the time saved by its assistance.<\/p>\n