{"id":3256899,"date":"2025-07-15T23:39:31","date_gmt":"2025-07-15T21:39:31","guid":{"rendered":"http:\/\/nhub.news\/?p=3256899"},"modified":"2025-07-16T07:27:55","modified_gmt":"2025-07-16T05:27:55","slug":"surprise-developers-using-ai-take-19-longer-on-tasks","status":"publish","type":"post","link":"http:\/\/nhub.news\/ru\/2025\/07\/surprise-developers-using-ai-take-19-longer-on-tasks\/","title":{"rendered":"Surprise: Developers Using AI Take 19% Longer on Tasks"},"content":{"rendered":"<p style=\"text-align: justify;\"><b>Developers using AI tools took 19% longer to finish coding tasks, a METR study found, challenging assumptions about productivity gains in open-source work.<\/b><br \/>\nArtificial intelligence tools can slow down experienced open-source developers rather than accelerate their work, according to findings released by the nonprofit AI research group METR. <br \/>Developers and experts widely anticipated that using advanced tools would significantly improve productivity and shorten the time required to complete tasks in familiar codebases by approximately 24%; however, a randomized study by METR found that the opposite was true.<br \/>\u201cSurprisingly, we find that when developers use AI tools, they take 19% longer\u201d than without the cutting-edge tools, according to the study\u2019s authors.Benchmarks \u2018sacrifice realism\u2019<br \/>The study also pointed out that \u201cWhile coding\/agentic benchmarks have proven useful for understanding AI capabilities, they typically sacrifice realism for scale and efficiency.\u201d<br \/>The report\u2019s authors go on to explain that coding tasks are often \u201cself-contained\u201d and do not require prior context to understand the tools\u2019 capabilities. They also \u201cuse algorithmic evaluation that doesn\u2019t capture many important capabilities \u2026 These properties may lead benchmarks to overestimate AI capabilities.\u201d<br \/>Notably, in this study, the researchers observed that because benchmarks lack human oversight, AI models often stall on minor obstacles that a developer could quickly resolve in practical scenarios. Consequently, benchmarks may overestimate AI capabilities due to their design.   Future models may be better<br \/>While the slowdown appeared broadly consistent across tasks, the authors emphasized that the results could be specific to the environment tested. \u201cThese results do not imply that future models will not speed up developers in this exact setting,\u201d which is \u201ca salient possibility\u201d given the rapid progress being made to increase AI capabilities.<br \/>Nonetheless, the findings challenge a broad perception that AI always makes highly paid software engineers much more productive, which has prompted significant investment in vendors whose AI coding tools, known as \u201cvibe coders,\u201d will enhance the software development process.<br \/>\u201cOur results reveal a large disconnect between perceived and actual AI impact on developer productivity,\u2019\u2019 the study\u2019s authors said. \u201cDespite widespread adoption of AI tools and confident predictions of positive speedup from both experts and developers, we observe that AI actually slows down experienced developers in this setting.\u201d<br \/>Researchers also found that AI coding tools can occasionally introduce errors and security vulnerabilities.Methodology<br \/>To measure AI\u2019s real-world impact on developer productivity, METR organized a randomized controlled trial in which 16 developers completed 246 tasks across open-source repositories they regularly contributed to. Each task was randomly assigned to allow or prohibit AI assistance, and researchers recorded the time it took participants to finish under each condition.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Developers using AI tools took 19% longer to finish coding tasks, a METR study found, challenging assumptions about productivity gains in open-source work. Artificial intelligence tools can slow down experienced open-source developers rather than accelerate their work, according to findings released by the nonprofit AI research group METR. Developers and experts widely anticipated that using [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":3256898,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":[],"categories":[93],"tags":[],"_links":{"self":[{"href":"http:\/\/nhub.news\/ru\/wp-json\/wp\/v2\/posts\/3256899"}],"collection":[{"href":"http:\/\/nhub.news\/ru\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"http:\/\/nhub.news\/ru\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"http:\/\/nhub.news\/ru\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"http:\/\/nhub.news\/ru\/wp-json\/wp\/v2\/comments?post=3256899"}],"version-history":[{"count":0,"href":"http:\/\/nhub.news\/ru\/wp-json\/wp\/v2\/posts\/3256899\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"http:\/\/nhub.news\/ru\/wp-json\/wp\/v2\/media\/3256898"}],"wp:attachment":[{"href":"http:\/\/nhub.news\/ru\/wp-json\/wp\/v2\/media?parent=3256899"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"http:\/\/nhub.news\/ru\/wp-json\/wp\/v2\/categories?post=3256899"},{"taxonomy":"post_tag","embeddable":true,"href":"http:\/\/nhub.news\/ru\/wp-json\/wp\/v2\/tags?post=3256899"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}