Model Evaluation and Threat Research is an AI research charity that looks into the threat of AI agents! That sounds a bit AI doomsday cult, and they take funding from the AI doomsday cult organisat…
The entire article is based on the flawed premise, that “AI” would improve the performance of developers. From my daily observation the only people increasing their throughput with “AI” are inexperienced and/or bad developers. So, create terrible code faster with “AI”. Suggestions by copilot are >95% garbage (even for trivial stuff) just slowing me down in writing proper code (obviously I disabled it precisely for that reason). And I spend more time on PRs to filter out the “AI” garbage inserted by juniors and idiots. “AI” is killing the productivity of the best developers even if they don’t use it themselves, decreases code quality leading to more bugs (more time wasted) and reducing maintainability (more time wasted). At this point I assume ignorance and incompetence of everybody talking about benefits of “AI” for software development. Oh, you have 15 years of experience in the field and “AI” has improved your workflow? You sucked at what you’ve been doing for 15 years and “AI” increases the damage you are doing which later has to be fixed by people who are more competent.
It’s a fair statement and personal experience, but a question is, does this change with tool changes and user experience? Which makes studies like OP important.
Your >95% garbage claim may very well be an isolated issue due to tech or lib or llm usage patters or whatnot. And it may change over time, with different models or tooling.
I’ll quote myself from some time ago:
deleted by creator
It’s a fair statement and personal experience, but a question is, does this change with tool changes and user experience? Which makes studies like OP important.
Your >95% garbage claim may very well be an isolated issue due to tech or lib or llm usage patters or whatnot. And it may change over time, with different models or tooling.