Will software engineers be replaced by AI? The measured evidence available today says no. The most careful study on the question found something stranger than replacement: experienced developers using AI tools finished their work slower, while remaining convinced they had gone faster.
That gap between what gets measured and what gets felt is the most useful finding anyone has produced on this subject. It also explains why the replacement question keeps generating confident answers in both directions.
Here is what coding agents clear today, where they fall down, and what the evidence actually supports.
What an autonomous AI software engineer can actually do
An autonomous AI software engineer is an agent that takes a described problem and produces a working code change without step-by-step human direction. You hand it an issue. It reads the repository, edits files, and returns a patch.
The standard public measure is SWE-bench Verified: 500 real GitHub issues drawn from Python repositories, each confirmed solvable by a human engineer. The agent receives the issue text and the repository state. Its patch counts as resolved when the project's own test suite passes.
Frontier models now clear a substantial share of that set. The progress since 2023 is genuine.
The scores deserve an asterisk. Epoch AI, which runs an independent evaluation of the benchmark, reports an estimated error rate of 5–10% in the samples themselves and excludes 16 of the 500 for infrastructure reasons. Models are also instructed not to modify tests, so a conceptually sound refactor can register as a failure.
A benchmark that measures whether a patch passes a test is measuring whether a patch passes a test. Engineers have understood since roughly the invention of the unit test that this represents a floor on quality, with the ceiling sitting somewhere well above it.
Where the AI systems engineering problem actually sits
Benchmark tasks arrive pre-specified. Someone has already written down what is broken and what "fixed" means. That framing quietly removes the part of the job that consumes most of a senior engineer's week.
Production work looks different. The ticket says "checkout feels slow." The cause is a query pattern that only degrades above 400 concurrent sessions, and the fix touches a service another team owns.
Deciding what to build is the work. Typing it was already the fast part.
This is the AI systems engineering problem in a sentence: agents optimise the step you can specify, and specification is the expensive step.
Context makes it harder still. The repositories in METR's trial averaged over a million lines of code and more than 22,000 GitHub stars. A model reading that codebase cold has no access to the argument from 2019 that explains why the caching layer looks wrong deliberately.
Your agent will refactor that caching layer with total confidence. It will also be extremely polite about it, which somehow makes the 3 a.m. rollback worse.
Will engineering be replaced by AI? What the measurements show
Answering honestly means admitting how thin the evidence is.
In 2025, METR ran a randomized controlled trial with 16 experienced open-source developers across 246 real tasks in repositories they already maintained. Each task was randomly assigned to allow or forbid AI tools. The developers expected a 24% speedup.
They came out 19% slower. Afterwards, they still estimated that AI had sped them up by 20%.
METR now labels that result historical, which is a standard of intellectual honesty worth copying. They ran the experiment again with late-2025 agents and published the update in February 2026: a −18% speedup among the original developers and −4% among newly recruited ones, both with confidence intervals wide enough to cross zero.
That same update documents why the numbers are hard to trust. Developers declined to participate because they did not want to work without AI. Somewhere between 30% and 50% avoided submitting the tasks they expected AI to handle well. Time reporting degraded because people multitasked while waiting for agents to finish.
The same developers self-report gains of 1.4× to 2×.
Two conclusions survive all of that. The measured effect on experienced engineers working in mature codebases sits near zero, with real uncertainty around it. And nobody has yet produced a credible measurement showing the job disappearing.
If you want a research area with guaranteed long-term employment, measuring developer productivity appears to be wide open.
The roles that are actually hiring
Search behaviour reveals where the work is moving. Several job titles that barely existed in 2022 now draw steady monthly search volume:
- Applied AI engineer: builds product features on top of existing models and owns their evaluation and failure modes
- AI automation engineer: connects models into existing business processes and defines the human approval boundaries
- AI security engineer: handles prompt injection and data exfiltration among other model-specific attack surfaces
Every one of these is a software engineering role. They need someone who can read a stack trace and reason about distributed state, with model nondeterminism layered on top.
The pattern is consistent: the human moved up the stack toward specification and verification. Someone still decides what correct means. Someone still notices when confident output is wrong in a way the tests never catch.
If you have ever reviewed a pull request from a colleague who writes flawless syntax and completely misunderstands the requirement, you already have the core skill. The volume is about to go up considerably.
Machine learning for software engineers: what to learn
A research background is unnecessary here. Machine learning for software engineers is a practical subject, and its highest-leverage parts sit closer to testing than to mathematics. If you are still finding your footing with the tools themselves, our step-by-step introduction to Claude starts further back.
Start with evaluation. If you can define what a good output looks like for your feature and build a harness that scores it automatically, you can ship model-backed features responsibly. Most teams skip this step and discover the gap in production, which is an expensive place to learn it.
Then context design, meaning which information reaches the model and in what order. The context window, meaning how much text a model can hold at once, is a budget you allocate deliberately. Our guide to building and maintaining prompt systems covers the evaluation and testing side in depth.
Then agent architecture: tool design and the boundaries where a human approves an action before it executes.
Finally, read agent output the way you read a colleague's pull request. The characteristic failure mode of these systems is plausible wrongness, and plausible wrongness is precisely what a tired reviewer approves at the end of a long day.
The rule of thumb
Ask what fraction of your week goes to typing code you already fully understand. That fraction is what current tools compress. For most engineers past their second year, it turns out smaller than the discourse assumes.
The remainder stays yours: deciding what to build, and carrying the consequences when it breaks at 3 a.m.
Will software engineers be replaced by AI? The measured evidence says the job is changing shape while the headcount question remains genuinely open. Plan for the shape change. That is the part you can act on this quarter.