AI-generated code can work well while remaining difficult for engineers to explain. That is useful until someone needs to inspect a change, diagnose a failure, or decide what the system should build next.
From writing code to supervising it
Jeremy Nixon, founder of Infinity, says programmers are increasingly losing sight of what language models produce. In his account, developers are moving away from building products line by line and toward supervising the model’s work.
A striking example comes from OpenAI. Jordan Nanos, a technical employee at SemiAnalysis, described engineers there reviewing GPU kernel code written to improve performance. They reportedly struggled to follow what the code did, even as the kernels produced correct, highly effective results.
Nanos sees that not as evidence that the engineers are incapable, but as a change in how the work is divided. Developers set the architecture and decide what counts as an acceptable result, while AI handles more of the implementation. The code may pass its tests, even if the people responsible for it cannot readily explain every part.
That arrangement changes the meaning of technical oversight. Reviewing a result is not quite the same as understanding how it was made, though the two can look similar when everything works.
The production risk
Working code is not automatically safe code. AI systems can make mistakes or produce plausible but false outputs, and those errors can reach production if engineers do not catch them.
Amazon introduced a 90-day code safety test earlier this year after a series of failures disrupted order processing. The company denied that AI was responsible. Internal logs and notes, however, showed managers concerned about the large potential impact of AI-generated changes.
The account also cites figures from software company Undo. Its analysis found that 35% of AI-generated code goes into production without engineers fully understanding it. In the same analysis, 29% of developers said they lose productivity several times a month while untangling AI-written code, and 94% said they had experienced that at least once in the previous six months.
Those figures point to a distinction that is easy to blur: generating code may save effort at one stage while creating more work later. The amount of that extra work, and how the figures were collected, is not detailed in the report.
A skills question, not just a tooling question
The concern extends beyond whether a particular code change is safe. The report also points to research describing cognitive decline and skill loss among specialists who stop writing code by hand without developing the skills needed to design larger systems.
That leaves developers with a difficult balance. If they rely on models for more implementation, they may have more time to focus on architecture and acceptance criteria. But those responsibilities require enough technical understanding to judge whether the model’s output is sound, especially when the tests do not expose a problem.
The practical challenge is not that engineers must personally type every line. It is that responsibility does not disappear when authorship shifts to a model. Teams still need ways to review changes, spot failures, and maintain the knowledge required to make decisions about complex systems.
AI may be changing programming from writing instructions to directing and checking them. The uncomfortable part is that a program can be correct today and still be poorly understood by the people expected to fix it tomorrow.