Almost every software team now uses AI.
That tells you remarkably little.
One developer might use an AI coding assistant to write tests. Another has three agents changing different parts of the codebase. Someone else pastes problems into ChatGPT. Your most experienced engineer may have built a sophisticated workflow around context, automated testing and specialist agents.
From the outside, leadership sees the same thing:
“Our engineering team uses AI.”
But these are completely different levels of capability.
And that difference matters commercially.
Used well, AI can help teams investigate problems, write and test software, modernize old systems, document code and deliver changes significantly faster.
Used badly, it can simply help an immature engineering team create more code, more inconsistency and more technical debt at greater speed.
That isn’t just our opinion. Google’s DORA research describes AI as an amplifier: strong organizations tend to get stronger, while weaknesses in struggling engineering systems can also be amplified. Its research has found higher AI adoption associated with greater delivery throughput, but also greater instability when the underlying engineering environment is weak.
So before asking:
“Are our developers using AI?”
Leadership should be asking:
“Has AI made our engineering function better?”
Here are seven of the worst ways to find out the answer is no.
The 60-second version
| The mistake | What it looks like | What mature teams do instead |
|---|---|---|
| 1. Everyone does AI their own way | Different tools, prompts, standards and workflows | Create a shared AI engineering system |
| 2. Use AI to produce more code | More commits = more productivity | Optimize for useful software and business outcomes |
| 3. Give AI bad requirements | Build first, think later | Improve the problem definition before accelerating delivery |
| 4. Let AI outrun your ability to verify it | Huge changes nobody fully understands | Make testing and verification part of the AI workflow |
| 5. Give agents freedom without boundaries | Convenience beats control | Design safe environments for autonomous work |
| 6. Use AI to compensate for weak engineering | Junior capability + AI = assumed senior capability | Combine AI with stronger technical judgment |
| 7. Bolt AI onto the old engineering model | Same process, just faster typing | Redesign how engineering works around AI |
If several of these sound familiar, the problem probably isn’t which AI tool you bought.
It is the engineering system around it.
1. Let every engineer figure AI out for themselves
This is probably the most common approach.
Buy some licenses.
Tell the engineering team they are encouraged to experiment.
Then leave everyone to develop their own way of working.
It feels progressive because lots of experimentation starts happening.
But six months later you may have:
- four different AI tools;
- five different approaches to prompting;
- engineers giving AI completely different levels of context;
- different standards for checking generated code;
- no agreement about which tasks agents should perform;
- no shared way to measure whether any of it is actually helping.
Some engineers become dramatically more capable.
Others essentially have a faster autocomplete.
And leadership has very little visibility into the difference.
The coordination problem
Five engineers can all be individually good at using AI and still create an inconsistent engineering system.
One may use agents heavily. Another may rely on autocomplete. Another may generate large chunks of code and review them manually. Another may build elaborate prompting workflows. Another may barely use AI at all.
Individually, each approach might be perfectly reasonable.
The problem appears when nobody has designed how those approaches fit together.
The company becomes dependent on the habits and judgment of individual engineers rather than building a repeatable capability. Quality varies by person, knowledge becomes harder to transfer and leadership has no reliable way to know whether AI is actually improving engineering performance.
The problem is not experimentation.
The problem is when experimentation never becomes an organizational capability.
A mature engineering function should give engineers room to experiment while agreeing on the things that affect everyone: how AI is used, where its limits are, how work is verified and what good looks like.
Better approach
Instead of simply encouraging AI adoption, create shared answers to questions like these:
| Question | What a mature team should know |
|---|---|
| Which AI tools do we use? | Which tools are approved, and what each is best used for |
| What context should agents receive? | How the codebase, architecture, requirements and business context are provided |
| Which tasks can agents complete independently? | Where autonomy is useful and where human judgment is still required |
| What requires human approval? | Which decisions or changes carry enough risk to require review |
| How is AI-generated work tested? | The checks that must pass before work is accepted |
| How are important AI-assisted decisions documented? | How another engineer can understand why something was built this way |
| How do we know AI is helping? | Whether delivery, quality, reliability or commercial outcomes are actually improving |
Individual experimentation should continue.
But it should feed a shared engineering capability, not permanent chaos.
2. Use AI to produce more code
This one sounds sensible.
If AI helps a developer write twice as much code, surely engineering productivity has increased?
Not necessarily.
Code is a cost, not an outcome.
Every additional line can eventually need to be maintained, tested, secured, understood, modified, migrated or removed.
An engineering team does not become more valuable because it produces more software.
It becomes more valuable when it solves more valuable problems.
Consider two engineers.
Engineer A
Uses AI to build a new internal workflow in three days.
Engineer B
Spends half a day investigating the requirement and discovers that an existing platform can solve 90% of the problem with configuration.
Engineer A produced substantially more code.
Engineer B may have created substantially more value.
This distinction becomes even more important as AI makes producing software easier.
DORA warns of exactly this type of feature-factory problem: without a strong focus on user and business outcomes, AI can simply accelerate teams in the wrong direction.
Better question
Don’t ask:
“How much faster are our developers coding?”
Ask:
“What has become faster, cheaper or commercially possible because our engineering function uses AI?”
Those are very different questions.
3. Give AI bad requirements and expect good software
AI is extraordinarily good at making a bad idea look like progress.
Give an agent a reasonably clear instruction and it may quickly produce:
- a database;
- an interface;
- an API;
- tests;
- documentation;
- integrations;
- deployment configuration.
That can create a dangerous illusion.
Something exists, therefore progress has been made.
But AI cannot rescue a team from asking the wrong question.
If the business requirement is unclear, the architecture is poorly understood or nobody has challenged whether the feature should exist in the first place, faster development simply gets you to the wrong destination sooner.
The immature workflow
Request → AI → Code
The stronger workflow
Business problem → investigation → constraints → options → technical decision → AI-assisted delivery → verification → measurement
The second workflow looks slower at the beginning.
It is often dramatically faster by the end.
This matters particularly when…
- modernizing legacy software;
- introducing AI into an existing product;
- replacing critical internal systems;
- integrating platforms;
- rebuilding failed projects;
- making architecture decisions;
- automating complex operational processes.
These are environments where understanding the system is often harder than writing the code.
And AI has made writing the code the easy part.
4. Let AI produce changes faster than your team can understand them
AI can now generate surprisingly large amounts of technically plausible software extremely quickly.
That creates a new bottleneck:
Verification.
- Who knows whether the code is right?
- Who understands what changed?
- Who knows what assumptions the agent made?
- What happens when the test passes but the implementation is still inappropriate?
- What happens six months later when a different engineer needs to modify it?
The answer cannot simply be:
“A senior developer reviews it.”
If agents dramatically increase the volume of work being produced, traditional manual review can itself become the constraint.
Current research illustrates why this matters. METR’s early-2025 study found experienced developers working on familiar repositories actually took 19% longer with the AI tools available at that time, despite believing AI had made them faster. Newer 2026 evidence suggests modern agentic tools are now producing genuine speed improvements, although researchers caution that accurately measuring the size of that improvement remains difficult.
The lesson isn’t that AI makes developers slower.
The technology has already moved far beyond that simple conclusion.
The more important lesson is:
Perceived productivity and actual engineering performance are not automatically the same thing.
Mature AI engineering needs verification designed into the system
That can include:
- automated tests;
- architecture rules;
- code-quality checks;
- security scanning;
- clearly defined acceptance criteria;
- agent evaluations;
- staged environments;
- human approval at important decision points.
The objective is not to stop agents working quickly.
It is to make safe speed possible.
5. Give autonomous agents access first and work out the controls later
There is a big difference between:
AI suggesting some code
and
an AI agent able to change files, run commands, access systems and execute tasks independently.
The latter is dramatically more powerful.
It also changes the risk.
A useful analogy is hiring someone.
You would not normally give a new employee unrestricted access to:
- production infrastructure;
- customer information;
- deployment systems;
- credentials;
- and every internal tool
simply because they’re productive.
You decide what they need.
You create boundaries.
You provide environments where mistakes are recoverable.
Agentic engineering requires the same thinking.
Even companies building the tools themselves increasingly design around concepts such as sandboxing, restricted permissions and controlled access because greater autonomy increases the potential consequences of mistakes.
Bad question
“How autonomous can we make the agent?”
Better question
“Within what boundaries can this agent operate autonomously?”
That small change in language leads to much better engineering decisions.
6. Use AI to make weaker engineers look like stronger engineers
This may be the most uncomfortable one.
AI has dramatically reduced the barrier to producing convincing technical work.
Someone can now generate:
- sophisticated-looking architecture;
- database designs;
- complex functions;
- tests;
- infrastructure configuration;
- security recommendations;
- detailed technical explanations.
The output can look senior.
That does not mean the person directing it has senior judgment.
And eventually somebody has to decide:
- Is this the right architecture?
- Is this abstraction necessary?
- Will this approach scale?
- Are we solving the right problem?
- What are the consequences of changing this system?
- Should we build this at all?
Those questions become more important as AI becomes better at implementation.
Not less.
AI changes where engineering value sits
Historically, a large amount of engineering effort went into producing the implementation.
Increasingly, AI can help with that work.
That pushes human value upward toward:
- judgment;
- architecture;
- decomposition;
- validation;
- system understanding;
- and commercial decision-making.
A weaker engineering team with AI may therefore become much more productive at executing decisions without becoming any better at making decisions.
That is a dangerous combination.
7. Bolt AI onto your old engineering process
This is probably where the largest opportunity gets missed.
A company adopts Copilot, Claude Code, Codex or another coding agent.
Developers become faster.
Perhaps substantially faster.
But almost everything around them stays the same:
- The same ticket process.
- The same planning.
- The same QA.
- The same architecture documentation.
- The same deployment process.
- The same way requirements move through the company.
- The same division of responsibilities.
That is like introducing the internet to your company and using it primarily to send your fax documents faster.
You have adopted the technology.
You haven’t redesigned around what the technology makes possible.
An AI-native engineering function asks a different question
Not:
“Where can we add AI?”
But:
“If today’s AI capabilities had existed when we designed our engineering function, would we still have designed it this way?”
Often the answer is no.
AI-native engineering may change:
- how requirements are investigated;
- how codebases are documented;
- how problems are decomposed;
- how prototypes are built;
- how tests are generated;
- how code is reviewed;
- how technical debt is identified;
- how legacy systems are understood;
- how specialist agents are used;
- how developers share context;
- how releases are verified;
- and ultimately how many people are required to achieve an outcome.
That’s much more significant than giving every engineer an AI subscription.
So how AI-native is your engineering team?
Use this as a quick leadership test.
| Question | Immature AI adoption | AI-native engineering |
|---|---|---|
| How do engineers use AI? | However they prefer | Shared system + controlled experimentation |
| What determines the tool? | Personal preference | Task and engineering requirements |
| What context does AI receive? | Whatever the developer provides | Designed and maintained context |
| What does success mean? | More/faster code | Better commercial and engineering outcomes |
| Who validates AI work? | Individual developer | Layered automated + human verification |
| How much can agents do? | Depends on the engineer | Explicit autonomy boundaries |
| Are workflows standardized? | Rarely | Where standardization improves quality |
| Is AI usage measurable? | Mostly anecdotal | Outcomes and workflow performance measured |
| Does AI reduce technical debt? | Unknown | Deliberately managed |
| Could leadership explain the AI engineering model to an acquirer? | Probably not | Yes |
You do not need every box on the right.
But if almost everything sits on the left, you probably don’t yet have an AI strategy.
You have engineers using AI tools.
Why this matters beyond engineering
For CEOs and COOs, this may sound like an internal technical issue.
It isn’t.
Engineering maturity affects:
- delivery speed;
- product reliability;
- security;
- technical debt;
- operating cost;
- your ability to launch new products;
- your dependence on individual employees;
and eventually:
how valuable the business is to somebody else.
If a potential acquirer examines the technology behind your company, they’re unlikely to be impressed simply because your engineers use cutting-edge AI tools.
What matters is what those tools have helped create.
- Can the software evolve?
- Can another team understand it?
- Are engineering decisions documented?
- Are systems dependable?
- Can the company release changes safely?
- Is technical debt under control?
- Does important knowledge live inside the business rather than inside one engineer’s head?
- Has AI improved the company’s technical capability or simply increased its output?
AI can help build a more valuable software business.
But only if the engineering system around it is mature enough to turn increased capability into commercially useful software.
The uncomfortable conclusion
The worst way to use AI in software engineering may actually be the simplest:
Assume that because your engineers use AI, your engineering function has become good at AI.
The tools are becoming extraordinarily capable.
That makes the people, processes and technical systems around them more important, not less.
There is an enormous difference between:
five engineers independently using AI
and
an engineering organization designed to take advantage of AI.
One is tool adoption.
The other is a capability.
And as AI gets better, that difference is likely to become increasingly visible in the quality, speed and commercial value of the software companies produce.
Are you using AI, or building an AI-native engineering capability?
Supervillains works with organizations building, modernizing and scaling commercially important software, including teams trying to understand how AI should change the way their engineering function operates.
If you suspect your developers are using AI but nobody has really designed how the company should engineer software with AI, that is a useful conversation to have.