What AI coding agents get wrong, measured

There is a particular moment in working with an AI coding agent where you stop reading the output. It happens around day three. The code keeps working, the explanations sound right, and reviewing every diff starts to feel like a waste of the thing you bought.

That moment is the whole risk, and there are numbers on it.

The code runs. That is not the same as correct

Veracode ran a longitudinal test across more than 150 large language models, using 80 coding tasks in four languages, checking the output with static analysis. Two findings matter.

The first: syntax correctness is now above 95%. The code runs. Whatever you asked for, you get something that executes.

The second: the security pass rate sat at roughly 55%. Close to half of the generated code introduced a known vulnerability — when the prompt contained no security instruction.

The gap between those two numbers is the job description. Producing code is solved. Producing code that is right is not, and the part that is not solved is the part that used to be yours.

Two details from the same testing are worth keeping:

Look at the last row for a second. Every model launch in that window was described as a leap. On this measurement, the needle did not move. Reasoning-focused models did better — roughly 70-72% — which is a real improvement and still not something you would ship unreviewed.

And look at the bottom of the list: cross-site scripting and log injection pass roughly one time in seven. Those are not exotic. They are what happens when user input reaches a page or a log without being escaped, which is most web features.

What to do about it

Two things, and the cheap one first.

Put the requirement in the prompt. The failures above were measured with no security guidance given. Naming the risk costs nothing:

Then verify against the categories that actually fail. Not "is this secure" — that gets you reassurance. Name the weak spots:

The full version of that check, with what a good answer looks like, is on the prove it works page.

It invents dependencies, and the inventions repeat

The second failure mode is stranger, and it has become an attack.

A study generating 576,000 code samples across 16 code models found that 19.7% of recommended packages did not exist — more than 205,000 distinct invented names. Open models hallucinated 21.7% of the time on average; commercial ones 5.2%.

The rate is not the interesting part. This is: rerunning the same prompts ten times, 43% of hallucinated package names came back every single time. The inventions are systematic, not random.

A name that is predictable is a name somebody can register before you ask for it. That is the attack, and it has a name — slopsquatting. A researcher demonstrated the mechanism by publishing an empty package under a plausible hallucinated name; it was downloaded tens of thousands of times.

Now add an agent that installs its own dependencies, and the one human checkpoint — noticing the package name in a suggestion and choosing to install it — disappears.

What to do about it

Those four questions separate a real library from a planted one. A genuine package has history: years of releases, a traceable publisher, a download pattern that did not start last month. A planted one has a plausible name and nothing behind it. You are not auditing code — you are checking whether the thing has a past.

Full version, with the habit that makes it stick: catching a package your AI invented.

It follows instructions it finds lying around

The third one has no percentage attached, because it is structural rather than statistical.

An agent reads your files, your dependencies' documentation, error output, issue threads and web pages you point it at. It cannot reliably distinguish your instruction from text inside that content which is phrased like one. A comment in a file, a line in a README, a string in an API response — all of it arrives in the same context window as your request.

The defence is a standing rule rather than a prompt:

It is not perfect. An agent told to report rather than obey is simply a much smaller target than one that was never told anything.

Worth pairing with an inventory, once per project:

Most people have never asked. The answer is usually broader than expected.

The part that is actually the work

None of this argues against using an agent. I build with one daily, and the leverage is real.

But the leverage is not where people think it is. Getting code out of the machine is the easy part now — that is what the 95% syntax number means. The leverage is in four or five prompts that catch what it got wrong before a client does: the circuit breaker when it starts looping, the dependency check before the install, the database question before launch, and the evidence demand before you believe anything is finished.

That is maybe ten minutes of prompting per project. Against a 45% failure rate on the security question, it is the highest-paid ten minutes in the workflow.

Where the numbers come from

• Veracode, Spring 2026 GenAI Code Security Update — 150+ models, 80 tasks across four languages, verified with static analysis.
• Package hallucination study — 576,000 generated samples across 16 code models; 19.7% non-existent packages, 43% of hallucinations reproducible across ten reruns.

Figures were current when this was written and are quoted as the sources state them. Look them up; that is the point of naming them.