Securing AI-generated code

An AI assistant optimises for one thing: that the code does what you asked. You asked for a contact form, you get a working contact form. You did not ask for one that withstands ten thousand requests a minute carrying a malicious payload, so you do not get that.

This is not a flaw in the model, it is a consequence of its training. It learned from public code: example repositories, forum answers, tutorials. That corpus is overwhelmingly made of code written to illustrate a concept, not to survive the open internet.

Why generated code is unsafe by default

Three mechanisms compound. First the corpus bias already mentioned. Then the fact that security is almost always extra code: a validation, a permission check, a rate limit. A model aiming for the most probable answer produces the short version, which is the unsafe one.

The third mechanism is the most insidious: you cannot see what is missing. An injection flaw looks like nothing on screen. The form renders, the record saves, the tests pass. A missing check stays invisible until somebody goes looking for it.

The flaws that come back most often

Hardcoded secrets

The most frequent and the most expensive. When you ask for a database connection or an API call, the model produces a complete example, so with a literal value where the key goes. If you paste your real key to test and forget to extract it, it lands in Git history — and history survives the fix.

// Generated: the key ends up in the repository
const client = new ApiClient("sk_live_4eC39H...");

// Expected: the key stays out of the code
const key = process.env.API_KEY;
if (!key) throw new Error("API_KEY is missing");
const client = new ApiClient(key);

The habit to build: before a project's first commit, search for strings that look like keys. If a key has already been pushed, revoking it is the only answer — removing it from the code is not enough.

Queries built by concatenation

Models know parameterised queries perfectly well, yet readily produce concatenation when the query is dynamic — a variable sort order, an optional filter. Those are precisely the cases where injection becomes possible.

// Vulnerable: the value is pasted into the query
db.query("SELECT * FROM users WHERE email = '" + email + "'");

// Safe: the value travels separately
db.query("SELECT * FROM users WHERE email = $1", [email]);

The review signal is simple: any query containing a concatenation operator or an interpolated variable deserves a second look.

Validation missing, or client-side only

Asked for a form, an AI spontaneously adds the visible validation: required field, email format. That validation lives in the browser, where anyone can bypass it by sending the request directly. The validation that matters is the server's, and it is almost never generated without an explicit request.

Always ask for server-side validation with a typed schema, and check that it also rejects the cases you did not anticipate: missing field, unexpected type, ten-megabyte string.

Dependencies that do not exist

A model can invent a plausibly named package that does not exist. The phenomenon has a name, slopsquatting, because attackers realised they only need to watch which hallucinated names recur and publish genuinely malicious packages under them. The install command you were handed becomes the way in.

Permissions and CORS opened too wide

Facing a permissions problem, the shortest path is to open everything up. Models take that path: an allowed origin becomes the wildcard, an application role becomes the database superuser, a file becomes world-readable. It clears the immediate blocker and creates lasting exposure.

A review protocol in six points

Reviewing generated code calls for a different method than a conventional review: you are not looking for an error in what is written, you are looking for what is absent. These six questions cover most of it.

1. Where are the secrets? No sensitive value should appear in the code, nor in a committed example file.
2. Is every user input validated server-side, with a type and a maximum size?
3. Are queries parameterised without exception, including the dynamically built ones?
4. What happens if this route is called a thousand times a minute? Is there a rate limit?
5. Do error messages returned to the user reveal internal structure, a file path or a query?
6. Does every added dependency genuinely exist, and since when?

Let machines do the checking

Human review gets tired and lets things through. Two automations pay off heavily for little cost: a dependency scanner that flags vulnerable versions on every install, and a secret detector wired in before commit, refusing code that contains anything resembling a key.

A third approach works surprisingly well: ask a second model to hunt for flaws in the first one's output, explicitly casting it as an attacker. An AI reviews far better than it writes, because critiquing existing code is a more constrained problem than producing it.