When AI Invents APIs: Handling Code Hallucinations in Practice

AI coding assistants hallucinate code. Not occasionally — routinely. The model confidently writes a call to fs.readFileSync with a { encoding: 'utf8', watch: true } option that doesn’t exist, or imports from @prisma/client/extensions in a version that predates the feature, or calls response.blob() on a Node fetch response in an environment where that method isn’t available. The code looks plausible. It often type-checks. It fails at runtime.

This is a structural property of language models, not a bug that better models fully eliminate. Models predict tokens based on patterns, and plausible-looking code is rewarded by that objective regardless of whether the specific method exists. The gap between “looks right” and “is right” is where hallucinations live.

The practical response is not to distrust AI-generated code wholesale — it’s to build workflows that surface hallucinations cheaply before they cost you.

What hallucinated code looks like

The pattern is almost always the same: a method call or import that looks like it belongs, surrounded by real code.

// Real: readFileSync accepts an encoding option
const raw = fs.readFileSync('./config.json', { encoding: 'utf8' });

// Hallucinated: the 'watch' option doesn't exist on readFileSync
const raw = fs.readFileSync('./config.json', { encoding: 'utf8', watch: true });

TypeScript catches the second case at compile time if @types/node is installed. Without types, the unknown property is silently ignored — or the implementation throws if it validates options strictly.

Hallucinations are worst in three situations:

• Rapidly evolving libraries: the model’s training data includes old versions, new versions, and blog posts about features that were proposed but never shipped.
• Less popular packages: the model has seen less real usage and extrapolates from analogous APIs.
• Composing unfamiliar APIs: the model understands each piece individually and invents a plausible composition that doesn’t exist.

Prevention: ground the model in real source

The most effective prevention is to give the model the actual interface, not just the package name.

Before asking for code that uses a library, paste the relevant type signatures from its source or documentation:

Here are the relevant types from the Stripe SDK (v14.5.0):

  interface PaymentIntentCreateParams {
    amount: number;
    currency: string;
    payment_method?: string;
    confirm?: boolean;
    return_url?: string;
  }

  interface Stripe {
    paymentIntents: {
      create(params: PaymentIntentCreateParams): Promise<PaymentIntent>;
      confirm(id: string, params?: PaymentIntentConfirmParams): Promise<PaymentIntent>;
    };
  }

Write a createAndConfirmPayment function that creates and immediately confirms a payment intent.

Pasting the actual types costs 30 seconds. It grounds the model in what’s real rather than what’s plausible, and the TypeScript compiler validates the result anyway.

For quickly verifying which methods a package actually exports, a one-liner works:

node -e "const m = require('some-package'); console.log(Object.keys(m))"
# or for ESM:
node --input-type=module -e "import * as m from 'some-package'; console.log(Object.keys(m))"

Paste the output into the context and the model has concrete ground truth.

Pin the version explicitly

Hallucinations spike when the model is uncertain which API version you’re using. A prompt that says “use Prisma” leaves the model guessing between the v4 client, the v5 client, and various extension patterns that exist in training data at different density levels.

Be explicit:

I'm using Prisma 5.9.0 with the standard generated client. Do not use
prisma-client-extensions or the $extends API — we haven't opted into that.

This narrows the model’s sample space significantly. “I’m using Express 4.x, not 5.x” prevents it from reaching for app.use(express.asyncErrorHandler()), which exists in 5.x but not 4.x. Version pinning is cheap to add and meaningfully shifts the distribution toward real APIs.

Detection: your fastest feedback loops

Hallucinated code surfaces at different layers of the feedback stack. From fastest to slowest:

TypeScript (immediate): If your types are installed and strict mode is on, most hallucinated method calls fail at compile time. This is the best case — a red squiggle before you run anything. It’s also the strongest argument for keeping TypeScript strict and type definitions up to date in any project where you use AI-generated code extensively.

Import resolution (seconds): A hallucinated module or package subpath fails at the import stage.

# Fails fast if the subpath doesn't exist
node -e "require('some-package/nonexistent-subpath')"

Linting (fast): Some linters catch patterns like calling non-existent methods on well-known types. eslint-plugin-n catches Node.js API misuse and version mismatches worth flagging.

Running the code (minutes): The slowest loop, but unavoidable for runtime-only behaviors. If you suspect a generated function has a hallucinated API call, write the smallest possible smoke test before building anything on top of it:

// Run this before wiring the function into five other places
import { myGeneratedFunction } from './generated.ts';
const result = await myGeneratedFunction({ id: 1 });
console.log(result);

If it throws TypeError: X is not a function, you’ve found the hallucination in 30 seconds rather than after three layers of integration.

Recovery: correcting the model effectively

When you find a hallucination, the correction prompt matters. Vague complaints produce vague improvements.

Weak correction:

That method doesn't exist, please fix it.

Strong correction:

`response.blob()` is not available in Node's native fetch — that's a browser API.
In Node, use `Buffer.from(await response.arrayBuffer())` to get the raw bytes.

Update the function to use that instead.

The strong form tells the model what’s wrong, why, and what the correct alternative is. The model doesn’t have access to your runtime errors — it can’t see what actually failed. The more precisely you describe the failure, the less it has to guess in the correction. This is the same principle as the feedback loop in Debugging with LLMs — you’re the one with runtime access, the model needs you to surface what it can’t see.

One trap worth naming: if the model offers a correction that also looks wrong, don’t keep patching in the same message thread. Start a fresh context, paste the real API types, and ask again. Continuing in the same thread carries along the model’s initial hallucination as context, and it will often reintroduce it — sometimes subtly rephrased.

A lightweight hallucination checklist

Before committing AI-generated code that uses external libraries, run through these five checks:

• [ ] Does every method call appear in the library’s actual documentation or type definitions?
• [ ] Do the imports resolve? (node -e "require('pkg')" or check the package’s exports map)
• [ ] Are any version-specific features being used that your pinned version doesn’t support?
• [ ] Does the code compile without TypeScript errors?
• [ ] Does it run without error on a minimal input?

This takes under five minutes. It doesn’t replace a full review — see Evaluating AI-Generated Code Before It Ships for that — but it catches the class of hallucinations that make it past “the code looks reasonable” and into a test suite that fails inexplicably three days later.

The underlying mental model

The most useful reframe is this: AI-generated code is a first draft from an author who has read a lot of code but can’t run any of it. The model’s strength is structure and logic — how a function should be organized, what edge cases to handle, how to sequence operations. Its weakness is currency: is this the right method name, in this version, for this runtime?

You own the runtime. The model doesn’t. Grounding prompts with real types, pinning versions, and running a quick smoke test on any new external call is how you close that gap — not with skepticism about every line, but with a verification layer targeted at the specific thing the model can’t do for itself.

Hallucinations are manageable. They just require a feedback loop the model can’t provide on its own.