How to catch AI-hallucinated CVEs before they reach a report

Q: Can an AI actually invent a CVE that does not exist?

Yes. CVE IDs have a fixed, predictable shape (CVE-YEAR-NUMBER), which is exactly what a language model fills in confidently. The only reliable defense is to look the ID up in NVD/OSV/GHSA rather than trusting that it parses.

If you point an LLM at a security task, it will eventually hand you a confident sentence like "this is vulnerable to CVE-2025-99999, a critical Struts RCE, actively exploited." Every part of that can be wrong: the CVE ID may not exist, the severity may be invented, and the product may be a different vendor's software entirely. The model is not lying on purpose. It is pattern-completing over CVE-shaped text, and CVE identifiers are exactly the kind of structured token a language model is happy to fabricate.

The short answer to the title: attach every CVE claim to a cheap check that refutes it, run against a source of truth, before the claim ships. Do not let a confident string become a stated fact. Below is what the three failure modes actually look like, with real refuted output, and how to wire the check in.

Why AI agents get CVEs wrong (three distinct failure modes)

These are not the same bug, and a single "add RAG" hand-wave does not cover all three.

Invented identifiers. The model emits a CVE ID that was never issued. It looks real (CVE-2025-99999) because it has the right shape. This is the CVE analogue of slopsquatting, where agents hallucinate package names that do not exist, then someone registers them.
Wrong attributes on a real CVE. The CVE exists, but the model gets the severity, the exploited status, or the affected product wrong. This is the dangerous one, because the ID checks out, so a human skims past it.
Right CVE, wrong reachability. The CVE is real and correctly described, but the model asserts the target is exploitable when the vulnerable version or precondition is not actually present. "Version in range" is not "exploitable."

The fix: a fact-check gate on every claim

The gate is one function that takes what the agent asserted and returns a per-assertion verdict (the verify_cve_claim tool) against a fused corpus (NVD + OSV/GHSA + CISA KEV + FIRST EPSS + CISA SSVC). Three real results follow, run live against that corpus.

Failure mode 1: the CVE does not exist

Ask the corpus whether CVE-2025-99999 is real:

verify_cve_claim("CVE-2025-99999")
  ->  exists: false   verdict: refuted
      "No record of CVE-2025-99999 in NVD / OSV / GHSA: likely hallucinated
       or not-yet-published. (Absence scoped to those sources.)"

Note the honesty of the scope: the tool says absence in these sources, not "this is fake, trust me." That distinction matters, because a not-yet-published CVE and a hallucinated one look identical, and the check tells you which sources it checked rather than inventing certainty.

Failure mode 2: a real CVE, described wrong

Here is a claim that a triager would almost accept, because the shape is plausible: "CVE-2021-44228 is a medium-severity Apache Struts bug, and it is not exploited." Every clause is wrong, and the gate refutes each one with cited evidence:

verify_cve_claim("CVE-2021-44228", product="Apache Struts",
                 severity="medium", exploited=false)

  refuted  "not exploited"          ->  in CISA KEV (added 2021-12-10)
  refuted  "severity medium"        ->  actual CVSS 10.0 (3.1) -> P1
  refuted  "affects Apache Struts"  ->  this CVE lists no product matching
                                        "Apache Struts" (it is Apache Log4j2)

That is Log4Shell, one of the most consequential vulnerabilities of the decade, described as a mild Struts issue. An LLM will produce sentences like this because "Apache Struts" and "Apache Log4j" sit near each other in its training distribution. The refutation is not a judgment call: it is three lookups against the record.

Failure mode 3: real and correct, but not reachable

The third mode needs product and version resolution, not just an ID. Ask what actually affects a product and at what priority. For Apache Struts, the corpus returns the real exploited history, ranked exploitation-first (KEV and EPSS ahead of raw CVSS):

check_technology("Apache Struts")
  P1  KEV  CVE-2017-5638   Jakarta Multipart parser RCE via crafted Content-Type
  P1  KEV  CVE-2013-2251   OGNL expression injection -> remote code execution
  P1  KEV  CVE-2017-9791   Struts 1 plugin RCE via a raw ActionMessage value
  P1  KEV  CVE-2020-17530  forced OGNL evaluation on tag attributes -> RCE

Now the agent's claim has a denominator. If it cited a Struts CVE that is not in this resolved set, or cited one of these against a version outside its affected range, the reachability assertion fails before anyone writes it up. The point is not the list. The point is that "affects Struts" becomes checkable instead of vibes.

Wiring it in

The gate is a single tool call an agent makes before it commits a CVE claim to a report, an issue, or a remediation ticket. In practice:

On generation: whenever the model outputs a CVE-... token with an assertion attached (severity, exploited, affected product), call verify_cve_claim with those attributes and treat any refuted as a hard stop.
On ingestion: when you ingest a third-party or AI-written report, run the same gate across every CVE it cites. Hallucinated CVEs in inbound bug reports are now common enough that triage teams burn real hours on them.
Prefer refutation over confidence. A model's own confidence score has a stake in the answer. A lookup against NVD/OSV/GHSA/KEV does not.

The corpus behind these checks is one record per vulnerability, fused across sources through an identity graph, refreshed daily. At the time of writing it holds 368,765 CVEs, 1,638 CISA KEV entries, and EPSS scores for 344,957 of them, with data under a day old. Those numbers drift, so query corpus_stats for the live figure rather than trusting a number on a page.

FAQ

Can an AI actually invent a CVE that does not exist? Yes. CVE IDs have a fixed, predictable shape (CVE-YEAR-NUMBER), which is exactly what a language model fills in confidently. The only reliable defense is to look the ID up in NVD/OSV/GHSA rather than trusting that it parses.

How do I verify a CVE is real and correctly described? Check three things against a source of truth: does the ID exist, does it affect the claimed product at the claimed version, and are the severity and exploited-status right. A single verify_cve_claim call returns a per-assertion supported / refuted / unverifiable verdict with cited evidence.

Why does my agent keep getting CVE severity wrong? Because it is recalling severity from training text, not reading the record. Log4Shell is CVSS 10.0 and in CISA KEV; a model can still call it "medium, not exploited" if that phrasing was statistically near the ID. Verify severity against the actual CVSS vector and KEV status.

Is retrieval-augmented generation (RAG) enough? RAG reduces invented facts by grounding answers, but it does not, by itself, refute a specific false attribute on a real CVE. You still want an explicit per-claim check that returns a verdict, not just relevant context the model may or may not use.

Where does the ground-truth data come from? NVD, OSV/GHSA, CISA KEV, FIRST EPSS, and CISA Vulnrichment (SSVC), fused into one record per vulnerability and refreshed daily.

The check itself is the point: a per-claim lookup against a source of truth, run before the claim counts, by something with no stake in the answer being yes. The corpus behind these examples is a hosted MCP for AI security agents, and a free key at mcp.rozetyp.com/signup will let you run the check yourself, but the technique matters more than the tool. For why this kind of memory and discipline turns out to matter more than raw model quality, see the bottleneck for AI security agents isn't reasoning; for exploitation-first prioritization of what the checks return, see vulnerability prioritization that works.