Plausible Incorrectness: The New Epistemological Crisis

In 2023, two New York lawyers filed a brief in a routine injury case, Mata v. Avianca, citing a string of supportive precedents: Varghese v. China Southern Airlines, Martinez v. Delta, and others, complete with quotations and internal citations. There was one problem. The cases did not exist. ChatGPT had invented them — plausible names, plausible reasoning, plausible page numbers — and when opposing counsel could not find them, the lawyers did not retreat. They doubled down, submitting AI-generated "copies" of the fabricated opinions. Judge Kevin Castel fined them $5,000 for "subjective bad faith," and the case became a landmark. By early 2026, the researcher Damien Charlotin had catalogued more than 1,174 documented court filings contaminated with AI-hallucinated citations worldwide, the sanctions climbing to $15,000 per attorney and, in one instance, an indefinite bar suspension.

The fake cases were not gibberish. That is the entire point. They were plausible — structured to look exactly like real law, right up until someone tried to rely on them. This is plausible incorrectness: output built to pass a glance and fail an inspection. It has become the characteristic epistemological problem of a world saturated with AI, and it breaks something human cognition has depended on for centuries.

The shape of the problem

Previous generations of error came in recognizable shapes. Broken code did not compile. Doctored photos had visible seams. Plagiarized prose contained discontinuities of voice. The errors announced themselves; detection was a matter of attention. Plausible incorrectness inverts this. The errors are smoothed across many small features rather than concentrated in one obvious mistake, so attention alone does not catch them — you have to already know what to look for. An AI-generated video of a couple dancing looks ordinary for five seconds; only at ten do you notice the hand passing through the partner's back, the shadow that does not track, the background that fails to hold its geometry. Small errors. Also fundamental ones — the kind that reveal the thing was not filmed but generated.

Why it touches almost everything

The consequence is not about fake videos. It is about every domain where a fluent surface is taken as a signal of a sound interior.

Consider medicine. An AI tool drafts a patient discharge summary from clinical notes. It reads smoothly and captures most of what matters. It also, occasionally, contains a subtle misstatement — the wrong dosage, the wrong differential, the wrong order of events — plausible in the exact sense that nothing about the summary flags it as wrong. Only someone who reads the original notes closely catches it, and most reviewers do not have the time. Consider science: a paper makes claims plausibly supported by its citations, except sometimes the cited studies do not say what the paper says they say, and the weak link survives peer review because reviewers, like everyone, work quickly and are swayed by surface fluency. Stanford's Human-Centered AI institute found that even purpose-built legal AI tools hallucinated on more than one in six benchmark queries — in the domain where getting it right is the entire job.

And the reach extends past hallucinated facts to institutional liability. In February 2024, Canada's Civil Resolution Tribunal ordered Air Canada to compensate a passenger, Jake Moffatt, whom its website chatbot had confidently misinformed about bereavement fares. The airline argued the chatbot was "a separate legal entity responsible for its own actions." The tribunal called this a "remarkable submission" and rejected it: a company is responsible for everything on its site, whether written by a lawyer or generated by a bot. Plausible incorrectness is now something you can be held liable for.

The cognitive cost

Living inside an environment of plausible incorrectness carries a specific, grinding cost: you can no longer trust your first impression of anything. You cannot treat fluency as evidence of accuracy. You cannot take the markers of expertise — confident, articulate, well-formatted output — at face value, because those markers can now be produced by tools that are not experts.

Human cognition runs on heuristics. Most of what we read and hear we evaluate with a skimming attention that scans for obvious errors and, finding none, grants provisional credence. That system worked because producing fluent output used to require enough effort that most fluent output came from people who had thought carefully. It breaks the instant fluency becomes cheap. The rational response — suspicion of everything, careful reading at every level — is cognitively unaffordable at scale. A doctor cannot read every AI summary with maximum suspicion; a lawyer cannot independently verify every citation; a reader cannot fact-check every article. The volume of output has outrun the supply of careful attention available to check it. This is the individual-scale twin of Coherence Collapse (#37): there, institutions drown in generated volume; here, individuals drown in generated plausibility.

The partial solutions, and their limits

Fixes are emerging, each bounded. Cross-verification tools check outputs against authoritative sources — good for citations and hard facts, useless for reasoning, interpretation, or claims that are subtly wrong in framing rather than fact. Specialized reviewers trained to catch AI's characteristic errors work in high-stakes domains where the cost justifies the review, and do not scale to the rest of the economy. Provenance and watermarking schemes help establish what was machine-made, but say nothing about whether a human-written claim is true. Each solution buys back a piece of the trust that cheap fluency dissolved; none restores the old default, in which a well-made surface was decent evidence of a sound interior.

What plausible incorrectness ultimately demands is the reconstruction, on purpose and at cost, of something that used to be free: the link between how right a thing looks and how right it is. The series calls the gap between claim and reality the Verification Gap (#9). Plausible incorrectness is what happens when that gap becomes invisible — when the error learns to wear the face of the truth, and the only defense left is the expensive, unautomatable act of looking past the face.

This is article #41 in The IUBIRE Framework series. Plausible Incorrectness was articulated by IUBIRE V3 around artifact #1481 — "The AI Dance Floor Paradox" — and its siblings (spring 2026). Real-world data: Mata v. Avianca (SDNY, 2023, $5,000 sanction for fabricated ChatGPT citations); Moffatt v. Air Canada (BC Civil Resolution Tribunal, Feb 2024); Damien Charlotin's AI-hallucination case database (1,174+ filings by early 2026); Stanford HAI benchmarking of legal-AI hallucination rates (>1 in 6).

Next in series: Cryptographic Constitutionalism (#42)