Case of the Week · Week 2

Mata v. Avianca — the brief that put AI hallucinations in every headline. This week we didn’t grade ourselves. The court already did.

Report run August 16, 2026

Special Edition · Benchmark

Not our normal weekly. Most weeks we pull a random public brief that mostly did its job. This one is the most famous AI-hallucination brief in the country — and it comes with an answer key: a federal judge already ruled on exactly which cases were fake. So we ran it as a test against a known scorecard.

cases cited

the court found fabricated

6 of 6

flagged by CiteCheck

real cases misused

What this is. This is a special edition, and we’re labeling it as one. Normally Case of the Week is a random public brief — and normally the story is that a lawyer did their job. This week is the opposite by design. Mata v. Avianca is the brief that made “AI hallucination” a headline: a 2023 filing whose citations were generated by ChatGPT, several of which turned out not to exist. We ran it for one reason — it’s a rare brief where the right answer is already public. Judge P. Kevin Castel’s sanctions order (S.D.N.Y., June 22, 2023) named exactly which cases were fabricated. That gives us a scorecard to check ourselves against.

What the court found. The court identified six fabricated cases — Varghese, Shaboon, Petersen, Martinez, Durden, and Miller — and sanctioned the attorneys. That’s the answer key. The question for us was simple: run the brief through CiteCheck cold and see what it does with those six, and with everything around them.

What CiteCheck did with the fakes. It flagged all six for review. Five came back “this citation isn’t in CourtListener — look into it.” The sixth, Miller, came back “this citation doesn’t resolve to that case.” Notice what CiteCheck does not do: it never stamps a case “fabricated.” It can’t. Absence from a database isn’t proof a case is fake — real cases sit outside CourtListener too (unpublished state opinions, Westlaw-only cites, older reporters). So we route it to a human instead of guessing. The court could prove the negative; we can only point the reviewer at the right doors. On this brief, all six doors were the right ones.

The part the headlines missed. The fabricated cases got the press. But a citation checker that only catches fake cases misses most of the problem — and this brief proves it. Underneath the six fakes were real cases used wrongly: two real decisions cited under fabricated citations, and real cases quoted for propositions the opinions don’t actually support. None of those are in the sanctions headline. All of them are wrong. That layer — real case, wrong use — is the one CiteCheck was built for, and it’s where the four extra catches live.

Six fabricated cases — 6 of 6 flaggedThe cases the court called fake, CiteCheck routed to review

Five of the six — Varghese, Shaboon, Petersen, Martinez, and Durden — surfaced as “cannot verify”: not found in CourtListener, sent to a human with a plain note to look into them. The sixth, Miller v. United Airlines, surfaced differently, up top, as “citation doesn’t match” — the cited 174 F.3d 366 doesn’t resolve to that case at all.

Why we don’t write “fabricated” on the report: because we can’t prove it from where we sit, and a false accusation is its own kind of error. “Cannot verify” is the honest verdict — it means look here, not it’s fake. The court did the work that turns “cannot verify” into “fabricated.” Our job was to make sure a human’s attention landed on all six. It did.

Two real cases · fabricated citationsEhrlich and Zicherman — real decisions, wrong citations

These two aren’t on the court’s fabricated list, because the cases are real. The citations are not. The brief cites Ehrlich v. American Airlines at 360 N.J. Super. 360 — but the real Ehrlich is a federal case at 360 F.3d 366 (2d Cir. 2004). It cites Zicherman v. Korean Air Lines at 516 F.3d 1237 (11th Cir. 2008) — the real Zicherman is at 92 F.3d 126 (2d Cir. 1996). Wrong reporter, wrong court, wrong year: a fabricated citation hung on a real case name.

How CiteCheck handles it: the citation is flagged up top as “doesn’t match,” with the real case and the real citation shown. The proposition itself goes to attorney review — because we can’t grade a quote against an opinion the citation never points to, and we won’t judge a filing on a case it didn’t actually cite. The catch is the citation, and it’s a real one.

Four real cases · used for the wrong pointThe layer nobody was looking at

Kaiser Steel v. W.S. Ranch is a state-water-law abstention case — the brief cites it for the automatic bankruptcy stay tolling a statute of limitations. It says nothing of the kind. In re Air Crash Disaster Near New Orleans is cited on the same tolling point at page 1165, where the opinion is actually discussing forum non conveniens. El Al Israel Airlines v. Tseng is quoted for a line about Congress enacting the Montreal Convention to “modernize and unify” the Warsaw system — that language isn’t in Tseng. And Doe v. United States is cited for a proposition only half of which the opinion holds; the case was, in fact, reversed against the plaintiff.

These are the catches nobody else was making. Every one is a real case with a real, correct-looking citation — so an existence checker waves it straight through. The error is in the use, not the cite. Two of these (Tseng and Doe) we verified by hand against the opinions. This is the whole reason CiteCheck checks whether the case supports the point, and not just whether the case exists.

Want to see more of our catches?

The full report, unedited

Open the report in a new tab ↗

See every past Case of the Week