Skip to content
Zrico
Menu

For law firms

AI Hallucinated Citations: Sanctions Cases and How to Check

Why AI tools invent case citations, what happened in Mata v. Avianca and Park v. Kim, what ABA Opinion 512 asks, and a routine to check every citation.

Updated 6 min read

AI tools sometimes make up cases. They produce a case name, a citation and even quotations that look right but have nothing behind them. Courts have sanctioned lawyers who filed them, and the only reliable protection is to pull and read every authority yourself before anything is filed.

What is a hallucinated citation?

A hallucinated citation is a reference to an authority that does not exist, or that exists but does not say what the AI claims. Language models write by predicting plausible text. Unless a tool is built to look sources up, a model can produce a case name, a reporter citation and a quotation in the right format with no real case behind them. ABA Formal Opinion 512 calls this kind of output a hallucination.

The errors come in several forms:

  • a case that does not exist
  • a real case with the wrong citation, court or year
  • a real case quoted with words it does not contain
  • a real case cited for a point it does not support
  • a case that has been reversed or overruled

The last three are harder to catch than the first, because a quick search finds the case.

What happened in Mata v. Avianca?

In Mata v. Avianca, Inc., a federal judge in the Southern District of New York sanctioned two lawyers and their firm for filing fake judicial opinions, with fake quotes and citations, created by ChatGPT. The filing, made in March 2023, opposed a motion by the airline. The airline’s lawyers said they could not find the cases, and the court then ordered copies.

In a June 2023 opinion, the court found that the lawyers kept standing by the fake opinions after the court’s orders called them into question, and found bad faith. One lawyer had asked ChatGPT whether one of the cases was real. ChatGPT answered that its authorities were real and could be found on Westlaw and LexisNexis.

The court:

  • imposed a $5,000 penalty on the lawyers and their firm, jointly and severally
  • ordered them to send letters to their client and to each judge falsely named as the author of the six fake opinions, with a copy of the court’s order

The court also said there is “nothing inherently improper about using a reliable artificial intelligence tool for assistance.” The problem was filing the output unchecked, and then defending it.

Have other courts acted on fake AI citations?

Yes. In Park v. Kim, decided in January 2024, the U.S. Court of Appeals for the Second Circuit could not locate a state appellate decision cited in a reply brief. When the court ordered a copy, the lawyer could not provide one and said she had used ChatGPT to find the case. The court held that Rule 11, at the very least, requires lawyers to read, and so confirm the existence and validity of, the authorities they rely on. It referred the lawyer to its Grievance Panel and ordered her to give her client a copy of the decision.

Some courts have also adopted local rules or standing orders on AI use, and the Park v. Kim opinion cites examples. Check the local rules and your judge’s standing orders before you file.

What does ABA Formal Opinion 512 say about checking AI output?

It says the duty of competence calls for an appropriate degree of independent verification or review of AI output, and that relying on output without it could violate Rule 1.1. How much review is enough depends on the tool and the task.

For court filings, the opinion is direct. Before submitting materials to a court, lawyers must review AI output, including analysis and citations to authority, and correct errors. It grounds this in Rules 3.1, 3.3 and 8.4(c), and names citations to nonexistent opinions among the problems that have already occurred.

The opinion also says lawyers with managerial authority must set firm policies on AI use and train the people they supervise. A written verification routine is one practical way to do that. ABA opinions are not binding on their own; your state’s rules and ethics opinions control. See our Opinion 512 checklist.

Check every citation, every time, whether or not the draft came from AI. Here is a routine any lawyer can follow:

  1. List every authority. Pull each case, statute, regulation, court rule and agency document out of the draft, including footnotes and string cites.
  2. Pull each one from a reliable source. Use an official or reputable database: Westlaw, Lexis or Bloomberg Law, the court’s own website or PACER, CourtListener, the U.S. Code, the eCFR, irs.gov, or your state’s official code and reports. Do not ask the AI tool to confirm its own citations. In Mata, ChatGPT told the lawyer its cases were real.
  3. Match the details. Confirm the case name, reporter, volume, first page, court and year. A real case under the wrong citation is still an error.
  4. Read it. Read enough of each authority to confirm it supports the point it is cited for. An AI summary of the case is not reading it.
  5. Check every quotation and pin cite. Find the quoted words at the cited page, word for word. Invented quotations can sit inside real cases.
  6. Check subsequent history. Run each case through a citator such as KeyCite or Shepard’s. Confirm that statutes and regulations are the current versions, or the versions in force at the relevant time.
  7. Take out anything you cannot find. If you cannot locate an authority after a reasonable search, it does not go in the filing.
  8. Keep a record. Note who checked each authority, when, and against which source. A simple table in the matter file is enough, and it lets a supervising lawyer see the work.
  9. Follow the court’s rules on AI. Make any disclosure or certification that a local rule or standing order requires.

Software can help with steps 2, 3 and 7. Steps 4 to 6 need a lawyer.

Can software check citations for you?

Software can confirm that a citation exists and matches its source, which catches invented cases and wrong citations. It cannot tell you whether an authority supports your argument, or whether you have missed controlling law. Treat any automated check as a first pass that tells a lawyer where to look, not as the review itself.

How Zrico fits

Zrico Pro includes cite-check. It checks case, statute, regulation and IRS citations in a document against CourtListener, the eCFR, the U.S. Code and irs.gov. It marks each one Verified, Mismatch or Not found, with a link to the source, so you can see what it could not find. Verified means the authority exists and matches the cited name and year; it is not a check that the authority supports the point. Only the citations are sent to those public sources, and the check itself uses no AI model. Zrico Pro is $89 per seat per month for everyone in the firm, and it is in early access, by request. Cite-check does not replace a lawyer’s review. See Zrico Pro.

Sources

General information, not legal advice.

Give your firm a private place to use AI.

$39 per seat per month after a 30-day free trial, 5-seat minimum. Your workspace is ready minutes after you sign up.

Your card is charged when the trial ends. Cancel any time before then and you pay nothing.

Start 30-day free trial