LAW.co Podcast

AI hallucinations in legal research aren't a user error problem — they're an infrastructure one. This episode breaks down why even purpose-built legal AI tools still fabricate citations, and what a properly architected research stack actually requires.

Show Notes

The courtroom consequences of AI-fabricated citations are no longer hypothetical — they're documented in sanctions orders, bar discipline referrals, and malpractice files. This episode of Law.co examines why hallucinations persist even in purpose-built legal AI tools, and argues that the fix isn't better prompting or associate training — it's engineering a research stack that structurally prevents fabrication from reaching a brief. The discussion draws on empirical testing data, published AI research, and real sanctions cases to show where current tools fall short and what sound architecture looks like.

The episode covers:

  • The scale of the problem: Over 2,200 court and tribunal decisions involving AI-fabricated citations have been logged in a dedicated hallucinations database — 1,500-plus from U.S. courts alone — making this a systemic pattern, not an outlier.
  • Why retrieval-augmented systems aren't enough: Stanford RegLab testing found that even tools purpose-built for legal research — connected to authoritative databases — hallucinate at meaningful rates; the structural incentive for language models to produce confident answers, rather than acknowledge gaps, is the deeper issue.
  • Closed corpus with hard abstention rules: A well-designed stack restricts the model to a defined, versioned set of sources and forces it to abstain rather than fall back on parametric memory or the open web — closing off the entire category of invented reporter citations and making the retrieval layer auditable.
  • Citation resolvers as hard gates: Every generated citation — case name, reporter, pin cite, parenthetical characterization — should be automatically parsed and verified before reaching an attorney; failures are removed and logged, not flagged softly. The episode walks through a federal sanctions case where this control would have caught every fabricated citation before the brief was filed. Legal RAG systems that incorporate this kind of resolver are meaningfully different from basic retrieval pipelines.
  • Holding-level grounding to catch misattribution: Real citations can still carry false propositions; passage-level attribution and automated treatment checks (flagging reversed or overruled authority) address the subtler misgrounding problem that a citation resolver alone won't catch.
  • The human gate — and what makes it substantive: No architecture removes the attorney's professional duty to verify filings, as courts applying Rule 11 have confirmed. The goal of the stack is to give the reviewing attorney the retrieved source, resolver result, treatment status, and a diff — and to capture that sign-off as a logged, matter-tied event. Human-in-the-loop design is the architecture principle that makes attorney review meaningful rather than ceremonial.

More from the show: Multi-Agent AI: The Smarter Way to Classify Your Case Files explores how coordinated AI agents can handle complex document workflows — a complementary lens on building reliable legal AI infrastructure.

Law.co

What is LAW.co Podcast?

Legal AI for lawyers and the firms they run. Where AI genuinely helps in research, drafting and review, what privilege and confidentiality actually require of a tool, how to evaluate legal software honestly, and the operational side of running a practice.

Each episode takes one question a practitioner is facing — whether to let a tool touch client data, how to verify AI-assisted research, what to change about billing when work gets faster — and works it through. Written for practising lawyers and firm administrators, not for legal futurism. Five or six minutes an episode.

Topics include AI-assisted research and verification, drafting and review workflows, privilege and confidentiality requirements for tools, evaluating legal software honestly, billing when work gets faster, matter management, and firm operations.

Produced by Law.co, legal AI for lawyers and law firms. Full details, services and further reading at https://law.co