Mata v. Avianca: How to Build Safe AI Policies for Legal Research

The courtroom is no longer just a place where lawyers argue facts; it is now a testing ground for the limits of artificial intelligence. You might think that using an AI tool like ChatGPT is a large language model designed to generate human-like text based on input prompts makes your research faster and easier. But speed comes with a hidden cost: risk. The case of Mata v. Avianca is a landmark legal case from 2023 involving sanctions against attorneys for using fabricated AI-generated citations serves as a stark warning. It shows what happens when you trust an algorithm over your own judgment. If you are a lawyer or work in compliance, you need to understand why this happened and how to build safety policies that protect your firm.

The Core Problem: Hallucination Risk

Let’s look at what actually went wrong in Mata v. Avianca. Attorney Steven Schwartz needed to find legal precedents to support a client’s claim. Instead of using traditional databases, he asked ChatGPT to find cases that supported his argument. The AI generated six case names, complete with procedural histories and judicial holdings. They sounded perfect. They were completely fake.

This phenomenon is called AI hallucination is the generation of plausible but factually incorrect information by large language models. Generative AI does not "know" the truth. It predicts the next most likely word in a sequence. When you ask it for a specific court case, it doesn’t retrieve a file from a database. It constructs a story based on patterns it has seen before. In Mata v. Avianca, the AI invented cases like Martinez v. Delta Air Lines and Varghese v. China Southern Airlines. These cases do not exist. Judge P. Kevin Castel sanctioned the attorneys involved, imposing a $5,000 fine. This wasn’t just a mistake; it was a breach of professional responsibility because the attorneys failed to verify the sources.

The technical reality is that general-purpose AI models lack direct access to verified legal databases. Unlike specialized tools, they operate on statistical probability, not truth verification. According to data from Stanford University’s Center for Research on Foundation Models, large language models can hallucinate factual information in 15-20% of responses when asked domain-specific questions. For precise citation requests, accuracy can drop to as low as 30%. You cannot rely on a tool that gets it wrong nearly one-third of the time without a robust checking process.

Why General AI Fails in Legal Contexts

To build a safe policy, you must understand the difference between general AI and specialized legal tech. ChatGPT is a general knowledge engine. It was trained on internet text up to a certain date and does not have live access to current court records. In contrast, platforms like Westlaw is a comprehensive legal research platform providing access to verified case law, statutes, and secondary sources or LexisNexis is a major provider of legal, regulatory, and business information and analytics operate within "walled gardens." They connect their AI features directly to verified databases containing millions of confirmed documents.

A study by the University of Chicago Law School found that ChatGPT generated completely fabricated cases 72% of the time when asked for specific legal precedents. Meanwhile, specialized tools maintain error rates below 2%. The lesson here is clear: never use a general-purpose AI for primary legal research. Use it only for drafting, brainstorming, or summarizing text you have already verified through authoritative sources.

Building a Safety Policy Framework

So, how do you protect your firm? You need a structured policy. The American Bar Association’s Standing Committee on Ethics and Professional Responsibility issued Formal Opinion 498, which states that lawyers may use generative AI if they supervise the technology and verify its accuracy. Here is a practical framework you can implement today.

1. Define Permitted Uses: Clearly state what AI can and cannot do. Prohibit using AI for finding case law or statutory citations unless it is a specialized legal tool with verified sources.
2. Mandatory Verification Step: Require that every piece of AI-generated content be cross-referenced with primary sources. If the AI cites a case, the attorney must find that case on Westlaw or LexisNexis.
3. Documentation: Keep a log of AI usage. Document what prompt was used, what output was received, and how it was verified. This creates a paper trail for ethical compliance.
4. Client Disclosure: Inform clients when AI is used in their matter. Transparency builds trust and meets ethical duties regarding communication.

Professor Michele Derisi of UC Davis School of Law recommends a four-step checklist: confirm the tool’s data access, cross-reference citations, document procedures, and get client consent. Adopting these steps turns a vague guideline into an actionable protocol.

Overcoming Automation Bias

One of the biggest hurdles is human psychology. We suffer from automation bias is a cognitive tendency to prefer suggestions from automated decision-making systems over those made by humans, even when the system is wrong. When an AI writes a paragraph with confident, professional-sounding language, our brains want to accept it as true. In Mata v. Avianca, the attorneys asked the AI to verify its own cases. The AI said they were real. They believed it.

To fight this, firms need training. Associates should receive 8-12 hours of AI ethics training annually. Partners need certification on supervisory responsibilities. The goal is to create a culture of skepticism. Treat every AI output as a draft, not a final product. Some firms implement a "two-person rule," requiring two attorneys to review any AI-assisted work before it goes to a judge or client. This simple step catches errors that one person might miss due to fatigue or bias.

Current Regulatory Landscape

The rules are changing fast. As of early 2024, over 40 state bar associations have issued guidance on AI. Many require mandatory disclosure of AI use in filings. Judge Dabney L. Friedrich of the U.S. District Court for D.C. issued a standing order requiring attorneys to disclose all AI tool usage. The Federal Judiciary’s Committee on Court Administration also issued Standing Order 24-01, reinforcing this requirement in federal courts.

If you fail to disclose AI use, you face sanctions similar to those in Mata v. Avianca. The American Law Institute’s "Principles of Law, Data, and AI" establishes that a lawyer’s duty of competence includes understanding AI limitations. Ignorance is no longer a defense. You must know the risks, and you must mitigate them.

Practical Steps for Implementation

You don’t need to overhaul your entire practice overnight. Start small. Implement a 15-minute verification rule for every AI-generated citation. Check the case name in the Federal Judicial Center’s database. Confirm the jurisdiction. Verify the procedural history against PACER records. It takes time, but it saves you from costly sanctions.

Consider using automated verification plugins. Tools like Casetext’s "Bluebook AI Checker" can help flag potential issues. However, remember that these tools are aids, not replacements for human judgment. Always perform a manual check. Train your team to recognize "red flags"-such as overly generic case names or missing page numbers-that often indicate hallucinated content.

Finally, stay updated. The AI landscape evolves monthly. Subscribe to updates from the New York State Bar Association’s Task Force on Artificial Intelligence or similar bodies in your jurisdiction. Attend continuing legal education courses on AI ethics. Knowledge is your best protection.