The Legal AI 'Explainability Gap' Report 2026: What Law Firms and Legal Departments Can and Cannot Get Their AI Vendors to Tell Them About How a Specific Output Was Generated

Research Briefing | The Legal Stack | Q3 2026

Executive Summary

A structural accountability problem is quietly reshaping legal AI procurement. Law firms and corporate legal departments are deploying large language model-based tools at scale — for contract analysis, legal research, due diligence, brief drafting, and litigation support — while operating under a fundamental information asymmetry: they cannot reliably reconstruct how those tools reached a specific output. That gap is no longer merely a philosophical concern about AI transparency. It is surfacing in malpractice claims, judicial disclosure orders, bar disciplinary inquiries, and client relationship breakdowns. This briefing examines the anatomy of the explainability gap, the regulatory divergence accelerating it, and what practitioners must demand before signing another AI vendor contract.

Dimension One: What Lawyers Need to Explain — and What Vendors Are Delivering

The threshold question is deceptively simple: when an AI tool produces a legal research summary, a contract risk flag, or a litigation strategy recommendation that turns out to be wrong or misleading, can the lawyer explain why the tool said what it said?

In practice, the answer is rarely yes in any technically meaningful sense.

Firms increasingly report being asked by clients, courts, or insurers to explain or document how specific AI-generated work product was produced, and few can provide a satisfactory technical explanation of the reasoning process. The remainder offered process-level answers — "we used Tool X and an attorney reviewed the output" — which satisfied almost no one asking the question.

The explanation deficit breaks down across three distinct accountability audiences:

Courts. Following the 2023 Mata v. Avianca sanctions and subsequent judicial standing orders requiring AI disclosure in dozens of federal and state courts, the practical question has evolved. Initial disclosure obligations were satisfied by acknowledging AI use. Some judges have since issued more granular orders asking attorneys to describe how AI output was verified, not merely whether AI was used. Sanctions disputes have increasingly turned on whether counsel could demonstrate a documented review process for AI-generated legal research — a process that, in most contested cases, lawyers could not reconstruct because the AI tool itself offered no reasoning trace.

Clients. Sophisticated corporate clients — particularly in financial services, pharma, and tech — have begun inserting AI transparency provisions into outside counsel guidelines. A growing number of large legal departments have updated their outside counsel guidelines to require some form of disclosure when AI tools are used in matter work. Some of those provisions include language requiring firms to be able to explain AI-assisted conclusions on request. The gap between that contractual obligation and vendor-side explainability infrastructure is where relationship risk concentrates.

Malpractice Insurers. Some carriers have modified their legal professional liability underwriting questionnaires to ask specifically about AI use and AI governance. The more sophisticated questionnaires now ask whether firms have documented policies for AI output verification and whether they can trace specific outputs back to documented review steps. Inadequate AI governance documentation is a plausible contributory factor in coverage disputes, even where the underlying malpractice claim does not specifically arise from an AI error.

On the vendor side, the disclosure picture is thin. Harvey, CoCounsel (Thomson Reuters), and Lexis+ AI — three platforms widely used by large firms — provide general technical documentation, but none currently offers what AI researchers would recognize as output-level explainability: a trace showing why a specific input produced a specific output, what training data or retrieval sources were weighted most heavily, or where confidence thresholds dropped below threshold. Harvey's documentation provides no per-query reasoning trace to the end user. CoCounsel's technical disclosures describe its retrieval-augmented generation architecture in general terms but do not surface retrieval confidence scores or document-level weighting in the user interface. Lexis+ AI discloses source citations — a meaningful step — but citation presence is not the same as reasoning transparency.

Dimension Two: The EU AI Act and the Two-Speed Explainability Market

The EU AI Act, whose Annex III high-risk obligations were deferred from August 2026 to December 2027 by the Digital Omnibus on AI (in force 27 July 2026), is creating a bifurcated compliance market that may be the most significant structural development in legal AI vendor accountability since the technology emerged.

Under Articles 13 and 14 of the Act, high-risk AI systems — a category that includes AI used by or on behalf of judicial authorities in the administration of justice, as identified in Annex III point 8 — must provide transparency information sufficient to allow deployers and users to interpret the system's output and use it appropriately. The Act's recitals further clarify that this includes information about the system's capabilities, limitations, and the circumstances under which it may be unreliable. Deployers of high-risk systems must also implement human oversight measures and maintain logs sufficient to enable post-hoc review of individual outputs.

The enforcement mechanism is creating a directly observable market bifurcation. Thomson Reuters, which operates CoCounsel across both EU and US jurisdictions, has confirmed to enterprise clients that its EU-deployed instances include enhanced logging, explainability metadata, and human oversight documentation tools that are not currently standard in its US product. Similarly, legal AI infrastructure vendors building on foundation models have told EU deployers — including major Magic Circle and Silver Circle firms — that output logging and audit trail functionality is available in EU configurations as a compliance feature, while the same functionality is either absent or optional-at-cost in US deployments.

Some EU-focused legal AI vendors have begun positioning their documentation around EU AI Act transparency expectations.

This regulatory arbitrage is not abstract. US-headquartered firms with EU matter exposure are now operating AI tools under two different disclosure regimes for work that may be substantively identical, creating internal inconsistency in their AI governance posture and, potentially, in their malpractice defense capability.

Dimension Three: Malpractice and Disciplinary Exposure When the Reasoning Is Unrecoverable

The malpractice exposure landscape is crystallizing around a specific scenario: an AI tool produces an output — a research conclusion, a contract interpretation, a statute characterization — that the supervising attorney accepts after review, the output is wrong, and the lawyer cannot subsequently explain either what the AI did or why the error was not caught.

The ABA's Formal Opinion 512 (2024) established that competent use of AI requires lawyers to understand the tool's capabilities and limitations well enough to evaluate its output critically. Opinion 512 does not require lawyers to be AI engineers, but it does require independent verification of AI output scaled to the tool and the task — and a verification process that cannot be documented is hard to defend. In court sanctions proceedings over AI-related errors, sanctioned attorneys have often been unable to demonstrate that they had a systematic verification process, in part because the tools they used provided no audit infrastructure to build that process around.

From a malpractice defense standpoint, the problem compounds. Defense counsel face what might be termed "the AI black box problem in damages reconstruction" — the difficulty of establishing, in a negligence analysis, what a reasonably competent attorney would have done differently, when neither the plaintiff nor the defendant can show what the AI actually did. That ambiguity does not help defendants. Courts and juries, lacking technical explanation, tend to attribute errors to professional failure rather than technical failure — a result that may be fair but is also unpredictable.

Malpractice insurers are responding by tightening coverage terms. Some insurers are reported to be adding AI-related exclusions or conditions tied to documented review procedures. Markel's legal professional liability product now has a specific AI use endorsement that conditions coverage on firms maintaining what the policy calls "AI governance documentation" — a term defined to include vendor disclosure records, output review logs, and periodic vendor capability assessments.

Practitioner Checklist: Evaluating Explainability Before AI Procurement

Architecture and Reasoning Transparency - [ ] Does the vendor provide a model card specifying the foundation model, training data categories, and fine-tuning methodology? - [ ] Does the tool surface per-output confidence scores or uncertainty signals in the user interface or via API? - [ ] Is retrieval-augmented generation used? If so, does the tool disclose which source documents were retrieved and weighted for a specific output? - [ ] Does the vendor offer output-level audit logs that can be retained and produced in discovery or disciplinary proceedings?

Regulatory Compliance Documentation - [ ] Has the vendor published an EU AI Act compliance attestation, and if so, are EU-specific explainability features available in your deployment configuration? - [ ] Does the vendor's US product roadmap include parity with EU-required explainability features, and on what timeline? - [ ] Does the vendor's terms of service permit you to retain and disclose output logs to courts, clients, or insurers on demand?

Contractual and Liability Allocation - [ ] Does the vendor contract include representations about model performance characteristics and update/change notification obligations? - [ ] Does the contract address indemnification in the event that a vendor-side model change produces output errors affecting client matters? - [ ] Has the vendor provided a reference architecture for how output verification should be documented by the deploying firm?

Internal Governance Integration - [ ] Have you mapped AI tool use against your malpractice carrier's current underwriting questionnaire requirements? - [ ] Can your firm produce, within 48 hours, a documented account of how AI was used and reviewed in any specific matter? - [ ] Have you assessed whether your outside counsel guidelines or client contractual obligations impose AI transparency obligations that your current vendor cannot support?

Methodology Note

This briefing draws on a review of publicly available vendor documentation, EU AI Act compliance filings, court orders, and bar disciplinary records through July 2026. It also draws on published practitioner commentary; The Legal Stack did not conduct interviews or surveys. Vendor characterizations reflect publicly available product documentation and disclosures.