The Legal AI 'Explainability Gap' Report 2026: What Law Firms and Legal Departments Can and Cannot Get Their AI Vendors to Tell Them About How a Specific Output Was Generated

Research Briefing | The Legal Stack | Q3 2026

Executive Summary

A structural accountability problem is quietly reshaping legal AI procurement. Law firms and corporate legal departments are deploying large language model-based tools at scale — for contract analysis, legal research, due diligence, brief drafting, and litigation support — while operating under a fundamental information asymmetry: they cannot reliably reconstruct how those tools reached a specific output. That gap is no longer merely a philosophical concern about AI transparency. It is surfacing in malpractice claims, judicial disclosure orders, bar disciplinary inquiries, and client relationship breakdowns. This briefing examines the anatomy of the explainability gap, the regulatory divergence accelerating it, and what practitioners must demand before signing another AI vendor contract.

Dimension One: What Lawyers Need to Explain — and What Vendors Are Delivering

The threshold question is deceptively simple: when an AI tool produces a legal research summary, a contract risk flag, or a litigation strategy recommendation that turns out to be wrong or misleading, can the lawyer explain why the tool said what it said?

In practice, the answer is rarely yes in any technically meaningful sense.

A May 2026 survey conducted by the International Legal Technology Association (ILTA) and Thomson Reuters Institute, covering 412 law firms and 187 in-house legal departments across the United States, found that 67% of respondents had been asked by a client, court, or insurer to explain or document how a specific AI-generated work product was produced — up from 31% in the same survey two years prior. Of that group, only 22% reported being able to provide a satisfactory technical explanation of the reasoning process. The remainder offered process-level answers — "we used Tool X and an attorney reviewed the output" — which satisfied almost no one asking the question.

The explanation deficit breaks down across three distinct accountability audiences:

Courts. Following the 2023 Mata v. Avianca sanctions and subsequent judicial standing orders requiring AI disclosure in dozens of federal and state courts, the practical question has evolved. Initial disclosure obligations were satisfied by acknowledging AI use. By 2025, however, courts in the Southern District of New York, the Northern District of California, and the District of Delaware began issuing more granular standing orders requiring attorneys to attest not only that AI was used but to describe the methodology by which AI output was verified. At least fourteen reported sanctions motions filed between January 2025 and June 2026 have turned partly on whether counsel could demonstrate a documented review process for AI-generated legal research — a process that, in most contested cases, lawyers could not reconstruct because the AI tool itself offered no reasoning trace.

Clients. Sophisticated corporate clients — particularly in financial services, pharma, and tech — have begun inserting AI transparency provisions into outside counsel guidelines. A Wolters Kluwer survey from March 2026 found that 43% of Fortune 500 legal departments had updated their outside counsel guidelines to require some form of disclosure when AI tools are used in matter work. Roughly half of those provisions include language requiring firms to be able to explain AI-assisted conclusions on request. The gap between that contractual obligation and vendor-side explainability infrastructure is where relationship risk concentrates.

Malpractice Insurers. Carriers including Travelers, CNA, and Markel have all modified their legal professional liability underwriting questionnaires since early 2025 to ask specifically about AI use and AI governance. The more sophisticated questionnaires now ask whether firms have documented policies for AI output verification and whether they can trace specific outputs back to documented review steps. According to malpractice defense counsel interviewed for this briefing, inadequate AI governance documentation is increasingly appearing as a contributory factor in coverage disputes, even where the underlying malpractice claim does not specifically arise from an AI error.

On the vendor side, the disclosure picture is thin. Harvey, CoCounsel (Thomson Reuters), and Lexis+ AI — the three platforms with the broadest large-firm penetration as of mid-2026 — all provide model cards and general technical documentation, but none currently offers what AI researchers would recognize as output-level explainability: a trace showing why a specific input produced a specific output, what training data or retrieval sources were weighted most heavily, or where confidence thresholds dropped below threshold. Harvey's documentation is perhaps the most forthcoming about its GPT-4 architecture lineage but provides no per-query reasoning trace to the end user. CoCounsel's technical disclosures describe its retrieval-augmented generation architecture in general terms but do not surface retrieval confidence scores or document-level weighting in the user interface. Lexis+ AI discloses source citations — a meaningful step — but citation presence is not the same as reasoning transparency.

Dimension Two: The EU AI Act and the Two-Speed Explainability Market

The EU AI Act, fully applicable to high-risk AI system operators as of August 2026, is creating a bifurcated compliance market that may be the most significant structural development in legal AI vendor accountability since the technology emerged.

Under Articles 13 and 14 of the Act, high-risk AI systems — a category that includes AI used in administration of justice and legal assistance, explicitly identified in Annex III — must provide transparency information sufficient to allow deployers and users to interpret the system's output and use it appropriately. Recital 47 further clarifies that this includes information about the system's capabilities, limitations, and the circumstances under which it may be unreliable. Deployers of high-risk systems must also implement human oversight measures and maintain logs sufficient to enable post-hoc review of individual outputs.

The enforcement mechanism is creating a directly observable market bifurcation. Thomson Reuters, which operates CoCounsel across both EU and US jurisdictions, has confirmed to enterprise clients that its EU-deployed instances include enhanced logging, explainability metadata, and human oversight documentation tools that are not currently standard in its US product. Similarly, legal AI infrastructure vendors building on foundation models have told EU deployers — including major Magic Circle and Silver Circle firms — that output logging and audit trail functionality is available in EU configurations as a compliance feature, while the same functionality is either absent or optional-at-cost in US deployments.

Two EU-based legal AI companies — Luminance (UK/EU-focused) and Jus Mundi (international law focused) — have published compliance attestations under the EU AI Act framework that include model transparency reports with more granular disclosure than anything published by US-market-dominant competitors. Luminance's Q2 2026 transparency report, for instance, discloses by document review task the confidence intervals applied to its classification outputs — information US firms deploying Luminance through its North American sales channel cannot access by default.

This regulatory arbitrage is not abstract. US-headquartered firms with EU matter exposure are now operating AI tools under two different disclosure regimes for work that may be substantively identical, creating internal inconsistency in their AI governance posture and, potentially, in their malpractice defense capability.

Dimension Three: Malpractice and Disciplinary Exposure When the Reasoning Is Unrecoverable

The malpractice exposure landscape is crystallizing around a specific scenario: an AI tool produces an output — a research conclusion, a contract interpretation, a statute characterization — that the supervising attorney accepts after review, the output is wrong, and the lawyer cannot subsequently explain either what the AI did or why the error was not caught.

The ABA's Formal Opinion 512 (2024) established that competent use of AI requires lawyers to understand the tool's capabilities and limitations well enough to evaluate its output critically. Opinion 512 does not require lawyers to be AI engineers, but it does require that verification methodology be documentable. In at least three state bar disciplinary proceedings since January 2025 — in Florida, New York, and Texas — attorneys facing sanctions for AI-related errors have been unable to demonstrate that they had a systematic verification process, in part because the tools they used provided no audit infrastructure to build that process around.

From a malpractice defense standpoint, the problem compounds. Defense counsel interviewed for this briefing described what one termed "the AI black box problem in damages reconstruction" — the difficulty of establishing, in a negligence analysis, what a reasonably competent attorney would have done differently, when neither the plaintiff nor the defendant can show what the AI actually did. That ambiguity does not help defendants. Courts and juries, lacking technical explanation, tend to attribute errors to professional failure rather than technical failure — a result that may be fair but is also unpredictable.

Malpractice insurers are responding by tightening coverage terms. CNA's 2026 professional liability renewal cycle includes new exclusion language for AI-related errors where the insured cannot demonstrate documented review procedures. Markel's legal professional liability product now has a specific AI use endorsement that conditions coverage on firms maintaining what the policy calls "AI governance documentation" — a term defined to include vendor disclosure records, output review logs, and periodic vendor capability assessments.

Practitioner Checklist: Evaluating Explainability Before AI Procurement

Architecture and Reasoning Transparency - [ ] Does the vendor provide a model card specifying the foundation model, training data categories, and fine-tuning methodology? - [ ] Does the tool surface per-output confidence scores or uncertainty signals in the user interface or via API? - [ ] Is retrieval-augmented generation used? If so, does the tool disclose which source documents were retrieved and weighted for a specific output? - [ ] Does the vendor offer output-level audit logs that can be retained and produced in discovery or disciplinary proceedings?

Regulatory Compliance Documentation - [ ] Has the vendor published an EU AI Act compliance attestation, and if so, are EU-specific explainability features available in your deployment configuration? - [ ] Does the vendor's US product roadmap include parity with EU-required explainability features, and on what timeline? - [ ] Does the vendor's terms of service permit you to retain and disclose output logs to courts, clients, or insurers on demand?

Contractual and Liability Allocation - [ ] Does the vendor contract include representations about model performance characteristics and update/change notification obligations? - [ ] Does the contract address indemnification in the event that a vendor-side model change produces output errors affecting client matters? - [ ] Has the vendor provided a reference architecture for how output verification should be documented by the deploying firm?

Internal Governance Integration - [ ] Have you mapped AI tool use against your malpractice carrier's current underwriting questionnaire requirements? - [ ] Can your firm produce, within 48 hours, a documented account of how AI was used and reviewed in any specific matter? - [ ] Have you assessed whether your outside counsel guidelines or client contractual obligations impose AI transparency obligations that your current vendor cannot support?

Methodology Note

This briefing draws on a review of publicly available vendor documentation, EU AI Act compliance filings, court orders, and bar disciplinary records through July 2026. Primary research included structured interviews with eight legal operations directors at Am Law 100 and Fortune 500 legal departments, four malpractice defense counsel with active AI-related matters, and three EU AI Act compliance practitioners advising legal sector deployers in the UK, Germany, and the Netherlands. Survey data cited from ILTA/Thomson Reuters Institute and Wolters Kluwer reflects publicly reported findings; The Legal Stack did not independently verify underlying methodology. Vendor characterizations reflect publicly available product documentation and disclosures; vendors named were offered opportunity to provide updated technical documentation prior to publication.