I did a comparison test and Fable is by far the best AI for attorney legal research

I ran a cool test and wanted to share it here.

Fable is obviously supposed to be super smart, but I wanted to try to measure how much better it would actually be than Opus/Sonnet/Haiku (or Westlaw/Lexis) if you're a practicing attorney. One possibility was that Opus and Fable would pretty much be equivalently good and spending the extra money wouldn't be worth it for legal research tasks.

In order to get a sense of just how good Fable is, I ran all the main AI models through a test. I gave them all the same research assignment, which I picked because it's an area of law I know.

Please prepare a memo analyzing whether a trade creditor can pierce the veil of a Delaware LLC whose sole member is a Texas-resident individual. The LLC was formed in Delaware in 2019 to operate a single Houston-area restaurant. The sole member routinely paid personal expenses (his home mortgage, his wife's vehicle lease, his children's tuition) directly from the LLC operating account; the LLC never adopted anything beyond a one-page operating agreement, held no member meetings, and was initially capitalized with $5,000 against monthly operating expenses of roughly $80,000. My client, a produce wholesaler, is owed approximately $220,000 on open account. The LLC has ceased operations and is insolvent. Suit will be filed in Harris County. Please address: (1) whether Delaware or Texas law governs the veil-piercing analysis under Texas choice-of-law principles (internal affairs doctrine vs. substantive tort/contract characterization); (2) the substantive standards under each jurisdiction; (3) whether reverse veil-piercing is available; and (4) whether a companion Texas Uniform Fraudulent Transfer Act claim against the individual member is viable and how it interacts with the veil theory.

Then I actually read the controlling cases and statutes myself to come up with the (in my opinion, as a Texas attorney who has practiced in this area) "correct" answers to be able to grade their performance. To state the obvious: If you're an attorney you know that most things in law are debatable, so I tried to focus my assessment on things where the law has a pretty clear predictive answer (e.g. a controlling statute or clear opinions) that the AI either did or didn't issue spot and address. There's inherently some amount of subjectivity in that exercise - picking the issues that count, the decision to weight them each equally vs some kind of point system, assessing based on "hard" vs "soft" factors, etc. This is a sincere effort to make a fair test, but there's no such thing as a perfect test.

The short answer is that Fable is really, really good - IMO it's definitely the best AI right now at legal research. On all the places I "graded," Fable set to high effort was the only AI model to spot all the issues and find all the answers.

That's NOT to say it's perfect. Fable made mistakes - interestingly, it made a mistake on "max" effort that it didn't make on "high" effort. But human attorneys also err. In fact, almost every time I read a brief from opposing counsel I find lots of "mistakes." If you bring a similarly jaundiced eye to reading these outputs you will find similar mistakes. But Fable High (whose citations I hand checked) had no hallucinations, it's analysis was IMO very good, and I don't think the majority of human attorneys would do significantly better with the same assignment.

To enable Claude to conduct its own legal research, I used a (free) connector my friends and I made ( DingDuff ) which (with Claude Cowork) lets Claude access and download statutes, rules, and court opinions as .md files. To be clear, the test doesn't need my connector. There are other free connectors ( Courtlistener has one with case law, but not statutes), and also some commercial ones. The results here could probably be obtained with any of them as long as they act as a pipe for Claude to access the primary legal authorities - the intelligence, analysis, and research ability is coming from Claude, not the connector. That's why, even with the same connector, you get dramatically different results from Haiku and Fable. (n.b. I did use a skill ( also free ) on all the Claude runs, but frankly Claude works pretty well even without a separate skill file so I don't know if it mattered.)

The Outputs & Citation Check Panel

Model Memo (PDF) Cite-Check Panel Opus (High) PDF Review panel Opus (Max) PDF Review panel Sonnet (High) PDF Review panel Fable (High) PDF Review panel Fable (Max) PDF Review panel Haiku (Extended Thinking) PDF Review panel Lexis Protege PDF Review panel Westlaw CoCounsel PDF Review panel A note on the review panel: This is a tool I made to check work product before filing / use - it pairs the memo on the right with the downloaded and extracted raw text of the cited source (e.g. case, depo transcript, statute) on the left. When you click on a citation, it brings up that source. The highlights are an AI guess about the relevant part, but it's mostly good for direct quotes (it struggles highlighting more complex points). Because it's working from extracted text the formatting isn't great, so I normally pull up the case on Westlaw or the PDF exhibit if I need to read at length. But it's useful for fast checks or minor points.

Normally I fill these review panel verdicts in myself, which gives me a methodical way to be sure I hit all of them and doubles as proof that I checked all the sources (it prints a log). But this was a test run, not a filing, and I was evaluating memo conclusions rather than citation accuracy. So I assigned the cite check task to Opus. It's OK at it, but definitely not perfect - I definitely would not recommend doing that for a filing. I personally checked the negative Westlaw and Lexis flags to make sure Opus wasn’t hazing them (it was mostly fair, but I reversed some nitpicking red flags). I also hand reviewed all the citation flags in the Fable High output as that’s the one I was most interested in, and they’re all correct (no hallucinations). Treat all the other citation verdicts in the panels as AI outputs and, therefore, potentially wrong. The whole point of the panel is to enable you (the human) to quickly look for yourself.

You'll see on the Fable High output that it cited Restatement provisions without having them (my connector doesn't include them since they're copyrighted). They aren't hallucinations (Fable got the content from cases) but that kind of cite is where hallucinations are more likely to occur. That's one of the useful things about the review panel - it shows the places where Claude is citing to something it didn't directly read.

The Accuracy Check

The prompt has four sub-questions, and I assessed accuracy based on things that (in my attorney opinion) a correct answer would hit. I tried to focus on things that make a good binary (e.g. "did the AI find the controlling statute") since the more intangible aspects of legal writing are hard to test for. If I was scoring based on those soft factors, I would also say that Fable wrote the best memos, fwiw.

Test Lexis Westlaw Haiku ET Sonnet High Opus High Opus Max Fable High Fable Max Q1A — Determine Delaware law governs ❌ ✅ ❌ ✅ ✅ ✅ ✅ ✅ Q1B — Identify BOC § 1.104 is controlling ❌ ✅ ❌ ✅ ✅ ✅ ✅ ✅ Q2A — Identify DE two-part test (single econ entity + injustice) ❌ ✅ ❌ ✅ ✅ ✅ ✅ ✅ Q2B-1 — BOC § 21.223 is controlling (TX) ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ Q2B-2 — BOC § 101.002 bridges 21.223 to LLCs ✅ ❌ ✅ ✅ ✅ ✅ ✅ ✅ Q2B-3 — TUFTA actual-fraud transfer connects § 21.223(b) ❌ ✅ ❌ ✅ ❌ ❌ ✅ ✅ Q3-A — Reverse piercing inapplicable (creditor/debtor direction) ✅ ✅ ❌ ✅ ✅ ✅ ✅ ✅ Q3-B — Lead DE case recognizing rev. piercing ( Manichaean ) ❌ ✅ ❌ ✅ ✅ ✅ ✅ ✅ Q3-C — BOC § 101.112(d) charging-order exclusive remedy ❌ ❌ ❌ ❌ ❌ ❌ ✅ ❌ Q4A — Identifies & discusses controlling TUFTA provisions ❌ ✅ ✅ ✅ ✅ ✅ ✅ ✅ Q4B — TUFTA applies ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ Total (of 11) 4 9 4 10 9 9 11 10 Notes / errata:

- Lexis, Q4B: Since I'm only grading conclusions, Lexis gets credit here, but it went waaaay off the reservation in its reasoning. Its main source was a 1973 Delaware case. The Texas statute it was citing that case to interpret was passed in 1987, so obviously the source has nothing to do with the statute. It somehow wandered onto the right conclusion, but did so despite addressing zero provisions. Frankly, I was shocked at how badly it handled this issue.

- Q2B-2 - 21.223 is in the part of the Business Organizations Code that applies to Corporations, not LLCs. Not all laws applying to A second statute, BOC § 101.002, applies 21.223 to LLCs by reference. This therefore serves as a test of whether the AI model is paying attention to the corporation / LLC distinction.

- Q2B-3 (the TUFTA / § 21.223(b) connection): Some cases have recognized that actual-fraud asset transfers covered by TUFTA can satisfy the fraud requirement under 21.223, I gave credit to models spotting In re Ritz (Westlaw, Fable Max) or Spring Street (Sonnet, Fable High). The Opus runs noted the factual overlap but no authority connection that a TUFTA actual-fraud finding satisfies § 21.223(b) I didn't give them credit.

- Q3-A (BOC § 101.112(d)): This was the biggest miss by all the models except Fable High, which at least spotted the issue. Texas statutorily foreclosed reverse veil piercing and similar common sense remedies for LLCs in 2023 with this emphatic bit of legislation . It arose after a man who owed his ex-wife $385k for a personal injury judgement stuck his assets in an LLC wholly owned LLC he made. The Fort Worth Court of Appeals allowed a cousin-remedy to reverse veil piercing to keep him from hiding his assets behind an "I don't own anything but my LLC does" fiction. Outraged at the injustice of a man being forced to pay the ex-wife he injured, our always wise legislature said "never again." The case law cited by the models predates this amendment and is abrogated by it. Surprisingly, only Fable High even flagged the statute, and even it framed the clash as an "unresolved collision" rather than controlling which it is (at least in my opinion as a Texas attorney).