I ran a test to see how AI measured up against my legal writing, and here’s what I found.

I took my last 10 projects and ran them through AI, one at a time.

The AI drafts looked good. They were organized, confident, and full of real case law. They would have survived a quick read.

Then, I reviewed the case documents, researched the issues, and edited the AI drafts to my standards.

Here is what I learned from comparing AI’s versions with my own.

AI missed the mark on a lot of important points, and a citation check wouldn’t have caught any of it. Here’s what went wrong.

1. It found the best fact and didn’t recognize it.

In a contract dispute (where my client represented a homeowner suing a business), the main issue was the applicability of a disclaimer. The legal test was whether an ordinary person could understand what they were giving up in the disclaimer.

The business owner admitted in his deposition that his own disclaimer was so technical that only someone in his industry could understand it. He testified that he wrote it himself and that not even a lawyer could understand it.

That admission was the best weapon, and AI glossed over it.

The AI mentioned it once, buried in the middle of a paragraph. In its second draft, it dropped it completely.

I put it up front so the judge couldn’t miss it, quoted the full testimony, and tied it straight to the legal test.

2. It spotted the winning point and never explained why we win.

In one case, the AI found a statute that created a presumption in our favor, triggered by specific contract language. The AI cited the statute but never showed the court the contract language that triggered it. I quoted the contract multiple times and drilled down on this issue.

In another case, the defendants didn’t file a sworn affidavit, which meant the burden never shifted to our client. The AI noticed the missing affidavit but never said why it mattered. I built the whole response around it.

3. It found great cases and didn’t properly connect them to our legal issue.

This is one of my biggest frustrations with AI. It finds an amazing case, summarizes the holding in one or two sentences, and moves on. It skips the part that actually persuades a judge: drawing out the facts and comparing that case to ours.

Over and over, the AI gave a key case one or two sentences, said it supported us, and jumped to the conclusion. The judge was left to figure out why that case mattered. And most judges won’t do that work for you.

Here’s how a key case should be applied:

  • Tell the judge what happened in that case, in plain English.
  • Line up the facts that matter.
  • Quote the case’s own words, so the judge sees the holding, not your spin on it.
  • Say out loud why the same result should follow in your case.

That’s the difference between citing a case and using it to your advantage. AI rarely does this part well. This still takes a lawyer’s judgment.

4. It hedged when it was winning.

In one case on an evidentiary matter, the other side filed no evidence with its motion. We were clearly winning, but the AI still offered the judge three options other than denying the motion: let us amend, give us discovery, hold a hearing. It asked for all three again at the end.

I asked for one thing: deny the motion.

Sometimes a fallback argument is the right call, but AI can’t tell when to use it and when to double down on a winning argument.

5. It picked cases that looked like ours instead of cases that solved our problem.

The AI picked its lead case because the facts looked very similar to ours. The problem with the case was that there was one critical factual difference that made the entire case distinguishable. In fact, the AI even admitted the case was different based on this one fact. So the case the AI led with was useless.

I did the research and couldn’t find similar facts, but I did find a case with the same legal problem, where the factual differences didn’t make the case distinguishable in any substantive way. Facts matter. But two cases can look different and still apply the law the same way. Knowing which differences matter (and which don’t) is judgment. AI doesn’t do this well.

6. It spread arguments over too many headings and subheadings and made them unnecessarily complicated.

AI is famously long-winded. In every draft, I cut and condensed the arguments so the point came through faster and clearer.

But many attorneys are not properly correcting AI, and courts are noticing. In September, the Fourth DCA in Lisandrillo v. Palozzi, 51 Fla. L. Weekly D1922 (Fla. 4th DCA Sept. 16, 2026), said a lawyer’s filings appeared to be “AI slop,” and the problem wasn’t fake cases. In addressing the issue, the court quoted from a Bloomberg Law article describing the predominant problem faced by AI legal filings: “The citations are real. The cases exist. The language is confident. Yet the analytical reasoning—or lack of it—reveals that the arguments were assembled by an AI tool rather than developed through a trained advocate’s independent judgment.” Id. at *1 n.1. The attorney in Lisandrillo was later referred by the court to the Florida Bar. Lisandrillo v. Palozzi, No. 4D2026-2262, 2026 WL 3147909, at *1 (Fla. 4th DCA Oct. 21, 2026).

The lesson is that just because AI provides a long and seemingly organized document does not mean it’s making good arguments. And even when the arguments are good, these long, convoluted documents bury the winning points in a word salad judges can’t easily digest.

7. It put things in the wrong order.

Deciding which argument leads, and how the rest lines up, is a judgment call. AI still doesn’t know when to follow the order of the other side’s arguments and when to move winning arguments ahead of all the rest.

In one case, a non-lawyer signed motions on behalf of a corporation. That alone meant the corporation’s motions were null and void. The AI put that argument last. I put it first because it made all the other arguments moot.

In another case, the other side filed a motion to vacate a judgment based on the wrong rule. They wanted the judge to change the actual ruling but were using an inapplicable rule meant to correct typos and clerical errors. That one point was enough to deny the motion because no rule applied in that situation, let alone the one cited. But the AI put that argument last. I put it first, and although I addressed the rest of the motion, I let the judge know he could deny the motion based on that one point.

To be fair

Two of the 10 AI drafts were close to usable, and the AI never made up a fact or a case. When it didn’t know something, it left a blank or asked for more information. But that’s due, in large part, to the training, instructions, and rules that I have developed for it.

Even with all the training I have done, the AI product is still not reliable, and making sure everything is accurate and complete can take just as long as drafting it myself.

So, if you are using AI for your legal drafting, remember that you still have to put the time in to correct and double-check all of these issues.

AI is our tool. Judgment is our job, and legal judgment cannot be delegated to AI.