Skip to content
← All Posts
Build notes

A one-page checklist for law firms vetting an AI tool

I published a one-page checklist for law firms deciding whether to put client files into an AI tool. It turns ABA Formal Opinion 512 into plain questions, grouped by the Model Rule behind each one, and it is available as a PDF. It is meant for any AI vendor, including me.

Why a checklist

Opinion 512, issued in July 2024, is the ABA’s guidance on generative AI. It is careful and long, and applying it to the tool a vendor is demonstrating that afternoon is harder than agreeing with it in principle.

A list of questions travels better. You can hand it to the person running the demo and see which answers come back specific and which come back vague.

What it covers

The questions fall into six groups.

  • Where client information goes (Rule 1.6). Where processing happens, whether the tool learns from what you enter, who at the vendor can see it and how long it is kept.
  • Inside the firm (Rules 1.6 and 5.1). Whether the tool can respect an ethical wall, whether access can be limited by matter or client, and whether staff have a written policy and training.
  • Telling the client (Rules 1.4 and 1.6). When informed consent is needed, and why boilerplate in an engagement letter isn’t enough.
  • Checking the vendor (Rule 5.3). References, written security policies, a confidentiality agreement before any files change hands and where you could seek relief.
  • Checking the output (Rules 1.1, 3.1, 3.3 and 8.4(c)). Who verifies work before it reaches a client or a court, and whether citations are checked every time.
  • Billing for it (Rule 1.5). Billing actual time, not time saved, and keeping the cost of learning a general tool off client invoices.

One item goes beyond the opinion and says so: how fast a vendor will tell you about a breach, and how your data is returned or destroyed when the engagement ends. It isn’t in 512, but I would ask it of anyone.

Limits of the checklist

It is a list of questions, not legal advice. It does not replace reading the opinion or talking to your own ethics counsel. Georgia lawyers should check each item against the Georgia Rules of Professional Conduct, and the page says that too.

How my own setup answers it

I set up private AI systems for small firms, where core processing stays on-site on hardware the firm controls. That changes some of the answers, but remote support, email and integrations are still real connections. The checklist is a fair test of whether a vendor describes them honestly.

The site has a data-handling page that answers most of the list for my own setup: where processing happens, the data-flow register, access controls, incident notice and offboarding. If you would rather go through it live, I’ll answer every question on the list for the system I would build for your firm on a 30-minute call, and in writing afterwards if you want it.

The checklist and the PDF are at ai.duncansmith.tech/law-firm-ai-checklist.