Why law and accounting firms need a private AI assistant, not public chatbots
A private AI assistant for law and accounting firms keeps client documents in your own cloud. What it looks like, how it is secured and what to ask first.
By FutureGen Systems
If you run a law or accounting firm, someone on your staff has almost certainly pasted client material into a public chatbot this year. They were not being careless. They had a long engagement letter to summarise or a messy ledger to explain, and the tool was right there in a browser tab. A private AI assistant for law and accounting firms is the way to give them that same convenience without the firm losing track of where client data goes.
This post covers why public tools are a problem for professional firms, what a private assistant actually involves, the two main ways to host the model, and the questions worth asking before you buy anything.
The problem with public AI tools in a professional firm
The risk is not that a chatbot is "evil". It is that the firm loses three things it is normally careful about.
- Control over where data goes. Once a contract or a set of accounts is pasted into a consumer tool, it sits on someone else's systems under someone else's terms. Those terms differ between free, paid and enterprise tiers, and staff rarely know which one they are on.
- Visibility. There is no log the firm can review. If a client, regulator or insurer asks "has our data been put into an AI tool?", the honest answer is usually "we do not know".
- Permissions. A public tool has no idea that a trainee should not see the partner-only file on a sensitive matter. Your document management system does. A chatbot that sits outside it ignores everything you have set up.
Banning AI outright rarely works. People keep using it on their phones, and the firm ends up with the same risk and less visibility. The more practical answer is to give staff a sanctioned tool that is as easy to use as the public one and sits entirely inside the firm's own environment.
What a private AI assistant actually looks like
From a staff member's point of view, it is a chat window. They type a question, and it answers using the firm's own documents, with references back to the files it drew on. Underneath, a sound setup has a handful of parts.
It runs in your environment
The chat application, the search index over your documents and the model endpoint all live in your firm's Azure or AWS account, in a region you choose, or on a private server you own. Nothing is processed on the supplier's systems. If the supplier disappears tomorrow, the assistant keeps running, because it is yours.
Sign-in is your existing sign-in
Staff log in with single sign-on through Microsoft 365 or Google Workspace. No new passwords, and when someone leaves, disabling their account removes their access to the assistant too.
It respects the permissions you already have
This is the part that separates a serious build from a demo. When the assistant searches your SharePoint, OneDrive, Google Drive or file share, each user should only get answers from documents they could already open themselves. The permission check happens at search time, before anything reaches the model. If a trainee asks about a matter they are walled off from, the assistant should behave as if those documents do not exist.
Logging, retention and redaction
Every prompt and response is logged inside your environment, with retention rules you set. You can redact client identifiers such as names, account numbers or tax references before they are stored or sent to the model, depending on what your policy requires. When someone asks what the AI has been used for, you have an answer.
A security summary you can hand over
Regulators, professional indemnity insurers and larger clients increasingly send questionnaires about AI use. A good deployment comes with a short written summary: where the data lives, who can access it, what is logged, how long it is kept, which model is used and under what terms. That document often matters as much as the software.
Private model endpoints vs open models on your own hardware
There are two broad ways to run the model itself. Neither is universally better.
| Private endpoint in your cloud | Open model on your own hardware | |
|---|---|---|
| Example | Azure OpenAI or a similar managed service in your own tenant | An open-weight model running on a server you own |
| Answer quality | Generally the strongest available models | Good and improving, but usually a step behind the frontier |
| Where data goes | Stays within your cloud tenant and chosen region, under the provider's enterprise terms | Never leaves your hardware |
| Running cost | Pay per use | Upfront hardware or a dedicated GPU server, then mostly fixed |
| Maintenance | Provider handles the model; you manage the app around it | You (or your supplier) patch, update and monitor everything |
| Good fit | Most firms already on Microsoft 365 or AWS | Firms whose regulator or clients forbid any third-party processing |
For most small and mid-sized firms, a private endpoint inside their existing cloud tenant is the sensible default. It keeps data inside an environment they already trust for email and documents, and it avoids buying and maintaining GPU hardware. A fully on-premise open model makes sense when a client contract or regulator rules out any external processing at all, or when usage is high and steady enough that fixed hardware is cheaper over time.
Whichever you choose, read the provider's data-use terms for the exact service and tier you are buying, and keep a copy with your security summary.
What to ask before you buy
Whether you build it with us or someone else, these questions separate a solid deployment from a risky one.
- Where exactly does each component run? The chat app, the document index, the logs and the model. "In the cloud" is not an answer. Ask for the account, the region and who owns the billing.
- Does the supplier ever hold a copy of our data? The right answer for a professional firm is no. Setup should happen through a scoped, time-limited admin account that you control and can revoke.
- How are document permissions enforced? Ask them to show a user being denied an answer from a document they cannot open. If permissions are "on the roadmap", walk away.
- What is logged, where, and for how long? And can you turn redaction on for specific identifiers?
- Which model, under which terms? Get the specific service and the terms that govern it, not a brand name.
- What happens when it is wrong? Answers should cite the source documents so staff can check them. A tool that answers confidently without sources will eventually cause a problem in client work.
- Who maintains it after launch? Models, libraries and connectors change. Someone needs to apply updates, check backups and review usage.
- What will you give us in writing? A scope, a data processing agreement and the security summary, before any work starts.
Start small, with one document source
The firms that get value quickly tend to start narrow: one document source, such as the precedents library or the internal knowledge base, a small group of users, and a short training session. Once staff trust the answers and the logs show how it is being used, adding further sources is straightforward.
We build these assistants on the same retrieval approach we have used before. Past AI builds include Medi-Insights and Mufai, retrieval-augmented assistants over custom knowledge bases on Azure OpenAI and AI Search. Our AI and automation services cover the wider picture, and our process explains how an engagement runs from first call to handover.
If your firm is weighing this up, our Private AI Workspace page sets out what is included, and it starts with a free readiness call and a written plan.