Where May the Model Run? LLM Hosting and Professional Secrecy

Most reviews of AI use in law firms and medical practices stop at the question of whether a data processing agreement is in place. That is the wrong endpoint. For professionals bound by confidentiality, the decisive body of law is not data protection but criminal law, and there the question reads differently: can any person at the provider read the input without being part of the chain of obligation?

In brief: Doctors, lawyers and tax advisers in Germany may use AI services, but only if everyone who gains access to the inputs has first been bound to secrecy in text form and informed of the criminal consequences. A data processing agreement under Article 28 GDPR does not achieve this; it governs a different layer. In practice the chain rarely breaks at the contract. It breaks at abuse monitoring: large providers retain inputs by default and surface flagged cases to a human reviewer. Switching that review off is possible but tied to enterprise agreements, which creates a size threshold no statute provides for. Open model weights are the way out that the law has allowed since 2017 and that the market has only recently made practical.

May a professional bound by secrecy use cloud AI at all?

Yes. Since the reform of Section 203 of the German Criminal Code, in force since 9 November 2017, engaging external service providers is expressly permitted. Subsection 3 allows disclosure to "other persons participating in their professional activity" insofar as this is required in order to use that activity. The legislator explicitly had in mind the operation, maintenance and external storage of IT systems, which is to say the cloud case.

The prohibition therefore does not target the technology but the unsecured handover. Anyone still reading that cloud AI is categorically off limits for law firms is reading a position from before 2017.

The price of that permission sits in subsection 4: the professional must ensure that the participating person has been bound to secrecy. Responsibility stays with them, not with the provider. Where it goes wrong, the penalty in subsection 1, up to one year of imprisonment or a fine, falls on the doctor or the lawyer.

Why is a data processing agreement not enough?

Because the two frameworks protect different things and address different parties. The agreement under Article 28 GDPR protects personal data and addresses the controller in the data protection sense. Section 203 protects the secret entrusted to a person and addresses someone holding a particular profession.

The distinction is not academic. The German Federal Chamber of Tax Advisers notes in its AI guidance that professional confidentiality protects, "unlike the GDPR, data of non-natural persons as well". The balance sheet of a limited company contains no personal data and still falls under professional secrecy.

In practice this means two documents, not one. The data processing agreement covers the data protection layer, the confidentiality undertaking covers the professional one. A provider offering only the first has delivered half.

What does the chain of obligation actually require?

The professional codes are considerably more precise here than the criminal code. Section 43e of the Federal Lawyers' Act and Section 62a of the Tax Advisory Act describe the same mechanism with the same elements.

Three points in this table are routinely missed.

First, the notice. It is not enough for the provider to promise confidentiality. They must have been informed of the criminal consequences of a breach. A confidentiality clause in a standard contract does not satisfy this.

Second, the timing. The Federal Chamber of Tax Advisers puts it plainly: providers must be bound "before they gain knowledge of this data". A retrospective agreement cures nothing.

Third, the pass-through. Where the provider uses sub-processors, and virtually every AI provider does, they must in turn bind those parties in text form. The chain may not break at any link.

Where does the chain break in practice?

Not where most people look. The contract can be obtained from the large providers. Microsoft, for instance, maintains a standardised addendum, the Professional Secrecy Amendment for Germany, concluded through a partner and designed for exactly this purpose.

The chain breaks in operations, specifically at abuse monitoring. Large model providers retain inputs and outputs by default for a limited period in order to detect misuse. With Azure OpenAI the period is 30 days. If automated detection flags a case, an employee of the provider may inspect it.

That inspection is precisely the disclosure Section 203 addresses. A reviewer reading a medical letter is a third party gaining knowledge. Whether they may do so turns not on whether it is justified under data protection law, but on whether they are part of the chain of obligation.

There is a remedy, and it comes with a catch. Modified Abuse Monitoring removes the human review while leaving automated checks in place. It is not a switch in the management console but an application that must be approved, and it presupposes an enterprise contract, either an Enterprise Agreement or a Microsoft Customer Agreement.

That produces a threshold no statute provides for. A practice with four treatment rooms or a firm with three partners regularly fails to meet the condition for the exception. Professional law knows nothing of company size. The procurement route does.

One further ambiguity is worth knowing. The publicly documented scope of the Microsoft addendum covers Microsoft 365. Whether it covers the AI services in the same way is not publicly confirmed and belongs in writing before any commitment.

Which routes exist, and what do they deliver?

Four routes are open. They differ less in price than in how far the chain of obligation reaches.

The first route is the one the professional chambers expressly rule out. The Federal Chamber of Tax Advisers observes that with publicly accessible services, inputs are "transmitted to the service provider and processed there, without any specific confidentiality agreement", and treats this without adequate contractual cover as a breach of the duty of confidentiality.

The fourth route is the cleanest in law because it dissolves the question rather than answering it. Where nothing leaves the building there is no disclosure to a third party and therefore no chain that could break.

The third route is the most interesting for most, and it is new.

What do open model weights change legally?

They change nothing about the rule and a great deal about complying with it. As long as capable models were available only as the maker's own service, every route led inevitably back to that maker. Using GPT meant building a chain all the way to OpenAI. Using Claude meant one to Anthropic.

With openly licensed weights that inevitability disappears. The model can run at a provider you already hold a contract with, domestically, under German law. The maker of the weights takes no part in the process and learns nothing about the inputs.

The case from late August 2026 illustrated this well. The model that ran anonymously for six days as Ox Alpha turned out to be GLM-5.3-Flash and was released under an MIT licence. We assessed the episode separately: Ox Alpha was GLM-5.3-Flash. For professionals bound by secrecy the interesting part is not the benchmark but the licence. A model of that calibre may be run commercially without asking anyone, including in a data centre in Frankfurt.

German and European providers are picking this up. Offerings now exist that serve open models through an API from domestic data centres, certified to ISO 27001 and the German BSI C5 catalogue. Whether such a provider is suitable for confidentiality-bound professionals is decided not by the certificate but by the question from the previous section: is a confidentiality undertaking with the required notice on offer, and what happens to the inputs in operation?

What hardware does self-hosting require?

Less than the talk of data centres suggests, and more than an office machine provides. What matters is graphics memory, because the whole model has to fit inside it.

As a rule of thumb for four-bit quantised models: memory needed in gigabytes is roughly the parameter count in billions times 0.6.

For the typical tasks in a firm or practice, meaning summaries, drafts and search across your own documents, mid-range models are generally sufficient. The very large models are needed where complex reasoning matters, and that is precisely where outsourcing to a properly bound provider is usually the more economical answer.

An honest calculation covers more than the purchase. A server in the building means maintenance, updates, resilience and somebody responsible for it. The Federal Chamber of Tax Advisers frames the trade-off neatly: a locally installed system under the firm's control may offer more data protection, whereas a cloud service takes updates and maintenance off your hands.

How do you vet a provider in ten minutes?

Six questions are enough for a shortlist. Any provider that will not answer one of them in writing is out.

1. Do you offer a confidentiality undertaking for Section 203 StGB, separate from the data processing agreement, including express notice of criminal liability?
2. Where are inputs processed, and where are they stored? Please give the location, not a region name.
3. Are inputs retained for abuse detection? If so, for how long, and can a human inspect them?
4. Can human review be disabled, and what type of contract is that tied to?
5. Which sub-processors are involved, and are they bound to confidentiality in text form?
6. Are inputs used for training? If excluded, where in the contract does that appear?

Questions three and four are where offerings separate. They are rarely answered unprompted in a sales conversation, because they mark the difference between a data protection promise and a professional-law commitment.

Question two deserves a warning. Labels such as "EU data zone" or "European hosting" are product names, not location commitments. Ask for the country and the operator of the facility.

What applies to doctors, lawyers and tax advisers respectively?

The criminal-law core is identical; the professional wrapper differs.

For doctors Section 203 applies directly, supplemented by Section 9 of the model professional code and the applicable state code. There is no dedicated provision with the level of detail of Section 43e BRAO, which in practice means the requirements have to be read together from criminal law and data protection law. What this looks like for a phone assistant we have described separately: AI phone assistant in a medical practice.

For lawyers Section 43e BRAO spells out the contractual duties item by item. Meeting that list also satisfies Section 203.

For tax advisers Section 62a StBerG performs the same function, with two particularities. Subsection 4 requires a level of protection comparable to the domestic one where services are performed abroad. Subsection 5 requires the client's consent where the service relates to a specific individual engagement rather than being a general working tool of the firm. The Federal Chamber of Tax Advisers concedes that the distinction cannot be drawn "beyond doubt" for AI, and recommends obtaining consent in case of uncertainty. We covered the professional-law side for firms here: AI in the tax firm.

A note on currency: the guidance from the Federal Chamber of Tax Advisers is dated 27 January 2026, and by its own statement the answers rest on the legal position as at July 2025. For the provisions discussed here that changes nothing, they apply unchanged. On points of detail the date is worth checking.

Frequently asked questions

May I use ChatGPT as a lawyer or doctor? For general tasks unrelated to clients or patients, yes. As soon as you enter details that allow conclusions about a specific matter, no, unless a confidentiality agreement with the provider is in place. The Federal Chamber of Tax Advisers treats the use of publicly accessible services with client data and no contractual cover as a breach of the duty of confidentiality.

Is a data processing agreement enough for Section 203 StGB? No. It governs data protection under Article 28 GDPR, not professional secrecy. What is required in addition is an undertaking of confidentiality in text form, combined with notice of the criminal consequences of a breach. The two documents sit side by side.

What is abuse monitoring, and why is it a problem? Providers retain inputs temporarily to detect misuse, in the case of Azure OpenAI for 30 days. If automated detection flags a case, an employee of the provider may inspect the content. For a professional bound by secrecy that is a disclosure to a third party, permissible only if that third party is part of the chain of obligation.

Can I switch off human review? With Azure OpenAI, through Modified Abuse Monitoring, which removes human review while keeping automated checks. It requires an approved application and presupposes an enterprise contract, either an Enterprise Agreement or a Microsoft Customer Agreement. Small units frequently do not meet that condition.

Do I have to run the model myself? No. Self-hosting is the legally simplest route because no external disclosure occurs, but it is not the only permissible one. A domestic provider that offers a confidentiality undertaking with the required notice and permits no human inspection of inputs meets the requirements as well.

What do open model weights change for firms and practices? They decouple the model from its maker. A model under an open licence may be operated at a domestic provider or on your own premises without the maker of the weights taking any part. That makes the chain of obligation short enough to actually close.

Who is liable if the provider makes a mistake? Under criminal law, the professional. Section 203 subsection 4 obliges them to ensure that the participating person has been bound. If they fail to do so, the penalty in subsection 1 applies to them regardless of how the provider behaved.

Conclusion

The question of permissible AI use in firms and practices is usually argued on the wrong level. Data protection assessments are necessary, but they do not answer whether a person at the provider may read the file. That question belongs to criminal law, and it has a clear answer: only if that person has been bound and given notice beforehand.

Taken seriously, this resolves into a manageable review. Two documents rather than one, a reliable statement about what happens to inputs in operation, and a chain closed down to the last sub-processor.

The practical news is nonetheless good. Until recently the road to a capable model led inevitably to a very large provider with whom a small practice negotiates no individual terms. Open model weights have changed that. A rule written in 2017 described a route the market has only made passable in 2026.

A closing note: this article explains the legal position and does not replace advice on an individual case. For actual contract drafting, a professional-law specialist belongs at the table.

Sources

• Section 203 of the German Criminal Code (StGB), violation of private secrets, as in force since 9 November 2017, in particular subsections 3 and 4
• Section 43e of the Federal Lawyers' Act (BRAO), use of services
• Section 62a of the Tax Advisory Act (StBerG), use of services, in particular subsections 2 to 5
• German Federal Chamber of Tax Advisers, FAQ on AI in the tax advisory profession, dated 27 January 2026, answers based on the legal position as at July 2025
• Microsoft documentation on abuse monitoring and Modified Abuse Monitoring for Azure OpenAI, and on the Professional Secrecy Amendment for Germany