Solutions & Procurement · Pharma & life sciences · Written and maintained by Haink’s AI adoption team · Updated August 2026 · 12 min read
Private LLM deployment for pharma — the decision, the stack and what it costs
Every pharmaceutical LLM project runs into the same gate before a line of code is written: where the model is allowed to run. It is usually framed as an infrastructure question, decided by IT, and revisited eighteen months later when legal or the quality unit finds out what was actually deployed.
It is better treated as four options with different consequences, decided once, deliberately. This page is the decision, then the stack that follows from it, then what it costs.
The four deployment modes, and what each one actually gives you
| Mode | Where data goes | Model version control | Realistic pharma fit |
|---|---|---|---|
| Public API | Third-party infrastructure, outside your control | None — the provider can change or retire a version | Non-confidential work only. Most IP counsel rule it out for compound, process or GMP data. |
| Vendor SaaS with contractual protections | Vendor infrastructure, under contract | Limited, set by the vendor's roadmap | Workable for commercial and general corporate use. Contracts constrain use, not architecture. |
| Private cloud tenancy | Your cloud account, your VPC | Good, if you host the weights yourself | Reasonable middle ground where cloud is already accepted for regulated workloads. |
| On-premises or air-gapped | Never leaves your perimeter | Complete — you pin the version | The only option where counsel forbids egress or where a site is isolated by policy. |
Two clarifications worth making early, because both get overstated in vendor material.
No regulation requires on-premises deployment. Neither the draft EU GMP Annex 22 nor 21 CFR Part 11 says where a model must run. What they require is control: documented intended use, frozen model versions under change control, provenance you can evidence, and audit trails you own. On-premises is one way to satisfy those; it is not the only one.
But two requirements push hard in that direction. A model must be static — and a managed endpoint whose version can change on the provider's schedule is difficult to hold still. And there is no vendor black box: responsibility for validation evidence stays with the regulated company regardless of who built the model. If you cannot describe what the model is, how it was trained and what version is running, that is your gap. Self-hosted open-weight models make both problems disappear, which is why most pharma deployments end up there even when confidentiality alone would not have forced it.
What actually forces the decision in practice
In our experience the deciding factor is rarely the regulation. It is one of four constraints, and it is worth naming which one applies to you before comparing architectures.
- IP counsel. Compound structures, formulations, process parameters and unpublished results are trade secrets. Counsel frequently forbids egress in any form, including encrypted transit to a trusted provider — the position taken by the computational chemistry team in our life sciences project, where cloud HSMs were explicitly not an acceptable alternative.
- GMP records. Batch records, deviations and OOS reports are regulated records with retention, integrity and audit-trail obligations. Putting them through a system whose logs you do not control creates a problem that is easier to avoid than to solve.
- Cost at steady load. Document workloads are not bursty. A review assistant processing every batch record produces a predictable, continuous token volume — the profile where owned hardware wins over metered inference, and the one where cloud bills surprise people at scale.
- Data residency. In several of the markets we supply, sector or national rules constrain where regulated data may be processed. This is a jurisdiction question, distinct from the confidentiality one — see cloud AI vs private AI.
The stack, layer by layer
A private LLM deployment is six layers. Most projects budget for the first and are surprised by the fourth and sixth.
| Layer | What it is | Where it usually goes wrong |
|---|---|---|
| 1. Compute | GPU nodes, network, storage. Sized to workload, not to ambition. | Bought before the workload is characterised. |
| 2. Serving | vLLM or NVIDIA NIM, model catalogue, quotas, batching. | Treated as trivial; concurrency and queueing are what users actually feel. |
| 3. Models | Open-weight models you host and pin. Size chosen per task, not one model for everything. | Assuming fine-tuning is needed when retrieval would have sufficed. |
| 4. Knowledge | Ingestion from DMS, QMS, LIMS, MES; chunking; embeddings; version-aware retrieval. | The real project. An answer grounded in a superseded SOP is worse than no answer. |
| 5. Application | The specific workflow — review assistant, SOP search, drafting — plus API for existing systems. | Built as a chat box, when the value is in a bounded workflow. |
| 6. Control | SSO and RBAC, full logging of prompt, output, sources, model version and reviewer; drift monitoring; validation artefacts. | Left until qualification, then retrofitted at three times the cost. |
Layer four is where the time goes. It is also the layer that determines whether the system is trustworthy: retrieval that returns the effective version of a document, with a citation the reviewer can click, is the difference between a tool people use and a demo they stop opening. The general pattern is covered in LLM applications and RAG; the document-specific work in document intelligence.
Sizing: match the hardware to the workload shape
Two workload shapes dominate pharma, and they size very differently.
Batch document processing — reviewing records, extracting fields, cross-checking. Throughput-bound, tolerant of latency, runs unattended. You can saturate a node and it does not matter if a document takes twelve seconds.
Interactive assistance — SOP search, question answering, drafting. Latency-bound and concurrency-bound. Ten simultaneous users who each wait eight seconds for a first token will abandon the system regardless of how good the answers are.
| Starting point | What it suits | Indicative anchor |
|---|---|---|
| Desktop AI (NVIDIA DGX Spark, RTX 6000 Ada workstation) | Development, evaluation, single-team pilots, isolated labs | From ~$4,000 — see private AI infrastructure |
| Single inference node (H200 NVL) | Serving 70B–180B models for one site or one department | From ~$78,000 — see inference infrastructure |
| Starter stack | First production deployment with storage and networking included | From $95,000 — reference architectures |
| Inference pod | Multi-site or multi-workload production, with headroom | From $350,000 — reference architectures |
| Fine-tuning and training cluster | Domain adaptation on proprietary corpora, model development | From $700,000 — reference architectures |
The most common sizing error is buying for the model rather than the load. A 70B model serving four concurrent reviewers and the same model serving four hundred are entirely different purchases. Characterise the load first: documents per day, average tokens per document, peak concurrent users, acceptable first-token latency. Those four numbers determine the node count, and they can be measured on a workstation before anything is ordered.
What it costs, honestly
Owned hardware wins on sustained, predictable load and loses on sporadic experimentation. That is the entire economic argument, and any comparison that ignores utilisation is selling something.
Two reference points from our own deliveries: an H200 inference cluster where the break-even against equivalent rented capacity landed at 14 months, and the pharmaceutical R&D deployment above where estimated payback against a comparable GPU reservation was about 12 months for that workload profile. Both assume the hardware is actually busy. A node at 15% utilisation has no payback story, and no amount of infrastructure enthusiasm changes that.
Costs that get omitted from the first budget and should not be: the data layer work in layer four, which is usually the largest line; validation artefacts if the system touches GxP; power and cooling if the site was not built for GPU density; and someone's time to own the thing after handover. If cloud spend is the trigger for the conversation, start with an infrastructure audit or a cloud exit assessment rather than a hardware quote — the answer is sometimes to stay.
What the deployment has to produce for a regulated site
If the system will touch GxP processes, these are deliverables, not documentation added afterwards. Scoping them at the start is what separates a system that gets qualified from one that gets shelved.
- Pinned versions. Model weights, prompts and configuration under change control, with a documented process for updating them.
- Complete audit trail. Prompt, output, retrieved sources with versions, model version, user, timestamp, reviewer disposition — retained under Annex 11 principles.
- Explainability and confidence. Confidence persisted with each output and an abstention path that routes to a human rather than forcing an answer.
- Intended use and boundaries. Written, approved by a process expert, stating what the system must not be used for.
- Acceptance evidence. Performance measured against the process being assisted, on an independent test set, with metrics agreed before testing.
- Qualification package. URS, IQ/OQ/PQ, drift monitoring plan and periodic review schedule.
This is also the answer to the question of who does the work. A hardware supplier will not produce these; a software house will not deliver the cluster. We do both, which is the reason the two halves land at the same time — described in AI solution implementation. For the document side of a GxP deployment specifically, document extraction under validation covers what these deliverables mean architecturally, and clinical document automation is what we build against them.
Four ways these projects go wrong
Hardware before workload. The cluster arrives, the data access does not. The single most reliable predictor of a stalled deployment is an approved capital budget and an unapproved data request.
Fine-tuning as a reflex. Most pharma use cases are retrieval problems — the model does not need to memorise your SOPs, it needs to find and cite the current one. Fine-tuning adds a validation burden under Annex 22 that retrieval does not, and it is frequently solving a problem the retrieval layer created.
A chat box instead of a workflow. Open-ended assistants produce open-ended risk and unmeasurable benefit. Bounded workflows with defined inputs, outputs and acceptance criteria are what a quality unit can approve and what finance can measure.
No evaluation harness. Without a retrospective test set and agreed metrics, nobody can say whether the system is better than the process it assists — which is precisely what Annex 22 asks you to demonstrate. Build it in the first sprint; retrofitting it is painful and always suspiciously flattering. The broader pattern is in why AI pilots fail.
Frequently asked questions
Does pharma regulation require an on-premises LLM?
No. Neither the draft Annex 22 nor 21 CFR Part 11 specifies where a model runs. They require documented intended use, frozen versions under change control, documentable provenance and audit trails you control. On-premises is the simplest way to satisfy all four, and confidentiality obligations often make it the only practical one — but it is a consequence, not a mandate.
Which models can we run privately?
Open-weight models, hosted and pinned by you — the class of model where you can state exactly what is running and keep it from changing. Model size follows the task: extraction and classification need far less capability than drafting or multi-document reasoning, and running one large model for everything is usually the more expensive way to get worse latency.
What does a private LLM deployment cost to start?
Development and evaluation can start on desktop AI hardware from around $4,000. A production inference node capable of serving 70B–180B models starts from around $78,000, and a complete starter stack with storage and networking from $95,000. The data and integration work is usually the larger line item, and it is not a hardware cost.
Do we need to fine-tune on our own documents?
Usually not. Retrieval over your documents answers most pharma use cases and keeps the validation burden lower, because the model stays a fixed, off-the-shelf artefact while your knowledge base changes. Fine-tuning becomes interesting for domain language or output format at scale — and it makes retraining a change-controlled event.
How long does it take to stand one up?
Infrastructure is rarely the constraint — deployments of this size are typically a matter of weeks once hardware is on site. Access to data usually is the constraint. Scope the data-access approvals in parallel with procurement, or the cluster will sit idle waiting for a permission that nobody owns.
Can we start small and grow?
Yes, and it is the right sequence. Characterise the workload on a workstation, prove the retrieval layer against real documents, then size the production node from measured numbers rather than estimates. Growing a private deployment is a procurement exercise; rebuilding a badly sized one is a project.
Private LLM deployment, hardware and system in one delivery
We size the compute against your actual workload, build the retrieval and application layers on your documents, and hand over a running system with the audit trail and qualification artefacts a regulated site needs.
Discuss a private deployment Private AI infrastructure → Reference architectures →