HIPAA-Compliant GPU Cloud: Self-Hosting LLMs for Healthcare

There is no such thing as a HIPAA-certified LLM. No certification like that exists for any piece of software, and every vendor claiming otherwise is selling you a marketing line. What can be HIPAA-compliant is a deployment: a specific model, running on specific infrastructure, wrapped in specific contracts and controls (definite.app).

That distinction matters more than it sounds. Most healthcare teams evaluating AI infrastructure start by asking "which LLM is HIPAA compliant" and end up chasing a BAA from a model vendor when the actual fix is architectural: don't send PHI to a vendor at all. Self-hosting an open-weight model on infrastructure you control means no model provider ever sees a patient record, which sidesteps the vendor-BAA question entirely. It doesn't sidestep HIPAA. The Security Rule's access control, audit logging, and encryption obligations land on whoever operates the infrastructure, full stop.

This post walks through the four architectures healthcare teams actually use, why the BAA chain gets complicated fast with hyperscaler and vendor APIs, and the concrete checklist for making self-hosting defensible if OCR ever asks. For the parallel regulatory picture in Europe, see our EU AI Act compliance guide, which covers the same audit-logging and data-residency ground under a different statute.

Why No LLM Is "HIPAA Certified" (And What Compliance Actually Means)

HIPAA regulates covered entities and their business associates, not software products. A model file sitting on a hard drive has no compliance status of its own. What gets audited is the full chain: where the data goes, who can touch it, how access is logged, and whether the right contracts are in place with everyone in that chain (definite.app).

This is why you'll see vendors market "HIPAA-ready" models and "HIPAA-compliant AI platforms," and why both phrases are doing a lot of unearned work. A HIPAA-ready label usually means the vendor will sign a BAA and has some controls in place. It does not mean using their product automatically makes your application compliant. You still have to configure it correctly, and you're still on the hook if you don't.

The Four Architectures Healthcare Teams Actually Use

In practice, there are four ways healthcare teams deploy LLMs against PHI-touching workloads in 2026, and the difference between them is entirely about who else's infrastructure your data crosses:

(Source: definite.app HIPAA-compliant LLM guide)

Architectures 1 and 2 put you in a contractual relationship with a model provider that sees your PHI. Architecture 3, self-hosting on rented infrastructure, removes the model vendor from that relationship but keeps the infrastructure provider in it. Architecture 4 removes everyone. That's the honest map, and most of the "which LLM should we use" conversations in healthcare AI skip straight past it.

What the HIPAA Security Rule Actually Requires (45 CFR 164.312)

The Security Rule's Technical Safeguards, codified at 45 CFR § 164.312, are the part that actually constrains your infrastructure choices. They require:

• Access control - unique user identification, automatic logoff, and encryption/decryption mechanisms for systems that touch ePHI
• Audit controls - hardware, software, or procedural mechanisms that record and examine activity in systems containing ePHI
• Integrity controls - mechanisms to confirm ePHI hasn't been improperly altered or destroyed
• Transmission security - protection against unauthorized access to ePHI being transmitted over a network

None of these are model requirements. They're infrastructure requirements. A self-hosted Llama or Qwen deployment can satisfy all four; so can a hyperscaler endpoint. The gap most teams miss is that on shared GPU cloud, PHI sits as plaintext in GPU VRAM during inference unless the hardware supports confidential computing, which is a real hole in the transmission-security and integrity story that TLS and disk encryption don't cover. We cover NVIDIA's TEE mode and what it closes later in this post.

Self-Hosting Open-Weight Models: What You Own vs What a BAA Covers

Self-hosting changes who's in your compliance chain, not whether you have one. Get this distinction right before you build anything.

The BAA Chain Problem With Hyperscaler and Vendor APIs

Signing a BAA is not a blanket grant of HIPAA coverage. Most BAA-covered endpoints require specific configuration to actually stay in scope, and the default setup is often the wrong one:

Amazon Bedrock is on AWS's public HIPAA-eligible services list. You sign the AWS Business Associate Addendum, self-serve via AWS Artifact, and that agreement extends to Bedrock's model sub-processors (Anthropic, Meta, Mistral) when you're actually running a HIPAA-eligible model and configuration. The catch is verification, not a signature: confirm the specific model and configuration you're using is on AWS's published HIPAA-eligible list and matches the reference architecture, since not every model or endpoint on Bedrock qualifies (AI BAA vendor guide, 2026). Skip that check and you may believe a model is covered when it isn't.

Azure OpenAI Service is Microsoft's HIPAA-eligible path for API access to OpenAI's models, routed through Azure under the Microsoft Online Services DPA and existing enterprise BAA (AI BAA vendor guide, 2026). The same rule from Bedrock applies here: confirm the specific model and endpoint you're calling is actually on Microsoft's covered list before you route PHI through it.

Anthropic's BAA covers only the first-party API and a HIPAA-ready Enterprise plan. It explicitly does not cover Claude Free, Pro, Max, Team, self-serve Enterprise, Workbench, Console, or Cowork (AI BAA vendor guide, 2026). A clinician who signs up for Claude Pro on a personal card and starts pasting patient notes into it is not covered by anything, no matter what her employer's enterprise contract says.

OpenAI only offers a BAA to sales-managed ChatGPT Enterprise or Edu customers. There is no BAA path for ChatGPT Business (AI BAA vendor guide, 2026). Same failure mode as above: a team on the wrong tier assumes coverage that was never signed.

The pattern across all four: the BAA is real, but it's narrower than the product surface, and the gap between "we have a BAA with this vendor" and "this specific workload is covered" is where most healthcare AI compliance failures actually happen.

Why Self-Hosting Sidesteps the BAA Question Entirely

Serving an open-weight model, Qwen, DeepSeek, Llama, or Mistral, on infrastructure you operate means no model provider ever touches PHI. There's no sub-processor list to check, no endpoint-tier gotcha, no "wait, is this the covered plan" moment. The model file runs on your compute. Nobody at Alibaba, DeepSeek, Meta, or Mistral AI ever receives your patient data, because you never send it to them (definite.app).

That's a genuinely simpler compliance story than chasing sub-processor acknowledgments across three vendors. It is not a compliance story with zero obligations. If you're renting GPU capacity rather than owning the hardware, you still need a BAA with the infrastructure provider, because they could physically access the machine PHI runs on. The only architecture that removes the BAA question completely is fully on-premises hardware you own outright, covered in more depth in our on-premise vs cloud LLM inference guide, which walks through the cost and break-even math for that option.

Self-hosting improves control. It does not automatically produce compliance. The Security Rule's obligations land on whoever operates the infrastructure regardless of whether that's a hyperscaler or your own DevOps team (definite.app).

Which Open-Weight Models Fit (Qwen, DeepSeek, Llama, Mistral) and VRAM Sizing

By early 2026, open-weight model families from Qwen, DeepSeek, Meta's Llama line, and Mistral had reached a point where licensing terms, output quality, and memory footprint all support running them inside a healthcare org's own infrastructure (definite.app). None of that matters if you can't size the hardware correctly, so here's the practical range for clinical documentation and RAG-over-records workloads:

For the full tier-by-tier breakdown with KV cache math and quantization tradeoffs across the model catalog, see the best open-source LLMs to self-host guide. One note specific to healthcare deployments: skip consumer RTX cards for anything customer-facing or production. NVIDIA's GeForce EULA restricts data center use of consumer GPUs, and having your infrastructure licensing come up in an audit is an unforced error. The L40S on Spheron is the enterprise-certified, data-center-compliant option at the 48GB tier, and runs the same 70B-class models at INT4 without the EULA question.

Architecture Checklist: Encryption, Access Control, Audit Logging, Data Residency

Self-hosting gets you out of the model-vendor BAA chain. It puts the entire Technical Safeguards burden on your own build. Here's what actually has to be in place.

Encryption at Rest, in Transit, and in Use (Confidential Computing for VRAM)

Encryption at rest and in transit is table stakes: encrypted model storage, TLS on every inference API call, encrypted backups of any logs containing PHI. The part teams miss is encryption in use. PHI passed through a model during inference sits as plaintext in GPU VRAM on standard hardware, readable by anyone with hypervisor-level access to the host. NVIDIA Confidential Computing mode, available on H100, H200, B200, and GB200, closes that specific gap with hardware-level VRAM encryption and remote attestation. It is not a HIPAA certification on its own, but combined with TLS and encrypted storage, it completes the encryption chain end to end. The full deployment walkthrough, including KMS integration and the performance tradeoffs, is in our confidential GPU computing guide.

Access Control and Role-Based Permissions on the Inference API

45 CFR § 164.312(a) requires unique user identification and the ability to control who can access ePHI systems. On a self-hosted inference stack, this means role-based access control in front of your model API, not just an API key everyone on the team shares. Put your inference endpoint behind an auth layer that ties every request to a named user, scope permissions by role (clinician, researcher, admin), and set automatic session timeout. If you're deploying retrieval-augmented generation over patient records, lock down read access to the underlying vector store with the same rigor as the model API; a RAG pipeline that's access-controlled at the model layer but wide open at the document store isn't controlled at all.

Audit Logging: What to Capture for a HIPAA Audit Trail

Audit controls under 45 CFR § 164.312(b) mean you need to reconstruct who accessed what, when, and what happened. For a self-hosted LLM stack, the minimum capture set is:

• Timestamp of every inference request
• User or service identity making the request
• Whether the request or response contained PHI (flagged, not necessarily logged verbatim)
• Model version that served the request
• Any access to the underlying document store or vector database

For vLLM deployments, request logging is a flag away:

vllm serve meta-llama/Llama-3.3-70B-Instruct \
  --enable-log-requests \
  --enable-log-outputs

Ship those logs to access-controlled, encrypted storage with a retention policy that matches your organization's records schedule, not just whatever the default log rotation happens to be.

Data Residency and Network Isolation on GPU Cloud

Renting GPU instances rather than owning hardware means paying close attention to what's exposed by default. On Spheron, every instance gets a dedicated public IP with all ports open by default, which is a real convenience for standing up an inference server fast, but it also means you're responsible for locking it down before PHI ever touches the box. Spheron's networking documentation covers using ufw to restrict inbound traffic to just your inference API port and SSH tunneling for anything you don't want publicly reachable during development. Don't run a HIPAA-relevant workload on a fresh instance with default networking; treat the firewall configuration as step one, not an afterthought.

For multi-site health systems that can't centralize patient records across facilities, federated learning on GPU cloud keeps raw data local to each site and shares only model updates, which sidesteps the data residency question for training entirely rather than trying to solve it after the fact.

The 2026 HIPAA Security Rule Update: What's Changing and When

On December 27, 2024, HHS OCR issued a Notice of Proposed Rulemaking to overhaul the Security Rule for the first time since its last major update. As of mid-2026 it remains proposed, not finalized, so the exact timing can still move (Federal Register, 2025). The direction of travel matters for anyone architecting a self-hosted LLM stack right now, because building to the proposed bar today avoids a rebuild later.

The headline changes:

• Encryption becomes mandatory, not addressable. Today, covered entities can document why an "equivalent alternative measure" to encryption is reasonable and skip it. The proposal removes that option for ePHI at rest and in transit (medcurity.com).
• Multi-factor authentication becomes mandatory for all systems accessing ePHI, moving MFA from a recommended practice to an explicit requirement.
• A 72-hour recovery time objective for restoring relevant electronic information systems and data after a loss, plus a separate requirement for business associates to notify covered entities within 24 hours of activating a contingency plan (Bradley, 2025).
• Annual penetration testing to validate security controls, rather than ad hoc or as-needed testing.
• Annual BAA verification, meaning covered entities have to document that they actually checked their business associates' compliance posture, not just keep a signed BAA on file and assume it's still good.

Once finalized, the proposal sets a runway of 60 days to the effective date and 180 days to full compliance (medcurity.com). HHS estimated the changes would cost the healthcare industry roughly $9 billion in year one and $34 billion across the first five years (medcurity.com). Those are industry-wide totals, not per-organization estimates, but they signal that the compliance bar is moving up, not staying flat. Teams self-hosting LLMs today should build to "mandatory encryption everywhere plus MFA on every access path" now, since that's where the rule is headed regardless of the exact finalization date.

Cost of Getting This Wrong: Healthcare Breach Economics

Healthcare has led every industry in data breach costs for over a decade, and the exposure keeps growing. In 2025, roughly 138.5 million individuals were affected across 772 large healthcare data breaches reported to HHS OCR, following 289 million individuals across 741 large breaches in 2024 (HIPAA Journal, updated June 2026). Since 2009, OCR has logged 7,670 large breaches affecting more than one billion Americans in total, more than three times the US population, because many individuals have been caught in multiple incidents.

The single largest healthcare breach on record, Change Healthcare in 2024, affected 192.7 million individuals on its own (HIPAA Journal, updated June 2026). That breach involved a third-party processor sitting between a health system and its data, which is exactly the kind of exposure a self-hosted architecture with no unnecessary intermediary is designed to reduce, not the exact scenario, but the same structural risk: every additional party in your data path is another party whose failure becomes your breach.

None of this means self-hosting is a silver bullet. A self-hosted LLM stack with default networking, no audit logging, and shared API keys is just a different way to have a breach, not a way to avoid one. The economics argument for getting the architecture right isn't abstract: fewer parties touching PHI plus real Technical Safeguards is the actual risk reduction, not the marketing claim that self-hosting alone makes you compliant.

Deploying a Compliant Self-Hosted LLM Stack on Spheron

Putting the checklist into a concrete build looks like this. Provision a GPU instance sized to your model (an L40S for 70B-class models at INT4, an H100 if you need FP8 headroom or larger context), lock down networking with ufw before deploying anything, then bring up your inference server with logging enabled from the first request:

vllm serve Qwen/Qwen2.5-72B-Instruct-AWQ \
  --enable-log-requests \
  --enable-log-outputs \
  --max-model-len 32768

Put an authenticated, role-scoped API gateway in front of that endpoint rather than exposing it directly. Ship request logs to encrypted, access-controlled storage with a retention policy matched to your compliance program. Sign a BAA with your infrastructure provider covering the instances processing PHI. If your workload needs the encryption-in-use guarantee, reserved CC-mode instances close that gap using the same TEE architecture covered above.

Spheron gives you SSH root access to the instance, which matters here: you install your own audit logging agents and configure your own network isolation rather than working within a managed container runtime's constraints. Current pricing on the GPU tiers most relevant to this workload:

Pricing fluctuates based on GPU availability. The prices above are based on 09 Jul 2026 and may have changed. Check current GPU pricing for live rates.

None of this is turnkey HIPAA compliance in a box, and nobody selling you that is telling the truth. It's the infrastructure layer done right, which is the part that's actually your responsibility once you decide to self-host.

FAQ

Is any LLM actually HIPAA certified?

No. There is no HIPAA certification for software, and HIPAA governs deployments, not models. The model itself is never the compliant unit; the architecture around it is.

Do I need a BAA if I self-host an open-weight model?

You don't need one with a model vendor, since no model vendor receives PHI in a self-hosted setup. You do need one with whoever operates the infrastructure you're renting, unless you own the hardware outright.

What GPU do I need to self-host a HIPAA-relevant LLM?

Depends on the model. 70B-class models like Llama 3.3 70B or Qwen2.5 72B run at INT4 on a single L40S or H100. Use enterprise-certified GPUs like the L40S over consumer RTX cards for any production or customer-facing healthcare workload.

Does confidential computing satisfy HIPAA's encryption requirements on its own?

No. It closes the encryption-in-use gap in GPU VRAM specifically. You still need access controls, audit logging, and a BAA with your infrastructure provider if you don't own the hardware.

When does the updated HIPAA Security Rule take effect?

It's still a proposed rule as of mid-2026. Once finalized, the proposal sets 60 days to the effective date and 180 days to full compliance, but the finalization date itself hasn't been set.

Self-hosting an open-weight model gets PHI out of a model vendor's hands entirely; the infrastructure you rent to run it still needs a BAA, root access for your own audit controls, and a firewall you configure before day one.

Get started on Spheron →