Self-Host Legal AI: Open Source Harvey/Legora Alternative

A law firm's IT director doesn't usually get quoted saying a free GitHub repo "changes the negotiation" with a $200,000-a-year vendor. But that's what James Harrison, former IT Director at Leigh Day, told Legal Technology about Mike OSS, the open source tool that replicates Harvey and Legora's core workflows for $0 (legaltechnology.com). "Mike doesn't kill Harvey or Legora, but it absolutely changes the negotiation," Harrison said.

Here's the part that gets lost in the coverage: Mike OSS, out of the box, still ships your documents to Anthropic or Google's API. That's the same third-party exposure that just cost a defendant attorney-client privilege in US v. Heppner, and the same API-versus-self-hosted tradeoff we work through in general terms in our Claude Opus 4.8 API pricing vs self-hosted LLMs comparison. Running Mike OSS for free doesn't solve the confidentiality problem law firms actually have. Getting a firm's documents off a vendor's servers entirely requires one more step: pointing the backend at a self-hosted, open-weight model running on infrastructure you control, whether that's your own hardware or GPUs rented by the hour on a GPU cloud.

This post covers why that step matters legally, what Mike OSS actually does, and exactly how to patch it to speak to a self-hosted, OpenAI-compatible endpoint and size the hardware to run against a model you host yourself.

Why Law Firms Are Rethinking Third-Party Legal AI

Two things happened in early-to-mid 2026 that changed how firms think about AI vendor risk: a federal judge ruled that chatting with Claude doesn't create privilege, and adoption data showed lawyers were using AI faster than firms could write policy around it.

What US v. Heppner Actually Held

On February 17, 2026, Judge Jed S. Rakoff of the Southern District of New York issued a written ruling in United States v. Heppner holding that a defendant's written exchanges with Claude were protected by neither attorney-client privilege nor the work-product doctrine (Harvard Law Review). A Forbes summary of the decision put it bluntly: "Claude isn't an attorney, the AI-generated documents weren't confidential and Heppner didn't use Claude to obtain legal advice" (Forbes).

The court gave three independent grounds, any one of which would have been enough on its own:

1. No attorney-client communication. Claude is not an attorney, so exchanges with it were never privileged communications between a client and counsel in the first place (Inside Privacy).
2. No confidentiality. Anthropic's terms of service reserve rights to log and use prompts and outputs, which the court found defeats any reasonable expectation of confidentiality with a public platform (Inside Privacy).
3. Not for legal advice. Heppner used the tool on his own initiative, not at the direction of counsel seeking legal advice, so the exchanges didn't meet the legal-advice requirement either (Inside Privacy).

The important nuance is that Heppner was a pro se defendant using a consumer chatbot on his own, not a lawyer directing AI use under a firm's vendor contract. Commentary since the ruling has flagged that distinction: analysis from Venable notes that when tools are "deployed by or at the direction of counsel, and subject to enforceable confidentiality commitments, the structural posture differs materially from the publicly available platform at issue in Heppner" (Venable). Contractual bans on training on customer data, defined retention limits, and audit controls could shift that analysis for enterprise or self-hosted deployments. The court didn't rule on that scenario. It's the scenario every firm now has to actively build toward instead of assuming.

The Adoption Gap: Firms Are Using AI Faster Than They're Governing It

Heppner landed on top of an adoption curve that was already outrunning firm policy. Nearly 70% of legal professionals now use general-purpose AI tools like ChatGPT for work, up from 31% in 2025 and 27% in 2024, yet only 46% of firms have formally implemented general-purpose AI tools and just 34% have adopted legal-specific AI platforms, according to American Bar Association reporting (americanbar.org). Litify's 2025 State of AI in Legal report found 66% of legal professionals use ChatGPT, 42% use Microsoft Copilot, and 24% use Google Gemini for work, while 53% say their firm has no AI policy at all (Forbes).

Data security is the top named barrier to firm-wide adoption, cited by 46% of legal professionals (americanbar.org). That's the gap Mike OSS walked into: individual associates were already pasting contracts into ChatGPT with no firm oversight. A free, self-hostable tool is a way to get that usage under a governed roof, but only if the roof actually keeps documents in-house. If it just moves the same unmanaged API calls into a nicer UI, it hasn't closed the gap at all.

Mike OSS: The Open Source Harvey and Legora Alternative

Mike OSS is what happens when someone who worked inside a firm decides the workflow doesn't need a nine-figure valuation behind it. Will Chen, a former Latham & Watkins associate, built it in about two weeks and published the code on GitHub in May 2026 (Legal Cheek). The name is a nod to Mike Ross, the unlicensed associate from Suits, and by the end of its first week live the repo had crossed 2,200 GitHub stars and 600 forks (Brightline Labs).

What Mike OSS Does (Vault, Tabular Review, Workflows)

Mike replicates the four workflows Harvey and Legora built their pitch decks around:

• Chat assistant for drafting and reviewing documents in natural language.
• Encrypted vault for storing matter files.
• Tabular review for running the same prompt across a batch of documents at once, the mass-analysis workflow firms use for due diligence and discovery review.
• Saved workflows that bundle a prompt and template together so a task doesn't have to be rebuilt from scratch every time.

That's the real feature parity, not a stripped-down demo. It's why Harrison's "changes the negotiation" line landed the way it did: a firm evaluating a renewal can point at a working, free alternative and ask what exactly the enterprise contract is paying for.

The Catch: Mike OSS Ships With Claude and Gemini API Keys by Default, Not Local Inference

Here's the part that doesn't make it into the launch coverage. Mike OSS has no built-in inference. It requires an API key from Anthropic Claude or Google Gemini that the firm creates and manages itself (mikeoss.com). Every document you feed the vault or run through tabular review gets sent to whichever of those two APIs you've configured.

Chen has been candid that self-hosting is possible: "the site online at mikeoss.com is meant to be a demo site... I have put the code out in open source (Github link in the website) so [you] can download it and run it locally in your computer," he said, adding, "Your files and data never have to leave your computer" (mikeoss.com). That's true of the application layer. It's not true of the model layer by default. Cloning the repo and running it on your laptop still routes every prompt through Claude or Gemini's API unless you change the inference backend. Free software plus a third-party model API is a real cost win over a $200,000-a-year Harvey contract (pasqualepillitteri.it), but it's the same privilege exposure Heppner just tested in court. The documents still leave the firm's infrastructure.

What It Takes to Self-Host Mike OSS on Your Own GPUs

Getting Mike OSS fully in-house means two changes: patching the backend to talk to a model you host, and sizing that model and its GPU correctly for the documents you're actually running through it.

Patching Mike OSS to Speak to a Self-Hosted Model

Mike's default OpenAI provider calls the newer Responses API (/v1/responses). Neither llama.cpp nor Ollama implements that endpoint out of the box; both speak the older /v1/chat/completions format instead (ozzy-tech.medium.com). vLLM's OpenAI-compatible server is built around the same /v1/chat/completions format, so it needs the identical fix. Point Mike's stock configuration at a self-hosted vLLM server and it will fail silently, because it's calling an endpoint your server doesn't have.

The fix modifies backend/src/lib/llm/openai.ts to switch from /v1/responses to /v1/chat/completions, add an OPENAI_BASE_URL environment variable so the client can be pointed anywhere, and add an OPENAI_MODEL_ID variable to force model selection instead of relying on Mike's hardcoded allowlist (ozzy-tech.medium.com). Once that's patched, the same code path works whether your inference server is a local llama.cpp box on the LAN or a vLLM instance running on rented GPUs.

A working self-hosted stack runs the rest of Mike as Docker containers, a Next.js frontend, an Express backend, local Supabase for auth and Postgres, and MinIO for S3-compatible object storage, with the inference server running outside Docker as a separate GPU process (ozzy-tech.medium.com). If you'd rather stand up a self-hosted OpenAI-compatible endpoint from scratch instead of patching Mike's provider by hand, our vLLM OpenAI-compatible server guide walks through exactly that setup, including the --host 0.0.0.0 and API key configuration Mike's patched client needs to authenticate against.

Model and VRAM Sizing for Contract-Length Documents

Document length drives context window, and context window drives which model and GPU you need. A 500-page contract needs roughly a 375K-token context window to analyze without splitting it; a 128K window only covers about 200 pages before you lose cross-section context between chunks (tokenmix.ai).

That rules out a lot of the smaller open-weight catalog for full-contract review. Qwen3-Max ships with a 262K-token context window natively (OpenRouter), and Qwen 3.6 Plus goes further, reaching a 1M-token window by combining linear attention with sparse mixture-of-experts to cut the compute cost of processing long sequences (digitalapplied.com). Llama 4 Scout advertises a 10M-token window, about 15,000 pages, but independent testing found effective recall degrades notably beyond roughly 1-2M tokens for synthesis tasks, reasoning across the whole context at once, even though it stays reliable for retrieval-style lookups across its full range (digitalapplied.com). For full-document contract synthesis rather than fact lookup, Qwen3-class context and accuracy is the safer default; treat Scout's extreme end of the range as a retrieval tool, not a substitute for reading the whole contract.

Model-to-GPU sizing follows the same tiers as any other self-hosted deployment:

For the full tier-by-tier VRAM breakdown with quantization tradeoffs across the open-weight catalog, see our best open-source LLMs to self-host guide.

If your legal AI stack also needs to ingest scanned filings or PDFs before any of this, our self-hosted document intelligence guide covers Docling, Marker, and MinerU for turning contract PDFs into clean text before they hit the model.

Cost Comparison: Self-Hosted Legal AI vs Harvey and Legora Per-Seat Pricing

Harvey doesn't publish pricing, but buyer reports put its mid-market tier around $1,200-$1,500 per seat per month for firms with 50-200 attorneys, a 20-seat, 12-month minimum commitment that works out to roughly $288,000 a year at the base rate, Am Law 100-scale deals nearer $100-$200 per seat per month with volume discounts, and 10-25% renewal uplifts on top (eesel.ai). Separate buyer reporting puts Harvey's unlimited-usage tier as high as $1,200-$2,000+ per seat per month (vaquill.ai), and separate market data puts the median Harvey contract at $175,000 a year, with enterprise-scale deals typically requiring a 25-50+ seat minimum (costbench.com). Legora's list price starts at $3,000 per user per year with a 10-seat minimum, meaning a $30,000 minimum contract before add-ons, and enterprise deployments at large firms run higher; Legora moved its Agent Pro tier to consumption-based pricing in June 2026 (vaquill.ai). Mike OSS's own comparison points out the software itself costs $0 against that backdrop, versus vendor contracts commonly quoted over $200,000 a year for a mid-to-large firm's standard tier (pasqualepillitteri.it).

Self-hosting Mike OSS against a third-party API swaps the per-seat license for a token bill, which is real savings but keeps the privilege exposure Heppner just tested. Self-hosting against your own model swaps the per-seat license for GPU-hours instead, on demand as of 18 Jul 2026:

Pricing fluctuates based on GPU availability. The prices above are based on 18 Jul 2026 and may have changed. Check current GPU pricing → for live rates.

A firm running a single H100 continuously for a month lands around $1,825, well under a single Harvey seat at the reported $1,200-$2,000/month range, before you even count the other 19+ seats a minimum contract requires. Run it only during business hours and the gap widens further. The tradeoff is that your team now owns the DevOps the vendor used to carry: patching, uptime, and the access controls covered next.

Deploying a Self-Hosted Legal AI Stack on Spheron

A concrete build looks like this: provision a GPU sized to your model, patch Mike's backend as described above, and bring up an inference server that speaks the endpoint Mike now expects.

1. Provision the GPU. Spin up an H100 or L40S instance; Spheron's quick-start guide covers provisioning in under three minutes.
2. Lock down networking before anything else. Spheron instances get a public IP with all ports open by default. Restrict inbound traffic to your inference port with ufw before you deploy anything that will touch client documents.
3. Bring up the inference server with an OpenAI-compatible endpoint:

vllm serve Qwen/Qwen3-32B-AWQ \
  --host 0.0.0.0 \
  --port 8000 \
  --max-model-len 131072 \
  --api-key your-secret-key

4. Patch and point Mike OSS at that endpoint by setting OPENAI_BASE_URL=http://your-instance-ip:8000/v1 and OPENAI_MODEL_ID=Qwen/Qwen3-32B-AWQ in the backend's environment, per the patch described above.
5. Bring up the rest of the Docker stack, Next.js frontend, Express backend, local Supabase, and MinIO, exactly as the self-hosted Mike deployment guide describes (ozzy-tech.medium.com).

At that point, no prompt, no document, and no client name has touched Anthropic's or Google's infrastructure. Every party that ever sees a privileged document is your own firm and the GPU provider whose hardware you're renting, the same reduced chain of custody firms are building toward for other regulated industries; our HIPAA-compliant GPU cloud guide covers the same BAA-chain logic applied to healthcare data instead of privileged legal documents.

FAQ

What is Mike OSS?

Mike OSS is a free, AGPL v3-licensed legal AI tool built by Will Chen, a former Latham & Watkins associate. It replicates Harvey and Legora's core workflows, a chat assistant, an encrypted document vault, tabular review for mass contract analysis, and saved workflows, and it costs nothing to run beyond whatever LLM API you point it at.

Does Mike OSS keep documents private by default?

No. Out of the box, Mike OSS sends every prompt and document excerpt to whichever API you configure, Anthropic's Claude or Google's Gemini. Your firm's documents leave your infrastructure and land on a third-party vendor's servers, the same exposure that undermined privilege in US v. Heppner. Keeping documents in-house requires patching the backend to call a self-hosted model instead.

Why does US v. Heppner matter for law firms using AI?

In a February 17, 2026 ruling, Judge Jed Rakoff held that a defendant's exchanges with Claude were protected by neither attorney-client privilege nor the work-product doctrine, on three grounds: Claude isn't an attorney, Anthropic's terms reserve rights to log and use prompts, and the defendant used the tool on his own initiative rather than at counsel's direction. Commentary since has noted that attorney-directed use under an enforceable confidentiality framework, including fully self-hosted infrastructure, may be analyzed differently.

What GPU do I need to self-host a legal AI model?

It depends on document length and model size. A 500-page contract needs roughly a 375K-token context window to analyze in one pass. Qwen3-Max covers that natively at 262K tokens and Qwen 3.6 Plus stretches to 1M using a hybrid linear-attention and mixture-of-experts architecture. Run a 70B-class model like Llama 3.3 70B or Qwen2.5 72B at INT4 on a single H100, or a smaller model in the 8B-32B range on an RTX 4090 or L40S for drafting and shorter-document review.

How much does self-hosting save compared to Harvey or Legora?

Harvey's mid-market tier commonly runs $1,200-$1,500 per seat per month, with unlimited-usage contracts reported as high as $2,000+ per seat per month, against a 20-seat minimum (about $288,000/year at the floor rate). Am Law 100-scale deals run $100-$200 per seat per month with volume discounts, and separate market data puts the median annual Harvey contract at $175,000. Legora starts at $3,000 per user per year with a 10-seat minimum. A self-hosted stack on rented GPUs replaces that per-seat fee with GPU-hours, often a small fraction of the licensing cost for firms with steady usage, though you take on the DevOps burden the vendor used to carry.

Mike OSS proves the Harvey and Legora workflow doesn't need a nine-figure valuation behind it, but the version that actually protects privilege is the one where your documents never leave your own infrastructure.

Check live GPU pricing →