Deploy Baidu Unlimited-OCR on GPU Cloud: Setup and VRAM Guide (2026)

Baidu open-sourced Unlimited-OCR on GitHub and Hugging Face on June 22, 2026, under the MIT license, and the pickup was real: over 1 million Hugging Face downloads in the first two weeks, top-3 trending on the platform, and 10,000+ GitHub stars within five days on its way to 23.8k stars and 2.4k forks today (Hugging Face model card, GitHub). The model card shows 3,048,228 downloads in the past month alone.

What makes it worth a dedicated deploy guide, rather than a footnote in an OCR roundup, is architectural. Unlimited-OCR continues training from a DeepSeek-OCR checkpoint but swaps the decoder's attention mechanism for something called Reference Sliding Window Attention (R-SWA), and that one change is why it can OCR dozens of pages of a document in a single forward pass instead of looping page by page (arXiv:2606.23050). If you've already deployed DeepSeek-OCR on GPU cloud, this is the same lineage with a fundamentally different serving profile.

This guide covers what one-shot long-horizon parsing actually buys you, where the model sits in the current open-source OCR field, VRAM and throughput sizing on real GPUs, and step-by-step vLLM/SGLang setup for production document pipelines.

What One-Shot Long-Horizon Parsing Actually Means vs Page-by-Page OCR

Most VLM-based OCR models, DeepSeek-OCR included, process a document one page at a time. Each page is a separate forward pass: encode the image, decode the text, discard the KV cache, move to the next page. That's fine for throughput on independent pages, but it means the model never sees page-to-page context, and a batch job over a 200-page contract still runs 200 separate inference calls with 200 separate cold KV caches.

Unlimited-OCR's GitHub README frames the alternative directly in its tagline: "Unlimited OCR Works: Welcome the Era of One-shot Long-horizon Parsing." The mechanism behind that claim is R-SWA, which the paper describes as replacing every attention layer in the decoder so that computation cost stays flat and the KV cache stays constant throughout decoding, rather than growing with every generated token (arXiv:2606.23050). In practice, the decoder attends to two things: a fixed set of reference tokens (the visual content and the prompt) and a short sliding window of recently generated output tokens. It never has to hold the full growing sequence in memory the way a standard autoregressive decoder does.

The paper's authors motivate this by pointing out that "humans exhibit no such decline in efficiency during long-horizon copying tasks," while a standard transformer decoder's throughput degrades as its KV cache balloons over a long document. R-SWA is an attempt to give a model the same flat performance curve a human transcriber has when copying page after page. Combined with a 32,768-token context window, that's enough to hold roughly 40 pages of a document in a single forward pass, which the paper's own evaluation backs up: 0.0572 edit distance and 99.89% repetition robustness at 20 pages, degrading to a still-usable 0.1069 edit distance and 96.90% robustness past 40 pages (digitalapplied.com). That 40-page figure is worth flagging explicitly: some social posts claimed 100-page single-pass parsing, but the 32K context budget doesn't support that at current accuracy.

The DeepEncoder feeding into this decoder is architecturally inherited from DeepSeek-OCR: a SAM-ViT plus CLIP-ViT combination that compresses a 1024x1024 page image down to 256 visual tokens, a 16x reduction (digitalapplied.com). That figure comes from a single secondary source rather than the official repo, so treat it as directionally correct rather than a guaranteed spec if you're building compression math into a pipeline.

Best Open Source OCR Model 2026: Where Unlimited-OCR Stands Against DeepSeek-OCR and PaddleOCR-VL

If you're deciding what to run in 2026, Unlimited-OCR is the strongest single-document accuracy option among the models we've deployed and benchmarked, but "best" depends on what your corpus looks like. On OmniDocBench v1.5, it posts a 93.23% overall score against DeepSeek-OCR's 87.01% baseline, a 6.22-point gain, with a text edit distance of 0.038 versus 0.073 (roughly halving the error rate), formula recognition at 92.61% (+9.24 points), table TEDS at 90.93% (+5.96 points), and decode throughput of 5,580 tokens/sec, a 12.7% improvement over the baseline (digitalapplied.com). A separate summary also cites a 93.92% score on OmniDocBench v1.6, though that figure is single-sourced so far and worth checking against the official leaderboard before you treat it as a settled headline number (baike.baidu.com).

This isn't a substitute for a full field survey. For the wider comparison across five-plus models, see the broader open-source OCR VLM roundup; the question this section answers is narrower: does Unlimited-OCR's architecture actually earn it a spot next to the model it's descended from and the other multilingual leader in the field.

Pick Unlimited-OCR when your workload is genuinely multi-page: contracts, filings, long reports where a single call replacing a 20-request loop is worth the newer, less battle-tested deployment path. Pick DeepSeek-OCR when you're OCRing independent single pages at high volume and don't need cross-page context. Pick PaddleOCR-VL-1.6 when language coverage across 100+ scripts matters more than single-pass document length. All three are MIT- or Apache-licensed and run comfortably on a single mid-tier GPU.

VRAM and Throughput on a Single GPU

Unlimited-OCR's Hugging Face model card lists it as a 3B-parameter image-text-to-text model at BF16. A separate source rounds it to 3.3B and pegs the on-disk weight size at roughly 6.7 GB in bfloat16 (Hugging Face, codersera.com). Treat 3B as the primary figure since it's what the official model card states; 3.3B is a rounding variant from a secondary source, not a conflicting spec.

VRAM requirements scale from there:

Source: digitalapplied.com.

For production self-hosting where you're processing real document volume, not just running a single request, size up past the 12 GB floor. A 3B model at BF16 leaves plenty of headroom for KV cache on a 24 GB card, and R-SWA's bounded cache means that headroom doesn't erode as documents get longer the way it would with a standard decoder. On Spheron, that maps cleanly to two tiers:

Pricing fluctuates based on GPU availability. The prices above are based on 16 Aug 2026 and may have changed. Check current GPU pricing → for live rates.

At this parameter count, an RTX 4090 is the practical sweet spot: full BF16 precision, room for the 32K context window's KV cache, and $0.53/hr on-demand. Move up to an L40S only if you're running several concurrent document streams on one instance rather than one document at a time.

Deploy Baidu Unlimited-OCR: Step-by-Step Self-Hosting Setup

The GitHub repo officially supports three serving paths: Hugging Face Transformers for quick testing, vLLM for general production serving, and SGLang for an OpenAI-compatible endpoint with a custom logit processor tuned for this model's repetition behavior (GitHub README).

1. Provision a GPU. Spin up an RTX 4090 or larger instance on app.spheron.ai, or follow the instance provisioning flow in the Spheron docs. SSH in and confirm the driver with nvidia-smi before pulling any container.

2. Pull the right image for your CUDA setup. Baidu ships two vLLM Docker variants:

# CUDA 13.0 (default)
docker pull vllm/vllm-openai:unlimited-ocr

# CUDA 12.9, for Hopper-class GPUs
docker pull vllm/vllm-openai:unlimited-ocr-cu129

3. Launch the server at the model's native context length. Unlimited-OCR was trained at a 32,768-token window; going lower truncates its long-horizon capability, and there's no benefit to going higher since the model was never trained past that point.

docker run --runtime nvidia --gpus all \
  -v ~/.cache/huggingface:/root/.cache/huggingface \
  -p 8000:8000 \
  vllm/vllm-openai:unlimited-ocr \
  --model baidu/Unlimited-OCR \
  --max-model-len 32768 \
  --served-model-name unlimited-ocr

4. Send a test request. Submit a rendered multi-page document image (or a sequence of page images in one call) and ask for structured output:

from openai import OpenAI

client = OpenAI(base_url="http://localhost:8000/v1", api_key="none")
response = client.chat.completions.create(
    model="unlimited-ocr",
    messages=[{
        "role": "user",
        "content": [
            {"type": "image_url", "image_url": {"url": "data:image/jpeg;base64,<page_1_b64>"}},
            {"type": "image_url", "image_url": {"url": "data:image/jpeg;base64,<page_2_b64>"}},
            {"type": "text", "text": "Extract all text from this document, preserving tables as Markdown, in page order."}
        ]
    }],
    max_tokens=8192,
    temperature=0
)
print(response.choices[0].message.content)

5. Cap document length at ingestion. Since accuracy holds best up to 20 pages and is still usable to roughly 40, split anything longer into chunks at that boundary rather than pushing a 100-page PDF through in one call. This is a deliberate design trade-off in exchange for the bounded KV cache, not a bug to work around.

Serving It With vLLM and SGLang for Production Document Pipelines

vLLM is the simpler path and the right default for general document workloads. SGLang is worth the extra setup step when you're running high-concurrency batch OCR and want RadixAttention's prefix caching on top of R-SWA's bounded decode cache, particularly if your document set shares common headers, footers, or template structure.

SGLang requires a dev-wheel install rather than the stable release, since the custom logit processor for Unlimited-OCR isn't in a tagged version yet. The official README installs a specific dev-build wheel plus two pinned dependency versions, not a generic PyPI package:

uv venv --python 3.12
source .venv/bin/activate

uv pip install wheel/sglang-*.whl kernels==0.11.7 pymupdf==1.27.2.2

python -m sglang.launch_server \
  --model baidu/Unlimited-OCR \
  --served-model-name unlimited-ocr \
  --attention-backend fa3 \
  --context-length 32768 \
  --enable-custom-logit-processor \
  --host 0.0.0.0 \
  --port 10000

Grab the exact wheel filename from the releases section of the GitHub repo, since the dev-build hash in the filename changes between commits and a stale one will fail to install.

The --attention-backend fa3 flag matters here specifically because R-SWA's constant-cache attention pattern benefits from FlashAttention-3's kernel support; the --enable-custom-logit-processor flag loads the repetition-suppression logic Baidu built for this model's decoding behavior (codersera.com).

For multi-GPU tensor-parallel setups or FP8 quantization flags beyond what a single 3B model needs, the vLLM production deployment guide covers that configuration in more depth, though at 3B parameters most single-GPU deployments won't need to reach for it.

Cost per 10,000 Pages: Self-Hosted vs Cloud OCR APIs

At roughly 40 pages per single-pass call, an RTX 4090 running Unlimited-OCR at $0.53/hr can push through a meaningful volume of multi-page documents per hour, and unlike a page-by-page pipeline, you're not paying for 40 separate cold-start inference calls per document. AWS Textract charges $0.0015 per page for basic text extraction, which is $1.50 per 1,000 pages and $15 for 10,000 pages, before any table-extraction add-on fees (AWS Textract pricing).

The precise self-hosted cost per page depends on your document length distribution and batch size, since Unlimited-OCR's one-shot design changes the unit of work from "one page" to "one document." As a rough anchor, the DeepSeek-OCR cost comparison found self-hosting breaks even against Textract somewhere around 50,000-100,000 pages per month for a page-by-page model; Unlimited-OCR's ability to collapse a multi-page document into one call should push that break-even point lower for corpora that are genuinely multi-page rather than a pile of independent single pages.

Pricing fluctuates based on GPU availability. The prices above are based on 16 Aug 2026 and may have changed. Check current GPU pricing → for live rates.

Below 50,000 pages a month, the fixed cost of running a GPU instance likely outweighs Textract's pay-per-page simplicity unless output quality is the deciding factor. Above it, and especially for long multi-page filings and reports, the math shifts toward self-hosting fast.

Plugging Unlimited-OCR into a RAG or Document Pipeline

Once Unlimited-OCR has extracted a document into structured Markdown or JSON, the rest of the pipeline is the same as any text-based RAG system: chunk, embed, index, retrieve.

Multi-page PDF -> Unlimited-OCR (single forward pass) -> Markdown/JSON
                                                                |
                                                     Chunker (section-aware)
                                                                |
                                                    Embedding model (BGE-M3, etc.)
                                                                |
                                                       Qdrant / Milvus

Because Unlimited-OCR returns the whole document's structure in one response instead of 40 separate page fragments, chunking gets easier: you can split on the model's own section boundaries instead of stitching page-level outputs back together and guessing where a paragraph or table got cut across a page break. For the vector store setup, the self-hosted Qdrant and Milvus guide covers indexing OCR output on the same GPU cluster as your inference nodes, and the agentic RAG infrastructure guide covers the full stack from embedding through LLM inference if you're building out retrieval end to end.

For document sets that mix native, text-based PDFs with scanned pages, the Docling, Marker, and MinerU guide covers routing native pages through a layout parser and only sending scanned pages to a VLM OCR model like this one, which saves GPU cycles on documents that don't need vision-based extraction at all. And if your corpus leans on visual layout more than linear text, dense financial tables, forms, annotated diagrams, it's worth testing ColPali's page-image embedding approach as an alternative to OCR entirely.

Unlimited-OCR's bounded KV cache makes it the right pick when your documents are genuinely multi-page, not a stack of independent scans. Spheron gives you on-demand access to the RTX 4090s and L40S instances this model runs on, with no minimum commitments.

RTX 4090 on Spheron → | Get started on Spheron →