Unlimited-OCR 2: Baidu’s Open OCR Model Turns Long PDFs into RAG-Ready Text

1. The spec card

Unlimited-OCR 2 keeps v1’s body: a DeepSeek-OCR-style vision encoder feeding a small mixture-of-experts decoder. What changed is how the decoder looks back at the pages. Here it is next to v1, from the cards, the HF API and config.json.

2. What’s new in v2: from long to unlimited parsing

v1’s idea was R-SWA, Reference Sliding Window Attention. It kept the KV cache constant while the model wrote, so it could transcribe dozens of pages under a 32K limit. But every new token still attended to every visual token of every page. At a hundred pages, that visual context becomes the bottleneck, and each decoding step gets slower as the document grows.

v2 swaps every decoder attention layer for SR-SWA, Sparse Reference Sliding Window Attention, and adds a new way to run long documents. The paper’s analogy is a person copying a book: eyes on the current page, with a quick glance back when a table runs over. It has three parts.

The three parts, in plain words

The payoff, per the paper: at the hundred-page scale, each decoding step is 7.0× faster than full attention and 5.6× faster than v1’s R-SWA (details in section 3). On MPDocBench, with documents of up to 30 pages, v2 beats v1 on every metric, most of all where content crosses a page break.

3. Benchmarks vs other models (self-reported)

Baidu’s leaderboard puts v2 against general-purpose vision-language models on two multi-page benchmarks. Dedicated OCR models such as DeepSeek-OCR 2 only show up in the paper’s single-page table, so I split this into two tables.

On MPDocBench the lead is real but moderate: 6.3 points over Qwen3.8-Max. On MarathonDocBench it’s 43 points over the next model, and the fine print explains a good part of that gap.

On single pages, v2 is basically v1: 94.02 vs 93.92 overall, with a slightly lower table score (89.60 vs 90.16). It beats DeepSeek-OCR 2 and dots.ocr, but two smaller models, TeleOCR and OvisOCR2, score higher. PaddleOCR-VL isn’t in the table; Baidu used PaddleOCR-VL-v1.6 as a pseudo-labeler instead. For one- or two-page documents, v2 isn’t the obvious pick. Its case is length.

4. Quickstart: Transformers, SGLang and a RAG ingestion step

The v2 card documents two paths: Transformers “for quick tests” and SGLang, “recommended for serving”. Every block below is copied from the card, except the last two, which are my own glue and say so. Both paths assume an NVIDIA GPU, and the SGLang setup also assumes Linux. If you’ve read that v2 ships vLLM and Docker Model Runner recipes, they aren’t in any revision of the card; the vLLM recipe is for v1.

pip install torch==2.13.0 torchvision pillow transformers==4.52.1 huggingface-hub==0.36.0 \
  tokenizers==0.21.4 addict==2.4.0 einops==0.8.2 easydict==1.13

import torch
from transformers import AutoModel, AutoTokenizer

model_name = "baidu/Unlimited-OCR-2"
tokenizer = AutoTokenizer.from_pretrained(model_name, trust_remote_code=True)
model = AutoModel.from_pretrained(model_name, trust_remote_code=True, use_safetensors=True,
                                  torch_dtype=torch.bfloat16).eval().cuda()

# Multi-page inference: all pages in one request, in page order
pages = [f"your_doc/page_{i}.jpg" for i in range(1, 8)]

text, num_tokens = model.infer_multi(tokenizer, image_files=pages,
                                     output_path="output/multi", save_results=True)

# Single-page inference: dynamic tiling (gundam mode) by default
text = model.infer(tokenizer, image_file="your_doc/page_1.jpg",
                   output_path="output/single", save_results=True)

uv venv --python 3.10
source .venv/bin/activate

uv pip install wheel/sglang-0.5.20.dev13+g75b9b06b02-py3-none-any.whl
uv pip install pymupdf==1.27.2.2
uv pip install fastokens==0.3.2

# Page-window scheduling for very long documents (the KV tail of the previous window is reused).
export SGLANG_UOCR_EANABLE_DOC_LIMIT=1          # 1 = enable, 0 = whole document in one window
export SGLANG_UOCR_DOC_GUNDAM_PAGE_WINDOW=10    # pages per window (gundam mode)
export SGLANG_UOCR_DOC_BASE_PAGE_WINDOW=10      # pages per window (base mode)
export SGLANG_UOCR_MAX_PAGE_TOKENS=20000        # per-page output cap; must be > 0
# Chunk cap for the SAM attention bias. The built-in default (512) allows a single
# ~462 MiB allocation and can OOM; 64 is equivalent with a lower peak.
export SGLANG_OPT_UOCR_SAM_REL_POS_BIAS_MB=64
# Pass large crop tensors through /dev/shm instead of pickling them into the
# tokenizer -> scheduler frame.
export SGLANG_UOCR_SHM_WRAP_MODEL_DATA=1
export SGLANG_UOCR_SHM_WRAP_MIN_MB=1
export SGLANG_VLM_CACHE_SIZE_MB=0
# Reduces fragmentation failures on the large contiguous allocations.
export PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True

python -m sglang.launch_server \
    --model-path baidu/Unlimited-OCR-2 \
    --served-model-name Unlimited-OCR-2 \
    --chat-template unlimited-ocr \
    --tokenizer-backend fastokens \
    --tokenizer-worker-num 8 \
    --mem-fraction-static 0.7 \
    --max-running-requests 64 \
    --max-prefill-tokens 2048 \
    --enable-custom-logit-processor \
    --enable-streaming-session \
    --allow-auto-truncate \
    --disable-radix-cache \
    --mm-preprocess-cache-size-mb 0 \
    --cuda-graph-backend-decode full --cuda-graph-max-bs-decode 64 \
    --cuda-graph-backend-prefill full \
    --cuda-graph-max-bs-prefill 16384 \
    --cuda-graph-bs-prefill 1024 2048 \
    --cuda-graph-config '{"prefill": {"backend": "full", "full_prefill_max_req": 64}}' \
    --host 0.0.0.0 \
    --port 10000

import base64
import json
import os
import tempfile

import fitz
import requests
from sglang.srt.sampling.custom_logit_processor import DeepseekOCRNoRepeatNGramLogitProcessor

server_url = "http://127.0.0.1:10000"
NGRAM_PROCESSOR = DeepseekOCRNoRepeatNGramLogitProcessor.to_str()

session = requests.Session()
session.trust_env = False


def pdf_to_images(pdf_path, dpi=300):
    doc = fitz.open(pdf_path)
    tmp_dir = tempfile.mkdtemp(prefix="pdf_ocr_")
    mat = fitz.Matrix(dpi / 72, dpi / 72)
    image_paths = []
    for i, page in enumerate(doc):
        image_path = os.path.join(tmp_dir, f"page_{i + 1:04d}.png")
        page.get_pixmap(matrix=mat).save(image_path)
        image_paths.append(image_path)
    doc.close()
    return image_paths


def encode_image(image_path):
    ext = os.path.splitext(image_path)[1].lower()
    mime = "image/jpeg" if ext in (".jpg", ".jpeg") else f"image/{ext.lstrip('.')}"
    with open(image_path, "rb") as f:
        data = base64.b64encode(f.read()).decode("utf-8")
    return {"type": "image_url", "image_url": {"url": f"data:{mime};base64,{data}"}}


def build_content(image_paths):
    prompt = "<PAGE><image>" * len(image_paths) + "Multi page parsing."
    return [encode_image(path) for path in image_paths] + [{"type": "text", "text": prompt}]


def generate(image_paths, crop_mode=True, tokens_per_page=8000):
    # A wider n-gram window for multi-page documents, as in our evaluation setup.
    window_size = 1024 if len(image_paths) > 1 else 128
    payload = {
        "model": "Unlimited-OCR-2",
        "messages": [{"role": "user", "content": build_content(image_paths)}],
        # Must be positive: it is the global output budget for the whole document.
        "max_tokens": tokens_per_page * len(image_paths),
        "temperature": 0,
        "skip_special_tokens": False,
        "images_config": {"crop_mode": crop_mode},  # True = gundam (tiles), False = base
        "custom_logit_processor": NGRAM_PROCESSOR,
        "custom_params": {
            "ngram_size": 35,
            "window_size": window_size,
            "page_token_id": 128827,   # <PAGE> marker id (= sparse_attention_config.page_token_id)
            "eos_token_id": 1,         # = eos_token_id in config.json
            "max_page_tokens": 20000,  # per-page output cap, <= 0 disables the page guard
        },
        "stream": True,
    }
    response = session.post(
        f"{server_url}/v1/chat/completions",
        headers={"Content-Type": "application/json"},
        data=json.dumps(payload),
        timeout=6000,
        stream=True,
    )
    response.raise_for_status()

    chunks = []
    for line in response.iter_lines(decode_unicode=True):
        if not line or not line.startswith("data:"):
            continue
        data = line[len("data:"):].strip()
        if data == "[DONE]":
            break
        event = json.loads(data)
        choices = event.get("choices") or []  # the final chunk carries usage only
        if not choices:
            continue
        delta = (choices[0].get("delta") or {}).get("content") or ""
        if delta:
            print(delta, end="", flush=True)
            chunks.append(delta)
    return "".join(chunks)


# Multi page: all pages of a document, parsed in one request
pages = [f"your_doc/page_{i}.jpg" for i in range(1, 8)]
generate(pages)

# Single page
generate(["your_doc/page_1.jpg"])

# PDF
generate(pdf_to_images("your_doc.pdf", dpi=300))

From PDF to chunks: a RAG ingestion step

Here’s how I’d wire it into ingestion. The card’s client already renders a PDF to 300 DPI page images and returns the raw output. The first block below saves it three ways: raw, as whole-document Markdown for chunking, and as one Markdown file per page for citations. The second is a minimal heading-aware chunker. I ran the chunker on a hand-written Markdown sample; neither block has seen real model output. If you’re choosing a strategy, my RAG chunking guide compares fixed, semantic and late chunking.

# Glue code (mine, not from the model card). Run it after the card's client above.
# Needs utils/process_unlimitedOCR2.py from github.com/baidu/Unlimited-OCR (stdlib only).
import sys
from pathlib import Path

sys.path.append("Unlimited-OCR/utils")
import process_unlimitedOCR2 as uocr

# My tweak: running headers, footers and page numbers repeat on every page
uocr.DROP_LABELS.update({"header", "footer", "page_number"})


def save_for_chunker(doc_id, raw, out_dir="ocr_out"):
    out = Path(out_dir)
    out.mkdir(parents=True, exist_ok=True)
    # 1) Raw layout blocks: one <PAGE> marker per input page, with bounding boxes
    (out / f"{doc_id}.raw.txt").write_text(raw, encoding="utf-8")
    # 2) Whole-document Markdown: cross-page tables and paragraphs merged
    (out / f"{doc_id}.md").write_text(uocr.convert(raw), encoding="utf-8")
    # 3) One Markdown file per page, for page-level citations (no cross-page merging)
    pages = raw.split("<PAGE>")[1:]
    for n, page in enumerate(pages, start=1):
        (out / f"{doc_id}.p{n:04d}.md").write_text(uocr.convert(page), encoding="utf-8")
    return len(pages)


raw = generate(pdf_to_images("your_doc.pdf", dpi=300))
print(save_for_chunker("your_doc", raw), "pages saved")

# Glue code (mine): split on Markdown headings, then pack blocks up to a size cap.
import re


def chunk_markdown(md, doc_id, max_chars=2000):
    chunks = []
    for section in re.split(r"(?m)^(?=#{1,6} )", md):
        section = section.strip()
        if not section:
            continue
        heading = section.splitlines()[0] if section.startswith("#") else ""
        buf = ""
        # convert() separates blocks with blank lines, so a table usually stays whole
        for block in section.split("\n\n"):
            if buf.strip() not in ("", heading) and len(buf) + len(block) > max_chars:
                chunks.append({"doc": doc_id, "heading": heading, "text": buf.strip()})
                buf = ""
            buf += block + "\n\n"
        if buf.strip() not in ("", heading):
            chunks.append({"doc": doc_id, "heading": heading, "text": buf.strip()})
    return chunks


md = open("ocr_out/your_doc.md", encoding="utf-8").read()
chunks = chunk_markdown(md, "your_doc")
print(len(chunks), "chunks ready for your embedding model")

5. Honest limits

Plan around these before you send your document archive through it.

Verdict: should it go into your RAG pipeline?

If your corpus is long PDFs (manuals, filings, reports, books, scanned archives) and you have an NVIDIA GPU, Unlimited-OCR 2 is the most interesting open OCR release this month. It reads a whole document in one request, keeps tables that cross page breaks together and ships under MIT. Those cross-page failures are exactly what wrecks chunking downstream.

It isn’t a universal upgrade. On single pages it’s v1 with a new attention layer, smaller models score higher, and every number is Baidu’s. Treat the benchmarks as a reason to test, not a reason to switch: run 20 of your own documents through it, diff the Markdown against your current parser, and look hardest at tables and headings.

Who should try it