Baidu researchers built an OCR model that processes dozens of document pages in a single pass. A 'soft forgetting' mechanism keeps the KV cache constant, dramatically improving speed and memory efficiency.

Introduction: The OCR Bottleneck

Baidu researchers have built an OCR model that handles dozens of document pages in a single inference pass, keeping memory use and speed constant regardless of text length. A redesigned attention mechanism makes this possible.

No current OCR model handles more than about ten pages in a single pass, the Baidu researchers write in their technical report. The bottleneck is the KV cache, a buffer where a language model stores all previously processed tokens during generation. As more tokens are processed, this buffer grows, driving up memory use and slowing down generation. Systems typically work around this by processing one page at a time, resetting the cache after every step — which adds latency and complexity.

A Fixed Window Caps Memory Use

The solution works through what the team calls Reference Sliding Window Attention (R-SWA). Each generated token still sees all reference tokens — the visual image tokens and the prompt. But when it comes to previously generated output, it only looks back at the last 128 tokens. This keeps the KV cache constant throughout the entire process instead of growing linearly with output length.

The KV cache operates as a queue where each new token pushes out the oldest one. With standard multi-head attention, memory use grows without bound as the token count rises. R-SWA caps it at the fixed sum of prefix length and window size — a fundamental breakthrough for long-document processing.

Built on DeepSeek OCR

Unlimited OCR builds on the DeepSeek OCR model as its foundation. The Baidu architecture consists of a DeepEncoder (3 billion parameters) with a Mixture-of-Experts decoder that uses 500M active parameters per token. The model achieves an impressive balance between computational efficiency and recognition accuracy. The model is open-source and available on GitHub and Hugging Face, with a quickstart demo for developers to experiment with.

Better Scores Despite Limited Attention

Despite its constraints, Unlimited OCR delivers competitive results. It achieves 93% on OmniDocBench v1.5 and 93.92% on v1.6. Its speed: 5,580 tokens per second — 12.7% faster than DeepSeek OCR. The kernel latency stays flat at around 9 microseconds regardless of decoding step count, while standard attention climbs past 16 microseconds.

Not Truly Unlimited Yet

As the name suggests, the model isn't truly "unlimited" yet. Baidu researchers plan to extend the model's context to 128K tokens in a future version and pool prefill tokens for even better memory handling. The current version can handle roughly 30 to 50 pages depending on content density — a massive improvement over the 10-page ceiling of existing alternatives.

Conclusion: Practical Applications and Impact

For users in Georgia and around the world, Unlimited OCR could mean faster digitization of lengthy legal and medical documents, more efficient archiving of historical documents published in the Georgian language (Mkhedruli script), and a significant improvement in business document processing across industries. The R-SWA technique demonstrated here also has broader implications for other sequence-processing tasks in AI — from language translation to code generation.