Apple's LensVLM-9B reads documents as compressed images, expanding only relevant pages

Apple has published LensVLM-9B, a 9-billion-parameter vision language model that reads documents as compressed page images and expands only the pages it needs. The model card is on Hugging Face alongside a code repository and a paper posted to arXiv on 7 May 2026.

How it works

Vision language models can process text rendered as images instead of tokenizing it into long sequences, and because the encoder maps a fixed-size image to a fixed number of visual tokens, rendering resolution acts as a fine-grained compression knob. The catch: accuracy falls apart as compression grows — characters shrink below the encoder's effective resolution and become indistinguishable.

LensVLM is a post-training recipe and inference framework that gets around that limit. The model scans a document rendered in compressed form (5x, 10x and 15x presets), then calls learned tools such as read_page to restore only the relevant pages to their uncompressed form, turn by turn. Apple trained it in three stages: hard-example filtering, synthetic tool-use traces for supervised fine-tuning, and reinforcement learning (RL). LensVLM-9B is built on Qwen3.5-9B-Base.

What the numbers show

According to the paper, LensVLM keeps accuracy comparable to the full-text upper bound at 4.3x effective compression and outperforms retrieval-based, text-compression and visual-compression baselines up to 10.1x effective compression across seven text QA benchmarks. It also transfers to multimodal document and code understanding, and the gap over the baselines widens as compression increases.

The analysis suggests why: training makes visual compression robust to rendering choices, and at higher compression ratios the model leans on expanded content rather than trying to read shrinking pixels. The authors also note that text expansion suits rendered text, while high-resolution image expansion fits documents whose layout carries task-relevant information.

Demo and release

In the bundled demo the model scans 15 compressed page images, flags page 10 as relevant, reads it with the read_page tool and links Shirley Temple, Corliss Archer and Chief of Protocol in two turns. The repo also ships data preparation for HotpotQA, NQ and Musique plus evaluation code.

The weights are released under the Apple Machine Learning Research Model License and the code under the Apple Sample Code License. Hugging Face lists 9B parameters in BF16, 233 downloads in the past month and two community quantizations.