DeepSeek-OCR

Counting positions

Imagine a page whose transcription takes 1,000 text tokens. Instead, render the page as an image, encode the image into 100 vision tokens and give those to a decoder. The input is now ten times shorter. These numbers are made up for illustration.

A vision token is a learned vector that summarizes features of part of an image. It is not a word, a character or a byte. One vision token can carry evidence about several characters, but decoding it can go wrong. The ratio of 1,000 to 100 describes sequence length only. It says nothing about what was lost, how many bytes each vector takes, or how much work the encoder did.

Sequence length matters because the cost of attention depends on the number of positions. A shorter input can make the downstream work cheaper, but building it takes work of its own, and writing out a 1,000-token transcription still takes 1,000 generation steps.

Compressing before the expensive stage

The encoder, called DeepEncoder, has about 380 million parameters in three parts. A first stage based on SAM, an image-segmentation model, looks at small neighborhoods of the image using window attention. A convolutional bridge then shrinks the number of positions. A second stage based on CLIP, an image–text model, applies global attention, in which every position can attend to every other. The decoder is a 3-billion-parameter mixture-of-experts language model with about 570 million parameters active per token. The parts existed before; the contribution is how they are arranged. [2]

For a 1,024 × 1,024-pixel image cut into 16-pixel patches, the first stage sees 64 × 64 = 4,096 positions. The bridge reduces that sixteenfold, to 256, before the global stage. The cheap local stage therefore looks at a detailed grid, and the expensive global stage receives a much shorter sequence. [2]

Reading a newspaper works in a similar order: look closely at nearby marks to make out the letters, then put them together into the page’s overall structure. The comparison describes the order of the stages, not how the network represents what it reads.

Fewer positions, less detail

Make the patches larger in the figure. Covering the same image with bigger patches gives fewer positions. In this grid, going from 48 positions to 12 shrinks the number of pairs that full attention compares from 2,304 to 144, sixteen times fewer, because the number of pairs is the square of the number of positions. This counts attention pairs only, not the whole model’s computation.

Small writing shows the cost. If two similar characters span only a few pixels, merging pixels into larger patches can erase the difference between them. The language decoder may still produce a plausible phrase while misreading a character. A plausible result and a faithful transcription are different tests.

The released model offers several resolution settings, squares of 512, 640, 1,024 and 1,280 pixels, plus a mode that adds crops of the page. [3] The figure uses an abstract grid; it does not reproduce those modes or DeepEncoder’s actual tokenization.

Reading text back is only the first test

The paper tests whether the decoder can reconstruct a page’s text from its compressed image. It explicitly leaves broader long-context reasoning to future work. Reproducing a paragraph is a different test from combining facts spread across many compressed pages. [2]

An independent follow-up study compared optical compression with other ways of compressing text. On 1,000-token passages of English Wikipedia, turning older text into images did not predict the next words any better than simply keeping the most recent text within the same token budget. A hierarchical text encoder did better. That study measured next-word prediction, not question answering over long documents, and other lengths, subjects and tasks remain untested. [4]

Checking what survives compression

A useful experiment takes one document, varies its font size, layout and vision-token budget, and scores two things separately: exact transcription, and answers to questions that need information from distant parts of the document. It tracks errors in dates, negations and table alignment, not just whether the output reads fluently.

Consider a two-column table of names and amounts. Every word could be transcribed correctly while the rows are misaligned, pairing each name with the wrong amount. Character accuracy stays high, but the answer to “How much did this person pay?” is wrong. Quality has to be measured against the task that will use the text, including relationships that depend on layout.

DeepSeek-OCR provides a concrete architecture and open weights for studying this trade-off. Its documented uses, converting documents to Markdown and plain OCR, are the right starting points. Claims that images could replace a language model’s long-term memory need further evidence. [3]

Simplifications and limits

• The position-count chart is made up; it is not a published quality or speed result.
• The patch figure is an abstract counting exercise, not DeepEncoder’s tokenization.
• Accurate text reconstruction does not show good reasoning over long contexts.

Follow the idea

DeepSeek-V2DeepSeek-V4

Sources and further reading

Primary papers, model cards, and release posts. Facts describe the versions named here; the interactive calculations are teaching models. Research cutoff: October 3, 2026.

1. DeepSeek-OCR official repositoryRelease news records October 20, 2025 public weights; usage and supported modes read.
2. DeepSeek-OCR: Contexts Optical CompressionDeepEncoder architecture, compression bridge, decoder, resolution modes, experiments and limitations read.
3. DeepSeek-OCR public model cardMIT weights, image resolutions and OCR/document prompts.
4. Optical Context Compression Is Just (Bad) AutoencodingIndependent primary experiments separating reconstruction quality from downstream language-modeling utility; results are setup-specific.