Phi-4-reasoning-vision-15B

Read the image before solving the problem

Microsoft released Phi-4-reasoning-vision-15B on March 4, 2026. The model card lists image and text input, text output, a 16,384-token context and the MIT license. The model joins the Phi-4-reasoning language model to a SigLIP-2 vision encoder. [1]

Suppose a question about a chart asks which month had the highest revenue. The model first has to find the bars, labels and scale. If it misreads “18” as “13”, a flawless comparison afterward can still pick the wrong month. Misreading and misreasoning can produce the same wrong answer, so they need to be tested separately.

Now ask instead for the growth rate between two months. The image is the same, but the computation is different. Naming a label should not need the same long response as deriving a percentage from two values.

More image tokens keep more detail

SigLIP 2 is an image encoder trained to match images with text. Its NaFlex variant accepts images at their own aspect ratio and at a range of resolutions instead of squashing them into a fixed square. Its paper evaluates it on locating objects and reading documents as well as on general image understanding. Its output is a set of learned vectors, not a caption. [2]

The Phi report compared ways of processing images using a smaller, 5-billion-parameter experimental model. Encoding images at their native resolution, with a larger budget of image tokens, worked best on tasks that need fine detail. Those experiments motivated the design; they do not show that higher resolution always helps every task. [3]

The figure uses an abstract image 16 units wide and 12 high. With 2-unit square patches, it contains 8 × 6 = 48 patches; with 1-unit patches, 192. An attention layer that compares every patch with every other compares 48 × 48 = 2,304 pairs in the first case and 192 × 192 = 36,864 in the second. Four times as many tokens means sixteen times as many pairs.

Connecting the encoder to the language model

The report calls its design mid-fusion. The image encoder produces visual vectors; a small learned network, the projector, maps them into the language model’s embedding space; and the resulting vectors are placed among the text tokens for the language model to read. This reuses a language model that was already trained instead of training everything from scratch. [3]

Think of a visual vector and a word vector as measurements in two different coordinate systems. The projector learns a conversion so that the language model can work with both. It does not turn the image into a sentence; its outputs are still vectors of numbers.

Papers use fusion terms inconsistently, so follow the data path rather than the label. Here the encoder processes the image first, and the projector then hands the result to the language model.

Reasoning only when it helps

Microsoft’s research post describes training on a mix of reasoning examples and direct-answer examples. The technical report says each example is marked with a <think> or <nothink> tag, and that reasoning examples make up about 20% of the mix. Users can ask for a mode, but when they do not, the model’s own choice can be wrong. [4] [3]

Take a receipt. “What is the printed total?” mostly means finding and reading one field. “Does the printed total equal the items plus tax?” means reading several fields and checking the arithmetic. How long the answer should be depends on the work the question needs, not on the image.

The tags give the training data a visible structure: they say which kind of response each example teaches. They do not create a perfect controller that knows in advance whether more thinking will help.

Finding which step failed

One way to diagnose a wrong answer is to pose the same problem three ways: as the original image, as a faithful text transcription of the image, and as text that gives only the relevant values. If the model fails only on the image, the problem probably lies in reading it. If it fails on all three, the reasoning, the understanding of the task or missing background knowledge is more likely at fault. Differences between the prompts make this a way to narrow down causes, not to prove one. It is a suggested test, not a result from the release.

Then compare the direct and reasoning modes on the same tasks, recording accuracy, answer length and response time. The research post stresses balancing useful reasoning against unnecessary text; a longer response should earn its cost with better results. [4]

The two budgets stay distinct. More image tokens can preserve evidence, and more output tokens can support a longer calculation. Neither makes up for a shortage of the other. This model is a useful example because it designs and evaluates the two separately.

Simplifications and limits

• The patch grid illustrates encoder cost in abstract units; it is not Phi’s tokenizer or a runtime simulator.
• Cutting an image into patches does not reduce each patch to its average color.
• The model’s learned choice between direct and reasoning answers is imperfect.
• Written reasoning is not guaranteed to explain the model’s internal computation.

Follow the idea

Phi-4-reasoning · reasoning-plusGemma 4 · E2B, E4B, 26B-A4B, 31B, later 12B

Sources and further reading

Primary papers, model cards, and release posts. Facts describe the versions named here; the interactive calculations are teaching models. Research cutoff: October 3, 2026.

1. Microsoft’s Phi-4-reasoning-vision-15B model cardMarch 4, 2026 release, 15B size, 16,384 context, modalities, dependencies and MIT license.
2. SigLIP 2: Multilingual Vision-Language EncodersSections 2–3: NaFlex aspect/resolution flexibility and representation evaluations.
3. Phi-4-reasoning-vision-15B Technical ReportSections 2.1–2.3 and 4: architecture, resolution ablations, mode tokens and data-mixture limitations.
4. Lessons from training Phi-4-reasoning-visionMarch 4 release and perception/reasoning/latency design goals.