Phi-4-reasoning · reasoning-plus
How well-chosen worked examples and a short round of rewards trained a small reasoning model
A model may know the facts a problem needs and still fail to assemble a solution. Phi-4-reasoning, released by Microsoft in April 2025, tackled that gap in a 14-billion-parameter model. It was fine-tuned on long worked solutions to problems chosen to sit just beyond what the base model could do. A second version, reasoning-plus, added a short round of reinforcement learning. The report says both outperform DeepSeek-R1-Distill-Llama-70B, a model five times their size, on several reasoning benchmarks. [3]
Release: . Microsoft released the reasoning and reasoning-plus weights on April 30, 2025. The underlying Phi-4 model was released earlier.Model: 14B dense · SFT and SFT+RL variantsWeight license: MIT
Starting from Phi-4
Microsoft released Phi-4-reasoning and Phi-4-reasoning-plus on April 30, 2025. Both start from Phi-4, a 14-billion-parameter model released earlier. Reasoning-plus adds reinforcement learning after the fine-tuning that both share. The same announcement included Phi-4-mini-reasoning, a separate smaller model not covered here. [1]
Phi-4 itself continued the Phi data strategy described on the Phi-3 page. Much of its training data was synthetic, generated from carefully chosen source material, alongside filtered web pages, code and licensed data. The Phi-4 report also found that removing ordinary web data entirely made the model worse on some tasks, so synthetic data did not replace everything. [2]
Knowing a formula is only the start of using it. For a geometry question, the model must find the relevant lengths, choose the formula, substitute the values and check the units. A worked solution makes each of those decisions visible, and fine-tuning on worked solutions teaches the model to make them.
Choosing problems at the edge of ability
The report describes choosing training prompts that Phi-4 could almost, but not reliably, solve, then collecting long worked solutions to them from OpenAI’s o3-mini. Supervised fine-tuning trained Phi-4 to produce solutions like these. This is imitation of written sequences, which differs from matching a teacher’s full probability distribution at every token. [3]
Picture three practice problems. The model always solves the first, sometimes solves the second and cannot make progress on the third. The first teaches nothing new, and the third is out of reach. The middle one offers the most to learn. That is the intuition behind the selection; the report does not present it as a measured law.
The teacher’s solutions need checking too. A correct final number can follow a mistaken step, and a fluent explanation can skip an assumption. Curating demonstrations means checking both the problems and the solutions the model will learn to imitate.
Examples first, then rewards
Comparing several attempts
Reasoning-plus was trained further with group relative policy optimization (GRPO), the method DeepSeek introduced with DeepSeekMath and used for DeepSeek-R1. The model writes several answers to the same problem, each answer is scored, and each score is compared with the group’s average. Phi’s score combines several things: whether the answer is correct, its length, its format and whether it repeats itself. Correctness is checked by extracting the final answer and testing whether it is equivalent to the reference, with a language model asked to judge when extraction fails, so the checking is not fully mechanical. The figure shows only the group comparison. [3]
Take the “Mixed results” example, with rewards of 0, 0, 1 and 1. The mean is 0.5 and the standard deviation is 0.5. Subtracting the mean and dividing by the standard deviation gives −1, −1, +1 and +1. The signs say which answers did better or worse than the group.
Now choose “All equal”. Every difference from the mean is zero, so the group gives no signal. That does not mean the answers were good: all four might be right, all four wrong, or the scoring rule might have failed to tell them apart.
Compute a group-relative learning signal
The reward defines what gets practiced
Suppose a checker tests only whether the last line contains “12”. It will reward broken reasoning that happens to end in 12. Adding format checks changes what the model learns, but format checks still cannot confirm that the mathematics is valid. Training can use only the evidence the scoring rule collects.
Phi-4’s own post-training had used another way to focus learning on the decisions that matter. Its pivotal token search finds the points in a generated answer where a single token sharply changes the chance of success, and builds preference pairs around those tokens. That technique belongs to the Phi-4 recipe, not to GRPO. [2]
Demonstrations and rewards complement each other. Demonstrations show a route worth imitating; rewards distinguish between the routes the model tries itself. Neither guarantees that the reasoning the model writes is a faithful account of how it reached its answer.
Longer answers cost time
The reasoning-plus model card lists a 32,000-token context and the MIT license. Its outputs contain a reasoning section followed by the answer. The stored parameters stay the same, but more of each response is spent thinking. [4]
Suppose the model generates 20 tokens per second. A 200-token answer takes about 10 seconds; a 2,000-token answer takes about 100. Real speeds vary, and reading the prompt adds its own time, but the arithmetic shows why longer reasoning is a decision about cost.
Measure accuracy together with response time and length. A long answer may solve a hard problem, repeat an error, or merely restate the question. Phi-4-reasoning showed a documented, data-centered way to train reasoning into a small model. It did not show that more visible thinking always gives a better answer.
Simplifications and limits
- The figure computes only a simplified advantage, not Phi’s full training objective.
- Imitating written solutions is different from matching a teacher’s token probabilities.
- Longer reasoning does not guarantee a better answer to any particular question.
Sources and further reading
Primary papers, model cards, and release posts. Facts describe the versions named here; the interactive calculations are teaching models. Research cutoff: October 3, 2026.
- One year of Phi: Small language models making big leapsApril 30 release and distinction among reasoning, plus and mini variants.
- Phi-4 Technical ReportSections 2–4: curated synthetic/organic mixture, ablations and pivotal-token preference construction.
- Phi-4-reasoning Technical ReportSections 2–4: teachable prompt selection, o3-mini demonstrations and GRPO reward design.
- Microsoft’s Phi-4-reasoning-plus model card14B architecture, 32K context, April 30 release, output format and MIT license.