Qwen3.5
How visual tokens join a hybrid language model
A chart is more than its text. Its axes, the positions of its points and its labels carry the meaning together, and transcribing the words loses most of it. Qwen3.5, released by Alibaba’s Qwen team in February 2026, [2] was trained on images and text together from the start. It is built on the hybrid architecture introduced with Qwen3-Next, in which most layers keep a fixed-size memory instead of a growing cache. Images and long documents produce long inputs, so that cheaper memory matters more once a model can see. [1]
Release: . Qwen released the flagship weights on February 16, 2026. Smaller models followed later.Model: 397B total · 17B active flagshipWeight license: Apache 2.0
From pixels to vectors
A language model works on vectors. An image encoder turns an image into vectors too, and a projection layer maps them into the form the language model uses. Early fusion means that images and text are trained together during the model’s main training, instead of a vision adapter being attached to a finished text model.
Qwen3.5’s first open model, 397B-A17B, is described as a language model with a vision encoder, trained with early fusion on a mixture-of-experts backbone. The name means about 397 billion parameters in total, of which about 17 billion are used for each token. [1]
Consider a scatterplot with one outlier. Text recognition can recover the title and axis labels but not the point’s position. A visual representation has to carry spatial information alongside the text. Even a perfectly captured picture still requires reasoning to decide whether the outlier undermines the conclusion.
One context for images and text
Three memory layers, then one attention layer
The flagship keeps the layer pattern introduced in Qwen3-Next: three Gated DeltaNet layers, which update a fixed-size memory, followed by one gated-attention layer, which can look back at any earlier token. Qwen3.5 has 60 layers in this pattern. Its expert layers have 512 routed experts, of which each token uses 10, plus 1 shared expert. These mechanisms come from earlier Qwen work and are developed further here for images and text. [1]
Why does this matter for images? A document of many pages becomes a long stream of visual and text representations. A recurrent layer can absorb each new piece without adding to a growing history, while the attention layers keep a route back to specific earlier pieces. The model has to decide what to keep before later questions reveal which details matter.
In the figure, double the number of tokens. The stored history doubles; the recurrent state does not change. The arithmetic shows the difference between storing a growing sequence and updating a fixed-size state. It does not say how much memory a photo, page or video takes in Qwen3.5.
Long inputs, two kinds of memory cost
Training examples that need the image
Suppose a training problem shows a line chart and asks which month had the largest increase. A good target depends on the chart: read the values for neighboring months, compute the differences, compare them. If the answer can be guessed from the title alone, the example rewards a shortcut instead of looking.
That suggests a test of whether a model uses what it sees. Swap the chart while keeping the question the same; a model that reads the chart should change its answer when the relevant points change. Remove the axis scale, and the right response may become uncertainty rather than a number. These are suggested tests, not published Qwen results.
The model card describes asynchronous reinforcement-learning infrastructure for agent tasks, in which generating attempts and learning from them run on separate schedules so the system can collect and score many interactions. More environments do not make every reward reliable. A visual answer can look plausible while misreading a small label, and a tool action can succeed without serving the user’s goal. [1]
The open model and the hosted product
The model card distinguishes the open checkpoint from Qwen3.5-Plus, the hosted product built on it. The checkpoint natively accepts 262,144 tokens and has a documented recipe for extending that. Hosted tools and default context settings are product features, and downloading the weights does not reproduce them. [1]
For a reproducible test, record the image preprocessing, resolution, prompt, thinking settings, context length and serving software. A smaller image saves tokens but may erase the very legend a question depends on. A larger one keeps detail but costs time and memory. Parameter count cannot settle that trade-off.
The design problem Qwen3.5 tackles has three parts: train the model to use visual evidence, keep enough context for that evidence to stay available, and keep the computation affordable. The figures on this page separate those parts so that each can be judged on its own, apart from benchmark rankings.
Simplifications and limits
- The diagram is conceptual; it does not specify Qwen3.5’s exact image tokenization.
- The perception tests described here are suggestions for teaching, not published measurements.
Sources and further reading
Primary papers, model cards, and release posts. Facts describe the versions named here; the interactive calculations are teaching models. Research cutoff: October 3, 2026.
- Qwen3.5-397B-A17B model cardArchitecture, early fusion, expert configuration, native context and distinction from hosted Plus.
- Qwen3.5 official announcementOfficial launch page; its dynamic body did not expose readable text during this audit. Architecture and training claims above are supported by the model card.
- Qwen3.5 official weight historyFirst complete weight upload and public release on February 16, 2026.