GLM-5.3-Flash

A new base, not just new tuning

The model card describes GLM-5.3-Flash as the first natively multimodal model in the GLM-5 series. It lists 320 billion parameters in total, 18 billion active, and a new base model trained on a 30-trillion-token corpus of text and images. It names two architectural features: hybrid sparse and linear attention, and manifold-constrained hyper-connections (mHC). [1]

That sets it apart from the larger GLM-5.3, which, according to the family repository, reused the GLM-5.2 base model and improved through post-training. Flash has a different architecture; it is not the same model served with a faster setting. [2]

Post-training changes which behaviors a model’s existing representations support. A new base changes how information is represented, stored and mixed before any behavior is taught. To compare the two, look at their configurations; the word “Flash” is a name, not a specification.

Three memory layers, then one lookup layer

The released configuration has 45 layers: eleven groups of three linear-attention layers, each followed by one DeepSeek-style sparse-attention layer, plus one final linear layer. That makes 34 linear layers and 11 sparse ones. Linear attention layers update a fixed-size state, much like the recurrent layers on the Qwen3-Next and Kimi K3 pages; the configuration lists them as kda_layers, after Kimi Delta Attention. The sparse-attention layers keep a cache and use an indexer to choose which earlier tokens to read. The configuration shows the chosen layout but not every detail of the code that runs it. [3]

Think of reading a long technical manual. A running summary can keep track of the current subject, but a later question may need one specific serial number. Compression and direct lookup do different jobs, although the trained model need not split them along these human lines.

The figure compares a made-up fixed state with a history that stores 64 numbers per token. Lengthening the context grows only the second. In the real model, the sparse-attention layers still keep a cache that grows with the context, so the model as a whole does not use fixed memory. Reading a selected subset of the cache also differs from a sliding window, which discards everything outside the window.

Mixing residual streams without runaway growth

A residual stream carries each token’s representation from layer to layer, with each layer adding its contribution. Hyper-connections widen this into several parallel streams and learn how to mix them between layers. DeepSeek’s mHC paper constrains the mixing matrix to be doubly stochastic: every entry is nonnegative, and every row and every column sums to one. A procedure called Sinkhorn–Knopp iteration, which repeatedly rescales the rows and then the columns, approximates that constraint in a fixed number of steps. GLM-5.3-Flash adopts this mechanism. [4]

Here is the constraint at work with two streams holding 2 and 8. Mixing with rows [0.75, 0.25] and [0.25, 0.75] gives 0.75 × 2 + 0.25 × 8 = 3.5 and 0.25 × 2 + 0.75 × 8 = 6.5. Each output is a weighted average of the inputs, and the total stays 10. Mixing again gives 4.25 and 5.75.

Now mix with unconstrained rows [1, 1] and [1, 1]. The same inputs become 10 and 10, then 20 and 20, doubling every time. The constraint prevents this kind of growth in the mixing step. It does not by itself make every other operation in the network stable.

Configuration versus capability

The released configuration sets hc_mult, the number of residual streams, to 4, and hc_sinkhorn_iters to 20. Twenty iterations approximate the constraint closely, but the rows and columns will not sum to exactly one in floating-point arithmetic. [3] A configuration file establishes sizes and switches. It cannot establish that a model understands a photograph or reliably fixes software.

Those claims need testing. For a visual task, change the evidence in the image and check that the answer changes accordingly. For a coding task, run the repaired program against meaningful tests. For a long-context task, place the relevant facts at several distances and include distracting material.

The architecture does suggest which costs to measure: the recurrent state, the sparse cache and its indexing, moving experts between chips, and the traffic in the residual streams. These effects can interact, so measure the whole system instead of multiplying favorable ratios from separate parts.

Simplifications and limits

• Z.ai’s blog post about Flash had no readable text when this page was researched; the page relies on the model card, the configuration file and the mHC paper.
• The mHC arithmetic describes only the mixing between residual streams. It is not the whole block and does not guarantee stable training.

Follow the idea

GLM-5Kimi K3Qwen3.8-Flash-Next

Sources and further reading

Primary papers, model cards, and release posts. Facts describe the versions named here; the interactive calculations are teaching models. Research cutoff: October 3, 2026.

1. GLM-5.3-Flash official model cardNew multimodal base, total/active parameters, corpus size, named architecture and MIT weights.
2. GLM-5 family repositoryDistinction between 5.3 post-training and 5.3-Flash base architecture.
3. Released GLM-5.3-Flash configuration45 layer_types entries (34 linear, 11 sparse), KDA-named indices, indexer fields, hc_mult=4 and hc_sinkhorn_iters=20.
4. mHC component paperSections 4.1–4.2 and 5.3; doubly stochastic residual mixing, finite Sinkhorn approximation and stability limits. The exact two-stream example is derived here.
5. GLM-5.3-Flash weight historyWeights uploaded August 26, 2026; repository initialization August 25 is not used as launch date.