GLM-5

How an agent can learn while other agents are still working

When a model learns from agent tasks, the tasks in a training batch take very different amounts of time: one needs a quick calculation, another waits minutes for a software build. If learning waits for every task in the batch to finish, most of the hardware sits idle behind the slowest one. GLM-5, released by Z.ai in February 2026, [2] separates generating experience from learning from it, so the slowest task no longer holds everything up. That separation creates a new problem, learning from experience produced by an older version of the model, and this page explains how GLM-5 handles it. The model has 744 billion parameters, 40 billion of them active per token. [1]

4 sections · about 7 minutes · Training

Release: 2026-02-11. The weights were uploaded on February 11, 2026. Z.ai’s blog post is dated February 12 and the report February 17.Model: 744B total · 40B activeWeight license: MIT model weights

The slowest attempt sets the pace

A rollout is one attempt at a task: the model’s actions and the environment’s replies, from start to finish. In a simple synchronous setup, every rollout in a batch must finish before the model is updated, so tasks of different lengths leave workers waiting.

Take three rollout workers whose tasks take 2, 3 and 10 time units. If all must wait until time 10, the first worker sits idle for 8 units and the second for 7. That idle time is a scheduling problem; it does not mean the long task is less valuable.

GLM-5’s report describes separate engines for generating rollouts and for training. The generation side produces rollouts continuously. The learner takes batches of completed rollouts, updates the weights and periodically sends the new weights back to the generators. Each kind of task environment runs as its own service under a common coordinator. [1]

Generation and learning on separate schedules

Simplified asynchronous reinforcement-learning pipeline. Rollout generation continues while the learner updates. Decoupling generation from learning removes the forced wait for every running task.

Overlap removes waiting, not work

The figure schedules four independent jobs that take 3, 2, 4 and 1 time units. One worker needs 10 units. Adding workers lets the jobs overlap. This is a small analogy for keeping workers busy, not a simulation of GLM-5’s training system.

Watch the longest row; it sets when the whole set of jobs finishes. If each job depended on the previous one’s result, extra workers would not help at all. The arithmetic separates how much parallelism the work allows from how many machines are available.

Training adds a complication. A rollout that finishes now may have been generated with older weights than the learner currently holds. Faster scheduling therefore changes the statistics of learning as well as the use of hardware, and a working system has to handle both.

The cost of an uneven mix of jobs

Made-up independent job durations with greedy scheduling. Nothing here estimates GLM-5’s training speed. Overlapping independent jobs shortens elapsed time without reducing total work.

Learning from an older policy’s experience

A policy is the model viewed as a decision-maker: its probabilities over possible next actions. Every update changes those probabilities. A rollout generated before the latest updates came from an older policy, and the learner cannot treat it as if the current policy had produced it. This mismatch is called policy lag, and data affected by it is called off-policy.

GLM-5 handles this in several ways. It refreshes the rollout workers’ weights periodically. It keeps the exact token IDs the model generated, because converting the text back into tokens later can split it differently and misalign the actions being learned from. [1]

It also stores the probability each generated token had when it was sampled. During learning, it compares that stored probability with the current policy’s probability for the same token. If their ratio falls outside a set range, the token is excluded from the update entirely, not merely capped. Whole rollouts are dropped when the policy that generated them is too many versions behind the learner. Together, these limit the influence of stale experience. [1]

Asynchronous learning is therefore more than putting a queue between two programs. The stored data must carry enough information for the training objective to stay meaningful when generation and learning no longer run in step.

A separate saving: sparse attention

GLM-5 also adopts DeepSeek Sparse Attention, described on the DeepSeek-V3.2 page. A lightweight indexer scores the earlier context, and the expensive attention step reads only the selected tokens. The cache and the indexing work still grow with the context; this is not a fixed-size recurrent state. The technique comes from DeepSeek, and it addresses a different bottleneck from the asynchronous training system. [1]

Sparse experts choose which weights process a token. Sparse attention chooses which earlier tokens to read. Neither fixes idle rollout workers. Keeping these mechanisms apart makes each efficiency claim easier to check.

Later versions changed things further. GLM-5.2 introduced IndexShare, which reuses one indexer across groups of sparse-attention layers. GLM-5.3 kept the 5.2 base model and focused on post-training, and GLM-5.3-Flash changed the base architecture again. The official family repository keeps these developments separate. [3]

The broader lesson is to identify the cost before choosing a remedy. Stored parameters, reading the context, time spent waiting for tools and stale training data are separate engineering problems.

Simplifications and limits

The scheduling figure shows independent jobs overlapping. It does not reproduce GLM-5’s training framework, called slime, or its asynchronous optimization.
The gains from sparse attention and from asynchronous training cannot be multiplied into one overall speed-up without measurements.

Sources and further reading

Primary papers, model cards, and release posts. Facts describe the versions named here; the interactive calculations are teaching models. Research cutoff: October 3, 2026.

GLM-5 technical reportSections 2.1 and 4.1; DSA, separate rollout and learner engines, exact token alignment, probability-ratio masking, and policy-version filtering.
GLM-5 official announcementArchitecture scale, adopted DSA and MIT release.
GLM-5 family repositorySeparates GLM-5.2 IndexShare, GLM-5.3 post-training and Flash architecture.
GLM-5 official weight historyWeight uploads February 11, 2026; blog dated February 12.