A developer has managed to run GLM-5.2, a mixture-of-experts model with roughly 744 billion parameters, on a consumer computer with around 25 GB of RAM and no graphics card. The project, called Colibrì, does not make the model fast. Its contribution is to treat RAM, graphics memory and storage as different tiers of the same memory hierarchy, loading from disk only the experts required to generate each token.
The key points about Colibrì and GLM-5.2 in 30 seconds
- GLM-5.2 contains about 744 billion parameters, but activates roughly 40 billion for each generated token.
- Colibrì keeps 9.9 GB of shared weights in RAM and stores about 370 GB of quantised experts on disk.
- Its main runtime is written in C and requires neither Python, BLAS nor a GPU during inference.
- On a system with 25 GB of RAM, it reaches between 0.05 and 0.1 tokens per second.
- The experiment proves technical feasibility, not an experience comparable with a cloud service.
The repository, created by developer JustVugg, has attracted more than 16,400 GitHub stars and 1,500 forks. Its first stable version was published on 19 July 2026 under the Apache 2.0 licence, while the GLM-5.2 weights are available under the MIT licence.
The 25 GB figure can lead to the wrong conclusion. Colibrì does not compress the entire model until it fits inside that amount of memory. The hundreds of gigabytes of weights are still required, but they remain on a storage drive and are transferred into RAM as the model requests them.
The result is far too slow for a normal conversation, but it makes it possible to run locally a model originally intended for infrastructure equipped with multiple GPUs and large quantities of high-speed memory.
Not all 744 billion parameters work at the same time
GLM-5.2 belongs to the Mixture of Experts, or MoE, family of models. Instead of using all its parameters for every generated fragment, it contains many specialised blocks and a router that decides which ones should participate.
Z.ai describes GLM-5.2 as an open model for long-running tasks, coding and workloads with context windows of up to one million tokens. Its architecture contains 256 experts in each MoE layer and selects eight of them per token. The complete model has around 744 billion parameters, but activates only about 40 billion during each inference step.
This does not mean the remaining parameters can be deleted. The router may select different experts for the following token depending on the text, language, task and accumulated context. All weights must remain available somewhere, although they do not necessarily need to reside in the fastest memory.
Colibrì divides the model into two main groups:
| Part of the model | Approximate size | Usual location |
|---|---|---|
| Attention, embeddings and shared experts | 17 billion parameters | RAM |
| Shared weights quantised to int4 | 9.9 GB | Permanently resident in memory |
| 19,456 routed experts | Around 370 GB | SSD or NVMe storage |
| Frequently used experts | Depends on available RAM | Memory cache |
| Experts selected for the current token | Around 11 GB of variable data | Loaded when required |
The correct figure reported by the project is 19,456 routed experts, not more than 21,000. Most come from 75 MoE layers containing 256 experts each, with additional weights associated with the multi-token prediction head.
The dense section includes components that participate continuously in inference, such as attention, embeddings and shared experts. Colibrì quantises them to four bits and keeps them in RAM, where they occupy approximately 9.9 GB.
The variable experts remain on disk. Each occupies around 19 MB in the int4 container used by the project. When the router selects a particular set, the runtime locates the corresponding files, loads them and performs the required calculations.
The memory saving comes from avoiding the need to keep all 370 GB of experts in RAM simultaneously. The trade-off is the time spent reading them from storage.
The cache tries to prevent the SSD from slowing down every word
A system that read every expert from disk for each token would work, but its performance would be even worse. Colibrì applies several techniques to reduce the number of storage accesses and hide part of their latency.
The runtime maintains a separate cache for each layer using a least-recently-used policy, retaining recently accessed experts and discarding those used less often. It also records which experts are selected most frequently for a particular workload. That information allows it to pin the most common experts in RAM and adjust their placement as the model is used.
The behaviour resembles the way an operating system manages virtual memory, although the managed objects are not conventional memory pages but complete matrices belonging to the model’s experts.
The engine also stores the three matrices belonging to each expert next to one another so they can be retrieved with a single pread operation. A bounded pool of asynchronous input and output tasks loads weights while the CPU works on experts that are already available.
It also includes a mechanism that attempts to predict which experts the following layer will require. According to the project’s measurements, routing is 71.6% predictable one layer in advance. This result comes from Colibrì’s own tests and may vary according to the task, language and distribution of prompts.
The process can be summarised in five stages:
| Stage | Function |
|---|---|
| Route | Determine which experts the token requires |
| Union | Avoid loading the same expert more than once |
| Place | Decide whether to retrieve it from VRAM, RAM or disk |
| Overlap | Read weights while other calculations continue |
| Learn | Record which experts should remain cached |
Colibrì does not change the router’s selection according to the hardware available. If the model requests an expert stored on disk, the system waits for it instead of substituting another expert that is already in memory.
This decision preserves the intended behaviour of the architecture but sacrifices speed. A more aggressive mode could restrict the model to resident experts, but that would alter inference results and would no longer execute the same model faithfully.
From 0.05 tokens per second to six RTX 5090 GPUs
The original system with 25 GB of RAM achieves between 0.05 and 0.1 tokens per second when the cache is cold. In practical terms, it may take between 10 and 20 seconds to produce a single token. A moderately long answer can require tens of minutes.
This is not competitive with an API or a smaller model running locally. Nor is it intended to be. That machine represents the minimum demonstrated by the author for keeping the model operational without dynamically reducing precision or changing the router’s behaviour.
Measurements gathered by the project show how performance changes when more experts can remain in faster memory tiers:
| Configuration | Reported speed |
|---|---|
| 25 GB system, CPU only, experts streamed from disk | 0.05–0.1 tokens/s |
| 128 GB CPU-only desktop with a warm cache | Around 1.8 tokens/s |
| System with one RTX 5070 Ti | 1.07 tokens/s |
| Six RTX 5090 GPUs with all experts resident | Between 5.8 and 6.8 tokens/s |
These figures come from the project and benchmarks contributed by its community. They are not independent comparisons and were not necessarily run with identical processors, storage, context lengths, cache temperatures or settings. They illustrate the effect of weight placement rather than establishing a definitive hardware ranking.
It is particularly striking that a CPU-only system with large amounts of RAM can outperform a laptop-class GPU in one reported test. The explanation is not limited to computing power. If the 128 GB machine keeps more experts in memory and avoids SSD access, it may offset some of the advantage provided by an accelerator with limited VRAM.
Colibrì therefore turns model deployment into a data-placement problem. The GPU accelerates matrix operations, but that advantage becomes less useful when it constantly waits for weights to arrive from a slower tier.
A C runtime, but not an entire application with no dependencies
The inference core is concentrated in the c/glm.c file and several small headers. It does not require BLAS, Python or a GPU while running. It uses C, OpenMP and system calls to load and process weights.
However, describing the entire Colibrì project as “pure C with zero dependencies” requires qualification. The repository also contains Python code for the model’s initial conversion and for an optional OpenAI-compatible gateway. It includes CUDA components, an experimental Metal backend, a web interface and a Tauri-based desktop application.
Python is used once to download and progressively convert the FP8 weights. This process avoids requiring the full source model, approximately 756 GB, to be present simultaneously, but the user still needs around 370 GB for the final int4 container, in addition to temporary files and caches.
Storage is therefore not a minor detail. A mechanical hard drive would make the delays even less practical. The project makes most sense with a fast NVMe drive and plenty of available space.
Colibrì also implements a compressed key-value cache for attention. It uses 576 floating-point values per token instead of 32,768, a reported 57-fold reduction. The cache can be saved to disk so a conversation can be reopened without processing the entire previous context again.
Another feature is speculative decoding through GLM-5.2’s multi-token prediction head. This component proposes several tokens that the main model can verify together. The project warns that the head must remain in int8: quantising it to int4 reduced draft acceptance to almost zero in its tests.
Running the model is not the same as serving it in production
Colibrì’s main value is not turning a laptop into a competitor for an AI cluster. The differences in speed, concurrency and ability to handle long contexts remain enormous.
Production infrastructure keeps weights in high-speed memory, serves multiple requests at once, batches tokens and distributes calculations across accelerators. The 25 GB system processes one conversation sequentially and spends much of its time waiting for storage.
There is also a difference between preserving the model’s logic and proving that int4 quantisation retains all of its quality. Colibrì says it validated a sequence of 32 tokens exactly against a reference implementation and publishes tests of the cost of quantisation. That supports the correctness of its inference path, but it does not replace a broad evaluation covering reasoning, programming, languages and long-context behaviour.
GLM-5.2 includes an advertised one-million-token context window, sparse attention and improvements for prolonged tasks. Running that full capacity on a system with 25 GB of RAM would add further pressure on memory and processing time. Z.ai presents the model as a tool for agentic coding and long-running projects, but its published results come from far more powerful infrastructure.
The experiment nevertheless proves an important point: in an MoE model, the amount of fast memory required can be partially separated from the total size of the weights. Parameters that do not participate in a particular token can remain in a slower tier, provided the runtime knows how to locate, predict and load them.
Hardware still determines performance. What changes is the minimum required for the model to run at all. Colibrì lowers that threshold through software architecture, caching and a precise understanding of GLM-5.2’s internal design.
Frequently asked questions
Does GLM-5.2 really fit in 25 GB of RAM?
Not entirely. Around 9.9 GB remains loaded in RAM, while approximately 370 GB of quantised experts is stored on disk. Experts are transferred into memory when the model needs them.
Does Colibrì require a graphics card?
No. It can run entirely on the CPU, although it also offers optional CUDA support and an experimental Metal backend. A GPU improves performance when it allows more weights to remain in fast memory.
How fast is it on a computer with 25 GB of RAM?
The project reports between 0.05 and 0.1 tokens per second with a cold cache. That is roughly one token every 10 to 20 seconds.
Is all of Colibrì written exclusively in C?
The runtime inference core is concentrated in C and does not require Python or BLAS. The complete repository includes Python for conversion and the optional API, as well as CUDA, TypeScript and other components.