The open-source project kimi-k3-in-c, published by developer Fareed Khan, runs the full 2.78-trillion-parameter Kimi K3 model on a single CPU with a measured peak memory of 8.24 GB, no GPU and a 176 KB binary written in C99. For sysadmins the interesting part is not so much the memory figure as what it implies: once the model does not fit in RAM, performance depends almost entirely on storage. With 8 GB it generates a token roughly every 26.5 seconds; with 128 GB or more, every 5.6 seconds.

Key points on running Kimi K3 on a CPU in 30 seconds

What it is: a C99 engine, with OpenMP as its only real dependency, that runs the full Kimi K3 on CPU alone.
Requirements: a CPU with AVX2 and FMA or NEON, at least 8 GB of RAM and about 1.7 TB of disk.
Performance: from 26.5 s per token with 8 GB to 5.6 s with 128 GB, always with identical output.
The catch: disk matters more than CPU; the engine reads with O_DIRECT and pulls 5.4 to 6 GB/s from NVMe.

Kimi K3, Moonshot AI’s open model, is a 93-layer mixture of experts (MoE) with 896 experts per layer, of which only 16 fire for each token. The engine exploits that structure: the 1.447 TB of routed experts, which ship in the 4-bit MXFP4 format, are never held in memory and are read from disk as the router requests them, while the roughly 113 GB dense trunk is repacked into a single 109 GB file that is read layer by layer. The author says the output is byte-identical at every memory budget.

Requirements and setup

Linux x86-64 is the reference platform; it also builds and passes every test on macOS with Apple silicon and on Windows through MSYS2 with MinGW-w64, and runs unmodified under WSL. You need a CPU with AVX2 and FMA on x86-64 or NEON on ARM (AVX-512 is not required), GCC 9 or Clang 10 or later, and Python 3.9 only for the download and packing tools. Storage is the real gate: 1.56 TB for the checkpoint and 109 GB for the packed trunk, ideally on local NVMe.

git clone https://github.com/FareedKhan-dev/kimi-k3-in-c.git
cd kimi-k3-in-c
./scripts/k3-doctor.sh          # checks toolchain, RAM, disk; suggests a preset
make -j && make test            # builds and validates with no model download
export HF_TOKEN=hf_...
./scripts/download-model.sh ~/k3model     # 1.56 TB, resumable
./scripts/pack-trunk.sh ~/k3model ~/k3trunk   # about 4 minutes, once
./bin/k3 ~/k3model --trunk ~/k3trunk --preset laptop 
  --tok ~/k3model --prompt "Hello" --gen 16 --incrementalCode language: PHP (php)

Running make test before committing to a 1.56 TB download is worthwhile: it compares the engine exactly against a PyTorch reference on a 13-layer model with the same tensor graph and needs no network. The download script checks the exact size of each of the 96 shards, because a partial download does not fail with an error, it silently produces wrong tokens. One thing to watch: if you omit --trunk, the whole trunk is loaded resident, about 113.5 GB, whatever preset you chose.

Memory presets and how they were measured

The engine splits memory between two budgets: the pinned trunk (plus a read ring for layers streamed from disk) and an LRU cache for routed experts. Presets range from laptop (8.2 GB peak) to desktop (31.9 GB), workstation (95.5 GB), server (about 128 GB, with 90 of 93 trunk layers pinned) and max (about 224 GB). An ultra preset of around 3 GB exists, but the author describes it as a proof of life rather than something to use. The engine refuses any plan above 95% of available memory, so server needs roughly 135 GB free.

Every figure comes from a single machine: two AMD EPYC 7763 CPUs with 124 cores and no SMT, 228 GB of RAM and 3.2 TB of NVMe. To simulate each memory tier, the author constrained the process with a systemd control group and no swap, which is easy to reproduce:

systemd-run --scope --user -q -p MemoryMax=8G -p MemorySwapMax=0 
  ./bin/k3 ~/k3model --trunk ~/k3trunk --trunk-gb 2.5 --cache-gb 0.5 
  --ids 1008,10484,318,15383,387 --gen 8 --incrementalCode language: JavaScript (javascript)

That means the 26.5 seconds per token at the smallest tier were measured with 124 cores and fast NVMe, not on a real 8 GB laptop. On a typical home machine with fewer cores and a slower SSD, expect it to be slower. The author also notes that unattended-upgrades and the apt timers had to be disabled before measuring, because the updater was eating 63% of a core and skewing timings more than most optimizations did.

Why storage decides everything

With little memory, the model is read from disk on every step. According to the run reports, between 41% and 61% of wall-clock time goes to I/O, and the trunk is read at a sustained 5,373 to 6,064 MB/s. One counter-intuitive finding shaped the design: on that machine, reading with O_DIRECT, bypassing the kernel page cache, was faster (3.2 GB/s cold) than reading through it (2.3 GB/s), so the engine opens the trunk with O_DIRECT and aligns every read to the block size. The repository ships tools/devbw.py to measure a disk the way the engine uses it, with large random reads at queue depths 1 and 16, which dd does not capture. The author warns that network volumes can be several times slower than local NVMe.

Another useful sizing lesson: at a fixed 128 GB budget, giving memory to the trunk before the expert cache was 1.69 times faster, and the max preset, with 96 GB more, did not beat server outside measurement noise. For scripting, the binary returns distinct exit codes: 2 for usage errors or a config it cannot read confidently, and 4 when a run completes but a routed expert failed to load, meaning the output cannot be trusted. --threads defaults to physical cores, because the engine runs slower with SMT.