‑k3‑in‑c is a pure C99 inference engine that demonstrates running the Moonshot AI Kimi K3 mixture‑of‑experts (MoE) model — 2.78 trillion parameters — on an ordinary CPU with only ~8 GB of RAM, producing bit‑identical outputs to a full‑memory run.
What it is
A from‑scratch, minimal C99 implementation of a Kimi K3 inference engine.
The full model weights are ~1.56 TB; the project uses clever streaming and caching so the model can be used on machines with very limited RAM.
It is not an approximate or heavily quantized “toy” version — the engine guarantees the same token sequence as a full in‑memory run.
Key features
Extremely lightweight: single binary ~176 KB, no deep‑learning frameworks required.
Memory engineering: runs with as little as 8.24 GB RAM by streaming backbone layers and caching only active experts.
Deterministic outputs: identical byte‑for‑byte results whether run with 8 GB or 224 GB of RAM; memory affects speed only.
Pure C99 codebase with optional AVX2 acceleration; no BLAS, PyTorch, or CUDA dependencies.
Test suite: includes self‑tests that run without model weights to validate core functionality.
How it works (plain terms)
Think of the model weights as a huge library stored on disk. Only the small subset of “books” (experts and backbone slices) needed for the current tokens are loaded into RAM. The engine uses direct disk I/O and an expert cache so most of the 1.56 TB stays on disk until needed.
Requirements and performance
Disk: ~1.7 TB free to download weights (weights ~1.56 TB).
OS/CPU: Linux x86‑64; single‑core CPU is supported.
RAM: minimum ~8 GB.
Speed: slow on low‑memory setups (example: ~30 seconds per token); disk throughput is the main bottleneck.
Usage: clone, build, and run tests locally:
git clone https://github.com/FareedKhan-dev/kimi-k3-in-c.git
cd kimi-k3-in-c
make -j
make test
Who should read or use it
Researchers and engineers interested in large‑model inference, MoE architectures, and extreme memory optimization.
Developers who prefer portable, auditable C implementations.
Enthusiasts curious about engineering tricks that let massive models run on modest hardware.
Bottom line
Kimi‑k3‑in‑c is a technical proof‑of‑concept showing that smart engineering — streaming, selective caching, and careful I/O — can make seemingly impossible model deployments feasible on commodity machines. It’s primarily a research and engineering demonstration rather than a practical, low‑latency production solution.