Twitter/X

The kimi-k3-in-c project runs Kimi K3 (2.78 trillion parameters) inference on a…

2026-08-03 · 10:41 UTC ·@hasantoxr ·1 min read

Brief

hasantoxr's post (2026-08-03) introduces kimi-k3-in-c, a 176 KB C99 engine that runs a 2.78T-parameter Kimi K3 model on a single CPU with 8.24 GB RAM using a 1.56 TB NVMe checkpoint. The MoE fires 16/896 experts per token, streams dormant experts on demand, maintains byte-identical outputs across memory budgets and execution paths, and ships as a 45 MB source tree.

Why it matters

The kimi-k3-in-c project runs Kimi K3 (2.78 trillion parameters) inference on a single CPU with only 8.24 GB of RAM and is claimed to be 100% open source; the entire engine is a 176 KB portable C99 binary that requires no BLAS, PyTorch, CUDA, GPU, or AVX-512.

Key details

  • The system uses a 1.56 TB NVMe checkpoint and a sparsity/MoE scheme where only 16 of 896 experts fire per token (≈93% of experts stay dormant and stream from disk on demand), allowing the dense trunk to run at multiple memory budgets (8, 32, 128, 224 GB) with identical weights and byte-identical outputs.
  • Implementation details: reads MXFP4 weights directly without dequantizing to float32; streams the trunk via O_DIRECT (claimed faster than page cache); keeps KDA attention state fixed regardless of context length; produces bit-identical outputs across scalar, OpenMP, and AVX2 paths; verifies kernels against a PyTorch reference; source is 45 MB (six C files + one Python packing script).
Source evidence

Now run Kimi K3 with a 2.78 trillion parameter model on a single CPU with 8.24 GB of RAM. 100% Opensource.

It's called kimi-k3-in-c.

Most inference stacks assume you need a GPU cluster and 5+ terabytes of memory to touch a frontier MoE.

The entire engine is 176 KB of portable C99.

Runs without BLAS, PyTorch, CUDA, any framework, a GPU, or AVX-512.

The 1.56 TB checkpoint sits on NVMe. Only 16 of 896 experts fire per token, so the sleeping 93% streams in on demand and never touches RAM.

The dense trunk runs in whatever memory you give it. 8 GB, 32 GB, 128 GB, 224 GB. Same weights. Byte-identical output at every budget.

What it does:

→ Runs Kimi K3 inference on a laptop with 8 GB free
→ Reads MXFP4 weights directly, never dequantizes to float32
→ Streams the trunk from disk with O_DIRECT, faster than the page cache
→ Keeps KDA attention state fixed regardless of context length
→ Passes bit-identical output between scalar, OpenMP, and AVX2 paths
→ Verifies against the PyTorch reference on every kernel

Ships with a 45 MB source tree, six C files and one Python packing script.

That's the whole thing.