Kimi on Mac Studio m3 ultra
Pipe Network (@pipenetwork)
Run Kimi K3 on a Mac Studio 🫰
K3 is 2.8T parameters and 1.6TB on disk, which makes it impossible to run on Apple Silicon.
Until now.
Our MLX port is now open source: github.com/PipeNetwork/kimi-…
To accomplish this, we solved two things:
We wrote a streaming converter that walks one layer at a time, so that mlx_lm doesn't need to materialize the whole model.
REAP pruning sits on top and scores all 896 experts against a calibration corpus to keep only ones your workload needs.
That's what brings K3 down to 350GB and inside a Mac Studio.
Link
GitHub - PipeNetwork/kimi-k3-mlx: MLX port of moonshotai/Kimi-K3 (2.78T multimodal MoE): streaming...
MLX port of moonshotai/Kimi-K3 (2.78T multimodal MoE): streaming converter, REAP expert pruning, and per-language expert-overlap analysis - PipeNetwork/kimi-k3-mlx
github.com
— https://nitter.net/pipenetwork/status/2081910870083285198#m