@emollick (2026-07-27) inspected the raw weights of the released model and…

Brief

Kimi K3 is a newly released 2.8T-parameter MoE model from Kimi.ai that claims native vision, a 1M-token context window, and 2.5x intelligence per unit of compute. @emollick inspected the raw weights on 2026-07-27 and noted the first page is dominated by negatives, ends with zeros, and contains one surprisingly large positive value; MoonshotAI also published weights, a tech report, attention kernels, an MoE comms library, and agent-running infrastructure.

Key details

Kimi.ai announced Kimi K3: a 2.8T-parameter Mixture-of-Experts (MoE) model with native visual understanding, a 1,000,000-token context window, a new architecture they claim gives 2.5x the intelligence per unit of compute, and released model weights (Hugging Face), a technical report (GitHub), high-performance attention kernels, an MoE communication library, and agent infrastructure.

Been reading the weights of the most powerful open weights Ai model yet released.

Page one starts strong: mostly negatives, a late run of zeros, and one unexpectedly large positive value.

Kimi.ai (@Kimi_Moonshot)

Releasing the model weights and technical report of Kimi K3.

Kimi K3 is our most capable model: a 2.8T MoE model with native visual understanding and a 1M-token context window.

New model architecture: 2.5x the intelligence per unit of compute, not just more params.

Alongside Kimi K3, we're opening up more of the stack behind it — high-performance attention kernels, MoE communication library, and infrastructure for running agent environments at scale.

Model weights: huggingface.co/moonshotai/Ki…
Tech report: github.com/MoonshotAI/Kimi-K…
Tech blog: kimi.com/blog/kimi-k3

— https://nitter.net/Kimi_Moonshot/status/2081760186235289764#m

Brief

Why it matters

Key details