Kimi-K3 on HuggingFace

🚀 Model Overview

Kimi K3 represents the pinnacle of Moonshot AI's development to date. It is a native multimodal agentic model released with open weights, designed to push the boundaries of frontier intelligence.

At its core, Kimi K3 is a massive $\approx 2.8\text{T}$ parameter model. It leverages innovative architectural components like Kimi Delta Attention (KDA) and Attention Residuals (AttnRes). With a native ability to process visual data and a massive context window of 1 million tokens, it is the first open-weight model in the 3T-parameter class. It is specifically engineered for complex, long-term tasks involving reasoning, professional knowledge work, and advanced coding.

🌟 Primary Capabilities

• Advanced Architecture: By utilizing a Stable LatentMoE framework, Kimi K3 optimizes sparsity by activating only 16 out of 896 experts. This results in a $\approx 2.5\times$ boost in scaling efficiency compared to its predecessor, Kimi K2.
• Long-Horizon Engineering: The model can manage extended coding sessions with minimal human intervention. It is capable of navigating vast codebases and utilizing terminal tools for high-level tasks such as:
  • GPU kernel optimization
  • Compiler development
  • Chip design
  • CAD and vision-integrated game development
• Agentic Knowledge Production: Kimi K3 transforms knowledge work by generating comprehensive research. Its multimodal nature allows it to create interactive dashboards, widgets, motion designs, and perform video editing.
• Native Multimodality & Scale: It processes text, imagery, and video within a single unified model, supported by a $1,048,576$ token context window.
• Open Access: Closed-source frontier models $\rightarrow$ Open Frontier Weights. The model is available under the Kimi K3 License to foster global research and innovation.

---

🛠 Technical Specifications

The following table outlines the structural composition of Kimi K3:

MoE Activation Logic

---

📊 Evaluation Results

Kimi K3 was tested against several industry-leading models. All Kimi K3 scores were achieved using reasoning effort = max and temperature = 1.0.