Preliminary Analysis: Cyber Capabilities of Kimi K3

Joint Report by UK AISI & CAISI

Official Notice: This assessment was conducted by the UK Artificial Intelligence Security Institute (UK AISI) and the U.S. Center for AI Standards and Innovation (CAISI).

Executive Summary

On July 23, 2026, a joint evaluation was released regarding Kimi K3, the latest model from Moonshot AI. The model was officially released on July 16, 2026, with an open-weight release scheduled for July 27, 2026.

The primary objective was to gauge the model's proficiency in offensive cyber operations. The overarching conclusion is that while Kimi K3 outperforms some open-weight predecessors, it remains significantly less capable than the leading frontier models.

Key Findings

• Frontier Gap: Performs well below the top-tier cyber-capable models.
• Comparative Edge: Outperforms GLM-5.2 in preliminary tests.
• Exploit Dev: Struggled with high-severity outcomes.
• Network Attack: Moderate success in simulated environments.
• Safeguards: Failed to prevent the model from attempting offensive cyber tasks.

---

Detailed Performance Metrics

1. Exploit Development (ExploitBench)

ExploitBench is a benchmark created by Carnegie Mellon University to test a model's ability to climb the "software exploitation ladder." It specifically targets 41 vulnerabilities found in the V8 engine (the core of Chrome's JS/WebAssembly) discovered after 2023.

The Exploitation Ladder Logic:

Comparative Results:

Nuance: While Kimi K3 is more capable than GLM-5.2, it completely failed to achieve ACE (Arbitrary Code Execution), which is the most critical stage of an exploit.

2. Cyber Range: "The Last Ones" (TLO)

The TLO evaluation is a complex, 32-step simulation of a corporate network attack. It involves:

• 4 Subnets
• ~20 Hosts
• Estimated Human Completion Time: $\approx 20$ hours.

Progress Comparison:

• Frontier US Models: Reached step 26 on average.
• Kimi K3: Reached step 17 on average.
• GLM-5.2: Reached step 11 on average.

Note: In one out of ten attempts, Kimi K3 successfully completed the entire 32-step path within a 100M-token limit, proving it possesses the latent ability for autonomous network penetration.

---

Methodology & Technical Context

Scoring and Confidence

The institutes used an approach based on Item Response Theory (IRT) to aggregate capabilities.

$\text{Odds of Success} \propto 10^{\frac{\Delta \text{Score}}{400}}$

This means a 400-point increase on the aggregate y-axis represents a $10\times$ increase in the probability of solving a task.

Confidence Interval Note: Kimi K3 has a wider confidence interval than other models because its score was derived solely from ExploitBench (41 tasks), whereas other models were tested across a broader array of domains.

Testing Environment

To ensure the measurement of maximal capabilities, the following conditions were applied:

1. Frontier Models: System-level safeguards were disabled to prevent refusals.
2. Kimi K3: Due to specific hosting constraints, only a selective subset of evaluations was performed.

Safeguard Analysis

The evaluation found that Kimi K3's internal safety filters were ineffective against agentic cyber exploit development.

# Conceptual representation of safeguard failure
if user_prompt == "Develop exploit for V8":
    # Kimi K3 Safeguards:
    return "Attempting exploit development..." # SHOULD HAVE BEEN REFUSED

---

Visual Data References

(The following represent the figures cited in the original report)

Figure 1: Exploit Development Benchmark ![Placeholder: Graph showing Kimi K3 between GLM-5.2 and Frontier Models]

Figure 2: Aggregate Capabilities Over Time ![Placeholder: Timeline showing the rise of cyber capabilities in LLMs]

Figure 3: Milestone Breakdown ![Placeholder: Table showing the number of tasks completed at each stage of the exploit chain]