🦥 Unsloth Dynamic 3.0 GGUFs

Welcome to the next evolution of quantization. Unsloth Dynamic v3.0 represents a significant leap forward from version 2.0, optimizing how we compress large language models without sacrificing intelligence.

🚀 The Big Announcement: Qwen3.8-27B

We are officially releasing the Qwen3.8-27B Dynamic v3.0 quants. When compared to every other provider at the same model size, our quants deliver a 10% improvement in top-1% accuracy.

🛠️ Compatibility & Performance

These new GGUF files are designed for seamless integration with:

• llama.cpp
• Unsloth Desktop (The first local app for running trained models)
• Various other industry-standard inference engines

By focusing on metrics such as KL Divergence and Divergence-300 @32, Dynamic v3.0 maintains superior model quality while keeping the footprint small. The community response has been overwhelming, with over 5.1 million downloads of the Qwen3.8 quants in the first five days!

---

🧬 Our New Methodology

The jump to v3.0 wasn't accidental. We overhauled our approach to ensure the highest possible fidelity.

Key Improvements:

1. Enhanced Calibration: We utilize a high-quality imatrix calibration dataset sourced from diverse origins, specifically tuned for:
   • 💻 Agentic Coding
   • 💬 Natural Chat
   • 🌍 Multilingual capabilities
2. Precision Engineering: We've refined our layer selection process and implemented a wider array of quantization techniques.
3. Pure PTQ: To ensure transparency and stability, we follow a strict protocol:
   • No training on the imatrix calibration dataset
   • No Quantization-Aware Training (QAT)
   • No Quantization-Aware Distillation (QAD)

Community Note: Our imatrix file is open for the public. We invite researchers and developers to use it to create their own fine-tunes or variations of Qwen3.8.

📊 Quantization Tiers & Space Optimization

To save disk space, we removed the MTP module from smaller quants (those $\le 8.37\text{GB}$). If you require it, a separate Q4_0 MTP module is available.

---

📈 Benchmarking & Analysis

The "Divergence-300 @32" Metric

While "top-1% accuracy" is a common benchmark, it is essentially an $\text{argmax}$ on a single prediction, which doesn't fully capture real-world inference. To solve this, we developed a more rigorous test:

The Process:

1. Create a held-out dataset of 300 examples (excluding calibration data).
2. Sources include: Terminal-Bench 2.1, DeepSWE, Harbor, MathArena 2025-26, and long-document/non-Latin prompts.
3. Perform greedy $\text{argmax}$ decoding for 32 tokens.
4. Compare the trajectory of the quant against the $\text{BF16}$ original.

This allows us to measure if the model is overfitting or if the output trajectories diverge over time.

KL Divergence (KLD)

Our KLD benchmarks show that Unsloth UD-3 quants provide up to +10% extra top-1% accuracy at the same disk space compared to other providers.

Note: To ensure a fair "apples-to-apples" comparison, the MTP head is removed from the x-axis when calculating disk space in our plots.

🛡️ Preventing Overfitting

We take aggressive measures to ensure our models generalize well:

• Dataset Separation: Calibration datasets are entirely different from testing datasets.
• Pure PTQ: By avoiding QAD/QAT, we significantly reduce the risk of overfitting.
• External Validation: Using the Divergence-300 test on unseen prompts confirms that UD-3 methods remain robust.

---

🗺️ Quantization Workflow

---

📜 Legacy & Updates

Dynamic v2.0 (The Predecessor)

Before v3.0, we introduced Dynamic v2.0, which set new records for Aider Polyglot, 5-shot MMLU, and KL Divergence. While we still use UD-2 for some larger quants (as they perform similarly to UD-3), v3.0 is the new standard for smaller sizes.

📅 Timeline of Improvements

• Feb 27, 2026: Qwen3.5 release; fixed tool-calling templates; Perplexity KLD benchmarking.
• Apr 20, 2026: New GGUF benchmarks released for Qwen3.6 and Gemma 4.

🛠️ Quick Start Checklist

• Download the UD-Q2_K_XL or UD-IQ1_S quant.
• Install llama.cpp or Unsloth Desktop.
• (Optional) Load the separate Q4_0 MTP module for enhanced performance.
• Run inference and verify against BF16 trajectories.

💻 Example Loading Logic

# Conceptual example of loading an Unsloth Dynamic Quant
from unsloth import FastLanguageModel

model, tokenizer = FastLanguageModel.from_pretrained(
    model_name = "unsloth/Qwen3.8-27B-Dynamic-v3.0-GGUF",
    max_seq_length = 2048,
    load_in_4bit = False, # Using GGUF quantization instead
)

---

🖼️ Visual Gallery

Performance Visuals: ![](https://unsloth.ai/docs/~gitbook/image?url=https%3A%2F%2F3215535692-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Fspaces%252FxhOjnexMCB3dmuQFQ2Zq%252Fuploads%252FA6YPmwKLaJ7IZ4U4bQE3%252Fimage.png%3Falt%3Dmedia%26token%3De003bf7b-5c78-4eb4-bbae-5358410ef3fa&width=768&dpr