DFlash 2: Keep Drafting Parallel

By Inco AI | August 18, 2026

In the current era of AI agents, inference has become the primary bottleneck. Because every single token requires a complete forward pass through the model, the cost and time of generation scale linearly.

At Inco AI, we are engineering an inference stack designed for the token economics of the future.

The Legacy of DFlash

Since its release in January, the original DFlash has seen widespread integration across the industry, now running in:

• SGLang
• vLLM
• TensorRT-LLM
• llama.cpp

Performance Benchmarks

The impact has been significant across various hardware architectures:

The ecosystem is rapidly expanding. NVIDIA, Red Hat, and Modal have published their own DFlash drafters. Furthermore, official drafters are now shipped with models from Meta (Muse Glimmer), Poolside (Laguna), Xiaomi (MiMo-V2.5-Pro), and NVIDIA (Nemotron 3.5 Lightning). As of August 2026, DFlash models have surpassed 3 million 3.5 million downloads on Hugging Face.

---

Evolution of Speculative Decoding

To understand DFlash 2, one must understand the foundation of speculative decoding:

A small "draft" model predicts a sequence of tokens, which the larger "target" model then verifies in a single forward pass. If the guesses are accurate, one pass yields multiple tokens; if they are wrong, the guesses are discarded.

Historically, drafting was autoregressive (one token at a time). DFlash revolutionized this by making the draft process a one-pass operation, predicting the entire block in parallel.

Introducing DFlash 2

DFlash 2 evolves this parallel approach. It increases the output of every verification pass by over 20% while adding only $\approx 1\%$ to cycle latency—all while ensuring the output remains provably identical to the target model.

Current Result: Using the new Qwen3.8-27B drafter in SGLang, throughput is 2.7–3.4$\times$ higher than standard autoregressive decoding (at batch size 1).

---

🚀 Run It Now

DFlash 2 is already compatible with the most popular inference engines.

Deployment Checklist

• Install SGLang, vLLM, or llama.cpp
• Download the incoai/Qwen3.8-27B-DFlash2 drafter
• Configure the speculative algorithm to DFLASH

Implementation Commands

SGLang

pip install sglang[all] @ git+https://github.com/sgl-project/sglang.git#subdirectory=python
python -m sglang.launch_server \
  --model-path Qwen/Qwen3.8-27B \
  --speculative-algorithm DFLASH \
  --speculative-draft-model-path incoai/Qwen3.8-27B-DFlash2 \
  --speculative-num-draft-tokens 8

vLLM

pip install -U vllm @ git+https://github.com/vllm-project/vllm.git@refs/pull/52816/head
vllm serve Qwen/Qwen3.8-27B \
  --speculative-config '{ method : dflash , model : incoai/Qwen3.8-27B-DFlash2 , num_speculative_tokens : 7 }'

llama.cpp

git clone https://github.com/ggml-org/llama.cpp.git
cd llama.cpp
git fetch origin pull/27342/head:pr-27342
git switch pr-27342
# Build for CUDA or Metal
cmake -B build -DCMAKE_BUILD_TYPE=Release -DGGML_CUDA=ON # or -DGGML_METAL=ON
cmake --build build -j
./build/bin/llama-server \
  -hf ggml-org/Qwen3.8-27B-GGUF:Q4_K_M \
  -hfd incoai/Qwen3.8-27B-DFlash2-GGUF:Q4_K_M \
  --spec-type draft-dflash \
  --spec-draft-n-max 7

Ollama

git clone https://github.com/ollama/ollama.git
cd ollama
git fetch origin pull/17865/head:dflash2
git switch dflash2
cmake -B build .
cmake --build build --parallel 8

TARGET=$(hf download mlx-community/Qwen3.8-27B-4bit)
DRAFT=$(hf download incoai/Qwen3.8-27B-DFlash2)
printf "FROM %s\nDRAFT %s\n" $TARGET $DRAFT > Modelfile
./ollama create qwen38-dflash2 --experimental --draft-quantize int4
./ollama serve
# Run with high-thinking mode
./ollama run qwen38-dflash2 --think high

oMLX (Apple Silicon)

1. Install prebuilt oMLX with DFlash 2 support.
2. Download mlx-community/Qwen3.8-27B-4bit and incoai/Qwen3.8-27B-DFlash2.
3. In the Model Manager, set:
   • DFlash: enabled
   • Draft model: incoai/Qwen3.8-27B-DFlash2
   • Draft quantization: enabled
   • Runtime block size: 5
   • Verify mode: dflash

---

The Logic: Finding the Right Tokens

The challenge with parallel prediction is that positions are predicted independently. This often leads to "incoherent" blocks (e.g., repeating the same word) that the target model rejects.

While other methods (like Domino or DSpark) use sequential heads to fix this, Inco AI found that the correct tokens are already present in the DFlash candidate lists.

The Recall Gap

If we look at the first position, the top pick is correct $85.4\%$ of the time. However, the correct token is present in the top 16 candidates $99.5\%$ of the time.

Table 1: Recall Analysis

Note: Acceptance length is conditioned on all previous positions being correct.

An "oracle" picking the best of the top 16 would increase the average acceptance length from $4.27$ to $6.79$. The goal of DFlash 2 is to find that optimal path.

Visualizing the Path Selector

By implementing a Lightweight Path Selector, DFlash 2 ensures that local coherence is maintained without reverting to slow, sequential processing.