From the Architect of Redis: Local LLM Execution via ds4

DwarfStar 4 (ds4) is a specialized inference engine written in C, designed specifically for high-memory hardware utilizing Mac (Metal), CUDA, and ROCm. It enables the local execution of frontier open-weights models, bridging the gap between massive cloud-based MoEs and local hardware.

🚀 Supported Model Ecosystem

The engine is optimized for a specific set of high-performance layouts:

• DeepSeek: V4 and V4.1 Flash
• GLM: 5.x series
• Qwen: 3.8 Flash Next
• Capabilities: Supports both text and vision modalities, integrated local APIs, a command-line interface, and a built-in native agent.

---

🛠 The Evolution of the Stack

The development of ds4 followed a three-stage conceptual pipeline:

1. The Giant: Starting with the massive $\approx 284\text{B}$ parameter scale of DeepSeek V4 Flash. While most users rely on remote APIs, ds4 was built to challenge that constraint.
2. The Collapse: Implementing Asymmetric Quantization. This process compresses the routed experts to save space without lobotomizing sacrificing the critical shared paths of the model.
3. The Dwarf Star: The final result—a dense, resident local engine. It provides a unified state and cache shared across a CLI, HTTP APIs, and a native agent.

Design Philosophy: ds4 is not a generic GGUF runner. It is narrow on purpose, focusing on a limited set of model families to ensure end-to-end validation of every supported layout.

---

🧠 Core Technical Innovations

01. Asymmetric 2-bit Quantization

To fit routed-MoE (Mixture of Experts) builds onto consumer or prosumer hardware, ds4 employs a precision-splitting strategy:

• Routed Experts: Highly compressed (2-bit).
• Critical Paths: Maintained at higher precision to preserve intelligence.

02. Persistent KV Cache

The engine treats the KV cache as a "disk citizen."

• Mechanism: The rendered prompt prefix is hashed using SHA1.
• Persistence: This hash is used as a key to save the cache to the SSD.
• Benefit: Upon restart, if a matching prefix is detected, the engine reloads the cache from disk rather than recomputing it.

03. Unified Interface Triad

A single engine powers three distinct interaction modes:

---

🏗 System Architecture

The following diagram illustrates how the ds4 components interact:

---

💻 Deployment Guide

Quickstart Checklist

• Clone the repository
• Fetch model weights
• Compile for specific backend
• Launch interface

Execution Steps

# Step 1: Fetch the weights
git clone https://github.com/antirez/ds4
cd ds4 && ./download_model.sh ds4f-q2

# Step 2: Build for your hardware
make              # For Apple Silicon (Metal)
make cuda-spark   # For Linux (NVIDIA DGX Spark)

# Step 3: Initialize
./ds4                             # Start interactive chat
./ds4-server --ctx 100000         # Start API server with 100k context

---

🖥 Hardware Compatibility Matrix

Depending on your available VRAM/RAM, different execution modes (local, streamed, or distributed) apply.

• Baseline: V4 Flash Q2 is the minimum requirement.
• Mid-Tier (128GB): Supports GLM 5.3 Q2 and Qwen Q4; V4.1 Q2 utilizes SSD streaming.
• High-Tier (512GB+): Provides headroom for V4.1 Q4 or PRO-class configurations.

---

📊 Performance Benchmarks

Metrics are divided between Prefill (initial processing) and Generation (token output), which is critical for agentic workflows.

Reference Hardware: M5 Max (128GB) and DGX Spark (128GB) using q2 quantization.

---

🤖 Integration with AI Agents

Because ds4-server implements standard API schemas, it can act as the backend for professional coding tools. By pointing the base URL to your local machine, you can use:

• OpenCode
• Claude Code
• Codex CLI

Supported endpoints include: /v1/chat/completions $\rightarrow$ /v1/messages $\rightarrow$ /v1/responses

Final Word: Take control of your inference. Use the quickstart, verify your hardware fit, and connect your favorite editor or agent to your own local silicon.