Qwen 3.8 27B Now Live on Cerebras: Achieving 1500 tokens/s

📚 Documentation Overview

Welcome to the Cerebras Inference Model Catalog. To get a comprehensive view of all available documentation, please access the index file located at /llms.txt. This file serves as the primary map for discovering all available pages within our system.

🚀 Accessing Models

Users can interact with models via our public endpoints through two primary tiers:

1. Free Trial
2. Pay-As-You-Go

Note: These tiers are subject to specific pricing structures and rate limits.

For organizations requiring production-grade SLAs, higher throughput, reserved capacity, or access to additional model families, we recommend exploring our Dedicated Endpoints.

---

🛠️ Model Catalog

Below is the current selection of models available on our public endpoints. Click a model name to view its specific capabilities and tier-based constraints.

[!TIP] Need more variety? Many more model families are accessible exclusively via Dedicated Endpoints.

---

📉 Model Compression & Integrity

Cerebras prioritizes transparency regarding how models are optimized for our hardware.

The Pruning Policy

We maintain a strict policy regarding model architecture:

• Public endpoints do not host pruned models.
• Every model served via the shared API is the original, unpruned version.

While we actively research REAP (Router-weighted Expert Activation Pruning), these experimental versions are shared with the research community via Hugging Face and are not integrated into our production API.

Quantization Strategy

To balance storage efficiency with output quality, we employ selective weight-only quantization.

Storage Logic: Weights are stored using a mix of 16-bit, 8-bit, and 4-bit formats, adhering to standard industry practices.

Execution Logic: To ensure maximum precision, we utilize a "dequantization on the fly" approach:

• Sensitive Layers: Stored at full precision or dequantized immediately before operation.
• Core Components: The following remain entirely unquantized and in full precision:
  • Activations
  • Attention mechanisms
  • KV Cache

Compression Workflow

---

❓ Frequently Asked Questions

Will you modify a model's architecture without telling me?

No. We are committed to providing the original architectures for all existing endpoints. We do not apply pruning to our hosted production portfolio. Should we introduce compression techniques like pruning in the future, they will be deployed as distinct endpoints with unique names to ensure total transparency.

Research Resources

If you are interested in our pruning experiments, you can find the Cerebras REAP Collection on Hugging Face. These are for experimentation only and are not part of the production API.

Understanding Compression Terms

To clarify the terminology used in our documentation:

• Quantization: The process of reducing the numerical precision of weights (e.g., $FP16 \rightarrow FP8$).
  • Result: Lower memory footprint; no change to architecture.
• Pruning: The permanent removal of specific components, such as experts or layers.
  • Result: Smaller model size; changes the underlying architecture.

---

🏁 Getting Started Checklist

• Review the /llms.txt index.
• Select a model ID (e.g., qwen-3.8-27b).
• Choose between Free Trial or Pay-As-You-Go.
• (Optional) Request a Dedicated Endpoint for production SLAs.

Example Model Configuration

{
  "model": "qwen-3.8-27b",
  "parameters": {
    "context_limit": 128000,
    "precision": "high-precision-activations"
  },
  "performance": {
    "target_speed": "1500t/s"
  }
}