TurboFieldfare: Running Gemma 4 26B on M-Series Macs with Minimal RAM

TurboFieldfare is an open-source inference engine designed to run the Gemma 4 26B-A4B model using only $\approx 2\text{ GB}$ of RAM. This allows the model to operate on any Apple Silicon Mac, including entry-level machines with only 8 GB of total system memory.

"Memory got expensive."

Unlike traditional methods, TurboFieldfare ~~loads the entire model into memory~~ avoids loading the full 14.3 GB weight set. Instead, it maintains a shared core of 1.35 GB and an FP16 KV cache in RAM, while dynamically streaming only the specific experts required for each token directly from the SSD.

🛠 Technical Architecture

This is a bespoke runtime developed using Swift and Metal. It is not a wrapper for existing frameworks like llama.cpp or MLX. The project is backed by a detailed record of 103 experiments covering I/O, prefill, decode, caching, and kernels.

📊 At a Glance: Specifications

Metric	Detail
Model	Gemma 4 26B-A4B IT (Total: 26B params; Active: $\approx 3.88\text{B}$ per token)
Quantization	MLX affine 4-bit (group 64); 8-bit router; 4-bit experts/shared weights
Memory Footprint	$\approx 2\text{ GB}$ (Weights + 4K KV cache)
Disk Space	$\approx 14.3\text{ GB}$ (Text-only installation)
OS/Hardware	macOS 26, Metal 4, Swift 6.2 on Apple Silicon
M2 Performance	$5.1 \text{--} 6.3\text{ tok/s}$ (on 8 GB MacBook Air)
M5 Performance	$31 \text{--} 35\text{ tok/s}$ (on 24 GB M5 Pro)

Note: These figures are reference benchmarks. Actual throughput varies based on hardware, page-cache state, and prompt/generation lengths.

🚀 Getting Started

To get the engine running on your machine, follow these steps:

Clone the repository
Build the release
Launch the Mac App
Download and Load the model

Installation Commands

git clone https://github.com/drumih/turbo-fieldfare.git
cd turbo-fieldfare
swift build -c release
.build/release/TurboFieldfareMac

Note: Upon the first execution, the Swift Package Manager will automatically retrieve and compile the necessary tokenizer packages.

📦 Software Components

The Swift package is comprised of six distinct products:

TurboFieldfare: The core library containing Metal kernels and the runtime.
TurboFieldfareMac: The primary GUI application for installation and chat.
TurboFieldfareDecodeService: The backend executable that manages the model and Metal.
TurboFieldfareCLI: A command-line tool for raw completions and chat.
TurboFieldfareServer: An experimental loopback server compatible with the OpenAI API.
TurboFieldfareRepack: A utility for streaming installation and verification.

⚙️ Requirements & Usage

System Prerequisites

Hardware: Apple Silicon Mac (Validated on 8 GB M2 Air).
Software: macOS 26, Metal 4, Xcode 26, and Swift 6.2+.
Storage: $\approx 14.3\text{ GB}$ of free space.
Connectivity: Internet access for the initial model fetch.
Architecture: arm64 only.

Prompting & Generation

The Mac app automatically handles the specific chat formatting required by Gemma. Users simply provide instructions and context.

Default Hyperparameters:

temperature: $0.2$
Top-K: $64$
Top-P: $0.95$

Warning: The model may still produce hallucinations or repetitive text; always verify critical output.

Model Installation Process

When you select Download in the app, TurboFieldfare performs a "streaming repack":

It requests specific byte ranges from a pinned Hugging Face revision.
It converts these ranges directly into the .gturbo format on the fly.
This prevents the need for a temporary full checkpoint, keeping disk and memory usage bounded.
The installation is only finalized after the manifest and file hashes are validated.

The app stores the resulting model in scratch/gemma4.gturbo when run from the local checkout. The interface provides real-time feedback on memory usage, decode speed, and generation progress.