Apple Silicon and macOS VMs: 11–16× Faster LLM Inference with llama.cpp

Published on August 11, 2026, by Francesco Bonacci and Johnny Franks

For those who have followed the evolution of Cua, you'll recall its origins with the Show HN launch of Lume, our specialized macOS virtualization stack.

The Bottleneck: Virtual GPU Capabilities

When running a macOS guest via Apple's Virtualization.framework, the system utilizes a virtual GPU that is backed by the physical Apple GPU of the host. However, in our standard "Tahoe" VM, this virtual device reports a very conservative Metal capability profile.

Because applications query these capabilities to determine which rendering paths and kernels to utilize, llama.cpp was forced to use significantly slower GPU code.

The Solution: We developed a process-scoped compatibility layer that modifies specific capability responses for a single guest process. This "tricks" llama.cpp into selecting more modern, high-performance Metal kernels.

This breakthrough is part of a larger initiative to integrate Lume's virtualization with the local computer-use environments of Cua Driver, as well as the Cua Cloud and Fleets infrastructure. This work is being released as a research project under a permissive license to help the community map out which macOS versions and Apple Silicon chips benefit most from these workloads.

---

Performance Benchmarks

The performance gains were dramatic. Testing was conducted on an M1 Ultra using llama.cpp.

1. TinyLlama 1.1B

• Prompt Processing: $\approx 11.08\times$ faster
• Token Generation: $\approx 16.36\times$ faster (Compared to the stock VM)

2. Google Gemma 4 12B (QAT Q4_0, 6.98 GB)

• Prompt Processing: $7.20\times$ improvement
• Token Generation: $14.54\times$ improvement
• Efficiency: The unlocked VM achieved $99.59\%$ of bare-metal prompt speed and $94.82\%$ of bare-metal generation speed.

3. Meta Muse Glimmer 30B (Q4_K-M GGUF)

Tested in a 64 GiB guest using llama.cpp b10359 (text-only; no Ollama or multimodal projectors):

• 512-token prompt: $7.55\times$ faster
• 128-token generation: $8.87\times$ faster

---

Technical Architecture: Paravirtualization vs. Passthrough

To understand why this is necessary, we must look at how Virtualization.framework handles graphics.

The Paravirtualization Model

The guest OS is presented with a virtual graphics device. The guest sends Metal commands through a specialized driver, which the host then executes on the physical hardware. This is paravirtualization—the host maintains control, and the guest uses a virtualization-aware interface.

The Passthrough Model (Comparison)

This differs from x86 Linux setups using QEMU/KVM, where VFIO can assign a physical PCI device directly to a VM via an IOMMU, granting the guest raw hardware access.

The Capability Gap

In the stock Tahoe VM, the device reported:

• An Apple 5-era family profile.
• A maximum threadgroup memory of only $32\text{ KB}$.
• SIMD-group matrix support: Unavailable.

Because Metal software follows these reported boundaries, llama.cpp defaulted to slower kernels, even though the underlying hardware was capable of more.

---

The Metal Capability Shim

We implemented a small Metal capability shim—a compatibility layer that intercepts API queries within a specific guest process.

What the Shim Changes

The shim modifies two primary values to unlock newer GPU paths:

Implementation Details

To ensure the shim is lean and auditable, we stripped several unnecessary components from the original research hook:

• Private feature-profile hooks
• Clock and timing interposition
• Mesh substitution
• Ray-tracing overrides
• Argument-layout guards
• Pipeline-compilation fallbacks

// Conceptual logic of the shim
if (query == GET_GPU_FAMILY) {
    return APPLE_FAMILY_9; // 1009
} else if (query == GET_MAX_THREADGROUP_MEMORY) {
    return 64 * 1024; // 64 KB
}

Scope and Safety

• Scoped: Changes only affect the injected guest process.
• Path: Workloads still travel through the Virtualization.framework graphics path to the host GPU.
• Exclusions: This is not PCI passthrough, VFIO, or a kernel modification.
• Fallback: If the configuration is missing or malformed, the process simply reverts to the stock capability path.

Summary of Achievements

• Identified Metal capability bottleneck in macOS VMs.
• Developed a process-scoped shim to report Apple Family 9.
• Increased reported threadgroup memory to $64\text{ KB}$.
• Achieved up to $16\times$ speedup in token generation for small LLMs.
• Reached near-native performance for medium-sized LLMs.