Engineering a GPU-Accelerated Backend for Emacs

A few months ago, I became preoccupied with a seemingly trivial question: why does Emacs rely entirely on the CPU to render text, even when running on hardware with a powerful GPU?

This curiosity spiraled into further inquiries, such as: Why is it impossible to play a video directly within a buffer?

I eventually realized that every single scroll, underline, and glyph is recalculated and repainted by the processor. The core of this is the redisplay engine, found in xdisp.c. This engine was designed for an era where GPUs weren't an option, and it is meticulously optimized for CPU rendering. Until now, no one had successfully integrated a GPU without essentially rewriting half of the editor.

What started as a casual weekend project evolved into a full-scale display backend. The result included:

• A Metal backend for macOS.
• An OpenGL backend for GNU/Linux.
• In-buffer video playback.
• Shader-driven cursor animations.
• A massive debate (100+ messages) on the Emacs developers' mailing list covering everything from cairo performance to the ethics of AI and software freedom.

I am writing this to document the journey and provide a roadmap for others. I'll conclude with the lessons learned and a surprising realization.

A Note on Transparency: I utilized an LLM as a copilot throughout the entire development process. This turned out to be the most significant plot twist of the project.

---

Phase 1: Architectural Strategy

The intuitive first step would be to modify the Cocoa backend (nsterm.m) and swap CoreGraphics calls for Metal calls. However, that approach is flawed because it creates platform lock-in. If I built "Emacs with Metal," I would only have a Mac version.

I needed a display-backend abstraction that supported multiple drivers. I sketched out a three-tier architecture:

The Layer Breakdown

The goal was to keep all drawing logic—such as how glyph strings are composed, how wavy underlines are placed, and how images are clipped—inside a plain-C file. The drivers only handle primitives like upload texture or draw quad.

The Golden Rule: xdisp.c must remain untouched. By hooking into the existing drawing interface while letting the redisplay engine compute glyph matrices as usual, I avoided the need for a massive rewrite.

---

Phase 2: Metal and the Quest for Pixel Perfection

Once the architecture was set, I focused on the Metal implementation. I followed the standard modern text rendering pipeline:

1. Rasterization: Use CoreText to render each glyph once into a grayscale texture (an R8 format glyph atlas).
2. Composition: Assemble the frame on the GPU using a persistent texture.
3. Presentation: Display the final frame.

My objective was for the output to be identical, pixel for pixel, to the original Cocoa backend. To ensure this, I built a Python-based testing harness using the PIL library that launched two instances of Emacs and compared screen captures.

The acceptable margin of error was: $\text{Difference} \le 0.055\%$

This rigorous process revealed several subtle bugs that required extreme scrutiny:

• Ink Weight: The thickness of the characters was slightly off.
• Relief Colors: The 3D effects on buttons and the mode-line were incorrect.
• Vertical Alignment: There was a persistent off-by-one error in glyph positioning.

---

Phase 3: The Frozen Cursor

The most educational bug involved the cursor. I wanted "visual candy"—shader-based effects like a comet trail or an expanding ring when the cursor moves. I implemented these as a compositor layer above the buffer.

However, I noticed a strange behavior: the animation would freeze the moment I stopped typing.

The culprit was CADisplayLink. In the macOS ecosystem, this synchronization mechanism "dies at rest"; it requires the Emacs event loop to feed it. When there was no user input, the clock stopped.

To fix this, I shifted the timing logic from the system to an Emacs Lisp timer. I created a single "pump" in Lisp that periodically signals the driver to advance animations and present the frame.

;; Conceptual logic for the animation pump
(let ((refresh-rate (if (animation-active-p) 60 30)))
  (run-with-timer 
   (/ 1.0 refresh-rate) 
   nil 
   (lambda () (gpu-driver-pump-frame))))

I eventually implemented auto-pacing:

• $60\text{ Hz}$ during active fades/animations.
• $30\text{ Hz}$ during idle periods.
• $0\text{ Hz}$ (off) when nothing is animating.

Final Feature Checklist

By the end, the backend supported:

• Standard text and decorations
• Images and animated GIFs
• Line numbers and custom bitmap fringes
• Mode-line, header-line, and tab-bar
• Retina/HiDPI (2x scaling)
• All four cursor types
• Window splits and dynamic text-scale

---

Visual Gallery