A Retrospective Look at Reverse-Engineering Apple's Neural Engine

By Eileen Yoon | Published August 10, 2026

I walked away from my work on the reverse-engineered Apple Neural Engine (ANE) driver about three years ago. I hit a depressing realization: the ANE block simply isn't that versatile. I decided my time was better spent upstreaming other, more impactful hardware blocks.

The fundamental issue was that the ANE's architecture is far too general-purpose opinionated. Creating a flexible accelerator platform around it was a losing battle; even a Linux driver that provided raw API access to the hardware wouldn't significantly expand the variety of workloads it could handle.

In fact, on macOS, the ANE is primarily relegated to mundane tasks, such as generating those upsampled preview thumbnails you see in Finder.

Resources:

• Source Code: https://github.com/eiln/ane/tree/main
• Hardware Visuals:

With the release of the M5 (2025), Apple leaned heavily into "LLM performance" as a marketing pillar. In a telling architectural move, they integrated the ANE cores directly into the GPU cores. I suspected this was coming, but it marks the official beginning of the end sunset of the standalone NPU.

To commemorate the ANE's decline, I've decided to do something entirely impractical: return to the M1 ANE and finish the reverse-engineering process.

It has been three years (fuck), and I am hopefully coming back to this with a deeper level of expertise.

The New Objective

Three years ago, the goal was utility: make the ANE run operations. Now, the goal is purely academic. I want to map the entire internal landscape:

• Compute: How the math is actually performed.
• Datapath: How data moves through the system.
• Scheduler: How tasks are orchestrated.
• Memory: How data is stored and retrieved.
• Execution Model: The overall logic of operation.

By dissecting these design choices, we can uncover the assumptions Apple made about Machine Learning (ML) workloads back when the A11 Bionic (2017) first introduced this silicon. It provides a window into the transition from the CNN-era (Convolutional Neural Networks) to the current Transformer-era dominated by GPUs.

---

The Compute Architecture

The 16 compute cores are arguably the most straightforward part of the ANE. Apple's original target was dense image-processing CNNs, which rely on dense tensor reductions and highly predictable data reuse.

While the M1 ANE utilizes a massive array of multiply-accumulate (MAC) units, the MACs themselves aren't what make it specialized. Whether you are performing a convolutional layer (dot product of an activation window and kernel weights) or an attention mechanism (dot product of query and key vectors), the basic math is the same.

The "secret sauce" for the 2017-era CNNs was the dataflow—the specific logic governing when and where inputs and outputs move.

The Transformer Conflict: Transformers, particularly during autoregressive decoding, broke the assumption of predictable reuse. The ANE's dataflow was optimized for phones, which meant it was too rigid for the shifting needs of modern LLMs. The M5's design confirms that while the MAC units are still useful for Transformers, they require a completely different dataflow environment (hence the move into the GPU).

The Internal Datapath

Each of the 16 compute cores follows a specific internal structure:

Multiply-Accumulate (MAC) Details

The ANE features 16 parallel cores. Each core contains:

• 128 FP16 lanes OR
• 256 INT8 lanes.

Each lane executes the following recurrence: $s \leftarrow s + a \times b$

By repeating this over $T$ cycles, the hardware computes a $T$-term dot product: $s_T = s_0 + \sum_{t=0}^{T-1} a_t b_t$

Across the entire 16-core array, we have: $128 \text{ lanes/core} \times 16 \text{ cores} = 2048 \text{ parallel MAC lanes}$

This allows for 2048 parallel reductions spatially per cycle, with time being the only axis of reduction: $S_T[q,p] = S_0[q,p] + \sum_{t=0}^{T-1} a_t[q,p]b_t[q]$

It is crucial to understand that the MAC lane is "dumb"—it doesn't know if it's part of a matrix multiplication or a convolution. That logic is handled by how operands are mapped and scheduled onto the core.

Precision and Saturation

The datapath consists of a multiplier, an adder, and a 32-bit accumulator register. The feedback loop ensures the partial sum stays local to the MAC lane, avoiding expensive external memory fetches.

• Reduction: Fixed-point.
• Readout: FP16.
• Internal Logic: 16-bit multiplier $\rightarrow$ 32-bit Q16.16 accumulator $\rightarrow$ sign-extension/conversion $\rightarrow$ FP16.

To test the accumulator's limits, I created a CoreML program using a vector of all (1)s. This caused the running sum to grow steadily: $s = \sum_{i=0}^{255} v = 256v$

Observation Table: Accumulator Range

Because 0x7800 is a valid FP16 word, the 0x7c00 result indicates that the overflow isn't happening at the FP16 output stage, but rather inside the accumulator at $2^{15}$. Therefore, the accumulator saturates at $2^{15}$, matching the range of a signed 32-bit fixed-point value with 16 fractional bits.

Nonlinear Activation

For fused layers, the ANE performs the following operation: $y = f\left(\sum_k x_k w_k + b\right)$

In this pipeline, the completed MAC sums are fed directly into the... [Article ends here]