An engineer has published detailed technical analysis of Apple’s Neural Engine (ANE), the dedicated ML accelerator that debuted in the A11 Bionic in 2017. The work represents a completion of reverse-engineering efforts abandoned three years earlier, motivated partly by what the engineer describes as the ANE’s apparent end-of-life following Apple’s M5 announcement.
According to the analysis, the ANE was originally designed around dense image-processing CNN workloads with predictable data reuse patterns. Each ANE contains 16 compute cores, with each core housing 128 FP16 (or 256 INT8) parallel multiply-accumulate (MAC) lanes—totaling 2,048 parallel MAC lanes across the entire unit. The core function of each MAC lane is a recurring operation: multiply two operands and add the product to a running sum (accumulator).
The engineer notes that the MAC operation itself is generic; what specialized the ANE for CNNs was not the compute unit but the surrounding dataflow architecture—specifically how inputs and outputs enter, move through, and exit the system. This dataflow was optimized for the predictable reuse patterns inherent in convolutional layers.
The reverse-engineering work also examines the ANE’s activation functions. By compiling CoreML models and inspecting hardware registers, the engineer determined that the ANE implements nonlinear activations using lookup tables. For tanh activation, the hardware contains 33 quantized samples covering the range [0,4] with a spacing of 1/8. The activation block fuses directly with MAC operations, avoiding an intermediate memory round-trip by applying pointwise nonlinear functions immediately after scalar reductions complete.
On accumulator precision, testing reveals the ANE performs 16-bit fixed-point reduction, accumulating in a 32-bit register as Q16.16 format, then reading out as FP16. The accumulator saturates at 2^15, the range of a signed 32-bit fixed-point value with 16 fractional bits.
The engineer observes that while the M5 (2025) highlighted “LLM performance,” Apple has now folded ANE cores into GPU cores. This architectural shift reflects a fundamental change in ML workload assumptions: transformers, particularly during autoregressive decoding, lack the predictable reuse patterns that made the standalone ANE efficient for CNN-era models. The analysis suggests this consolidation marks the beginning of the end for dedicated NPU designs in Apple silicon.
Key facts
- The ANE contains 16 compute cores, each with 128 FP16 or 256 INT8 MAC lanes, totaling 2,048 parallel lanes
- The ANE was designed for CNN workloads with predictable data reuse; transformer workloads broke those reuse assumptions
- Activation functions use lookup tables (33-point table for tanh covering [0,4])
- Accumulators use Q16.16 fixed-point format and saturate at 2^15
- Apple’s M5 integrates ANE functionality into GPU cores rather than maintaining separate NPU hardware