At Hot Chips 2026, Samsung presented their progress on Processing-in-Memory (PIM), a technology that embeds compute units directly within LPDDR5X memory chips. The approach aims to overcome a fundamental bottleneck: traditional memory access between DRAM and compute cores suffers from long latency and limited external bandwidth.
Samsung’s LPDDR5X-PIM design places a PIM block at each of the chip’s 16 DRAM banks. Each block contains a multiply-accumulate (MAC) tree, register files, and control logic. Crucially, these PIM blocks access their attached DRAM banks without being constrained by the chip’s external interface bus. This allows them to collectively utilize the chip’s full internal bandwidth of 614 GB/s—compared to just 76.8 GB/s for standard DRAM accesses that can only hit two banks in parallel.
Each PIM block’s MAC array supports low-precision formats and can sustain four INT8 or FP8 operations per data clock cycle (eight per cycle counting double data rate). A single LPDDR5X-PIM chip delivers 2.4 TOPS of throughput. While modest alone, eight chips would provide roughly 9.6 INT8 TOPS—comparable to Intel’s Meteor Lake NPU—though such a configuration would require 128 GB of system memory.
Samsung ingeniously maintains compatibility with standard LPDDR5X protocols by repurposing special row addresses as control registers. Software can activate reserved rows to switch the chip into multi-bank mode, then use standard read and write commands to access PIM registers instead of DRAM contents. After loading model weights into DRAM in normal mode, software primes the PIM blocks with activation vectors and scale factors, issues compute commands, and reads back results.
The implementation includes an Address Align Mode to handle memory controller reordering, ensuring instructions correctly reference source register elements even when accesses are reordered.
However, Samsung’s elegant hardware design creates steep software challenges. Because PIM modes fundamentally change how DRAM commands are interpreted, the chip cannot safely execute PIM and regular memory operations simultaneously—even across different threads. A non-PIM thread reading memory during PIM computation could trigger unintended calculations or corrupt results. Samsung requires isolating PIM regions to specific memory channels, but this approach sacrifices normal bandwidth interleaving and fragments available bandwidth between PIM and non-PIM applications.
Multithreading and multitasking further complicate deployment. Applications using PIM would need locks to prevent thread conflicts, while modern multitasking operating systems would struggle to coordinate PIM access across multiple processes without explicit awareness and coordination.
Key facts
- Samsung’s LPDDR5X-PIM embeds MAC compute units in each of 16 DRAM banks, achieving 614 GB/s internal bandwidth vs. 76.8 GB/s for standard DRAM
- A single chip delivers 2.4 TOPS; eight chips together provide ~9.6 INT8 TOPS, matching Intel Meteor Lake’s NPU but requiring 128 GB of memory
- The design maintains standard LPDDR5X protocol compatibility by repurposing special row addresses as control registers for PIM modes
- PIM cannot safely run simultaneously with regular memory operations due to command interpretation conflicts, requiring isolated memory regions and sacrificing bandwidth interleaving