Show HN: Running PrismML's Bonsai inside DRAM by breaking DDR4 timing rules
Show HN: Executing PrismML's Bonsai within DRAM via DDR4 Timing Violations
The traditional computing architecture is plagued by the von Neumann bottleneck, where the constant shuttling of data between the CPU and RAM consumes the vast majority of energy and time. To combat this, the team behind PrismML has developed Bonsai, a system that shifts computation directly into the memory chips.
The secret? They aren't just using the memory; they are following the rules intentionally breaking DDR4 timing specifications to achieve Processing-in-Memory (PIM).
The Core Concept: Processing-in-Memory (PIM)
Most systems treat DRAM as a passive warehouse. Bonsai transforms it into an active processor. By manipulating how rows are activated and closed, the authors can perform logic operations without ever sending data across the memory bus to the processor.
"The goal is to stop moving the data to the code and start moving the code to the data."
The "Hack": RowClone and Timing Rules
Standard DDR4 memory relies on strict timing parameters to ensure data integrity. By violating these, specifically the intervals between activating rows, the authors implement a technique known as RowClone.
| Parameter | Definition | Standard Behavior | Bonsai's Approach |
|---|---|---|---|
| RAS to CAS Delay | Wait for row to open | Minimized/Ignored | |
| Row Precharge Time | Wait for row to close | Aggressively shortened | |
| Row Active Time | Minimum time row stays open | Manipulated for charge leakage |
How RowClone Works
Instead of the standard Read $\rightarrow$ Bus $\rightarrow$ Write cycle, RowClone allows a direct copy from one row to another within the same subarray.
From Row Copies to Neural Networks
Once you can clone a row, you can build logic. By carefully timing the activation of three rows, you can create a "charge-sharing" effect that mimics boolean logic.
The Logic Hierarchy
- RowClone: The primitive for moving data.
- Logic Gates: Using RowClone to implement , , and operations.
- Bonsai: A specialized ML framework that maps neural network weights and activations to these DRAM-based gates.
The mathematical representation of the charge transfer can be simplified as: Where represents the voltage level (binary 0 or 1) and represents the capacitance of the memory cells.
Technical Implementation
To achieve this, the authors had to bypass the standard memory controller's safety guards. This required writing custom drivers to send raw commands to the memory bus.
// Conceptual snippet of a timing-violation sequence
void execute_rowclone(uint64_t src_row, uint64_t dst_row) {
send_cmd(ACTIVATE, src_row);
// Violation: No t_RP wait here
send_cmd(ACTIVATE, dst_row);
// The charge leaks from src to dst due to the timing overlap
send_cmd(PRECHARGE_ALL);
}
Figure 1: Conceptual layout of the Bonsai logic mapping within a DRAM bank.
Results and Trade-offs
The performance gains are staggering in terms of energy efficiency because the power consumption is nearly eliminated. However, this comes with significant risks.
The Pros and Cons
- Advantages:
- Massive internal bandwidth.
- complexity for certain bulk data movements.
- Extreme energy reduction.
- Risks:
- Hardware Stress: Breaking timing rules can lead to cell degradation.
- Instability: Slight temperature changes can alter the requirements.
- Compatibility: Only works on specific DDR4 modules.
Project Roadmap
- Implement RowClone primitives.
- Map basic boolean logic to DRAM rows.
- Successfully run a small-scale Bonsai ML model.
- Optimize for different DRAM vendors.
- Develop a compiler to automate the mapping of PyTorch models to DRAM.
Final Thoughts
By treating the DDR4 specification as a "suggestion" rather than a law, PrismML's Bonsai demonstrates that the hardware we already own is capable of far more than we are told. It turns the memory array into a massive, parallel computational engine, effectively killing the von Neumann bottleneck distance between storage and logic.