Micron's memory wall chart. Compute up ~3x every two years, HBM bandwidth under 2x

A Micron memory wall chart, presented at Hot Chips 2026, illustrates a growing disparity between compute power and HBM bandwidth. While compute (TPU v3 through R200) increases approximately 3x every two years, HBM bandwidth (HBM2e through HBM4) grows by less than 2x. Solutions being implemented include placing memory closer to compute, using shorter links, and integrating processing units within memory, as seen in Samsung's LPDDR5X, which achieved a 3.01x token increase on Llama 3.1 8B.

This report uniquely details the specific solutions being implemented, including Samsung's LPDDR5X achieving a 3.01x token increase on Llama 3.1 8B, unlike other general discussions.