A broader map of the LLM inference memory problem

A survey and research agenda for inference hardware that situates emerging memory tiers, including HBF, inside the wider model-serving bottleneck.

Why it matters

It provides neutral system context for deciding where HBF might help and where compute, networking, or software remains the real constraint.

Exact source title

Challenges and Research Directions for Large Language Model Inference Hardware

Authors

Xiaoyu Ma, David Patterson