HILOS validates near-storage attention on 16 SmartSSDs

HILOS offloads attention to near-storage accelerators and combines cooperative caching, delayed KV writeback, and a memory-efficient attention engine, evaluated on a real 16-SmartSSD system.

Why it matters

It supplies real-system evidence for moving KV-cache computation toward flash and explicitly measures write-latency and write-amplification concerns that HBF proposals must also confront.

Exact source title

A Cost-Effective Near-Storage Processing Solution for Offline Inference of Long-Context LLMs

Authors

Hongsun Jang, Jaeyong Song, Changmin Shin, Si Ung Noh, Jaewon Jung, Jisung Park, Jinho Lee

HILOS validates near-storage attention on 16 SmartSSDs

It supplies real-system evidence for moving KV-cache computation toward flash and explicitly measures write-latency and write-amplification concerns that HBF proposals must also confront.

Related records