tierKV: Shattering the LLM Memory Wall with Distributed KV Caching

tierKV bypasses the quadratic cost of attention recomputation by offloading evicted KV blocks to a high-speed LAN vault, achieving restoration speeds that outperform local GPU cache hits. This architecture enables near-instantaneous processing for long-context prompts, offering a scalable solution for VRAM-constrained inference environments.