How I Built a Two-Level Cache to Serve Millions of Lookups in Under a Millisecond

The team replaced single-layer caching with a two-level strategy using Caffeine as an in-process L1 cache (sub-0.1ms reads) and Redis as a distributed L2 cache, with Elasticsearch as the source of truth. Caffeine absorbs hot-key traffic with short TTLs and W-TinyLFU eviction, while Redis bridges shared state across instances. This architecture resolved tail latencies that had crept past 80ms in their high-traffic product lookup service.