LRU vs GDSF vs Recency-Weighted Eviction for Long-Context KV Cache
Three eviction policies battle for control of shrinking KV cache memory.
Omar Ford
Staff Writer
Omar Ford
Omar Ford is a staff writer at Inference Engineering covering kv cache systems. Based in New York, Omar has written for Inference Engineering since 2021.
1 story · New York