Caching Strategies & Eviction

A Cache is a high-speed data storage layer which stores a subset of data, typically transient in nature, so that future requests for that data are served faster than accessing the data's primary storage location (the database). Caching is one of the single most impactful techniques for improving system performance.

1. Why Caching?

A database query might take 50-200ms (involving disk reads, SQL parsing, and network round trips). A cache lookup in Redis or Memcached takes 0.1-1ms (pure in-memory access). That is a 100x to 1000x speedup.

If 80% of your application's traffic is reading the same popular data (the homepage, trending posts, a product catalog), caching that data in memory eliminates millions of unnecessary database queries per second.

2. Cache Write Strategies

How does data get into the cache in the first place?

Cache-Aside (Lazy Loading)

The application manages the cache manually.

1. Read: App checks the cache. If found (Cache Hit), return it. If not found (Cache Miss), query the database, store the result in the cache, then return it.
2. Write: App writes directly to the database. The cache entry is either invalidated or updated.

• Pros: Only requested data is cached (no wasted memory). Cache failures don't break the app.
• Cons: Cache Miss penalty (first request for any data item is always slow).

Write-Through

Every write goes to the cache AND the database simultaneously.

• Pros: Cache is always consistent with the database. No stale data.
• Cons: Higher write latency (every write operation waits for both the cache and database to confirm). Writes data to cache that may never be read.

Write-Behind (Write-Back)

The application writes only to the cache. The cache asynchronously writes to the database in the background (in batches).

• Pros: Extremely fast writes. Database is not a bottleneck for write-heavy workloads.
• Cons: Risk of data loss if the cache crashes before flushing data to the database.

3. Cache Eviction Policies

Caches have limited memory. When the cache is full and a new item needs to be stored, an old item must be evicted. The eviction policy determines which item to remove.

• LRU (Least Recently Used): Evicts the item that has not been accessed for the longest time. The most commonly used policy.
• LFU (Least Frequently Used): Evicts the item that has been accessed the fewest number of times.
• FIFO (First-In, First-Out): Evicts the item that was added to the cache first, regardless of how often it was accessed.
• TTL (Time-To-Live): Each item is given an expiration time. Once it expires, it is automatically removed. Excellent for data that becomes stale (e.g., weather forecasts, stock prices).

4. Where to Cache?

• Client-Side Cache: Browser cache for static assets (CSS, JS, images). HTTP headers like Cache-Control and ETag manage this.
• CDN Cache: Geographically distributed caches for static content.
• Application-Level Cache: In-memory stores like Redis or Memcached sitting between your application servers and your database.
• Database Query Cache: Some databases (like MySQL) have built-in query result caches.