The cache that fixes your latency and creates three new problems

A page is slow, so someone adds a cache. The page gets fast, everyone is pleased, and the system acquires three new failure modes that will not present for months: users seeing stale data after an update, a thundering herd when a popular key expires, and an eviction policy nobody chose that quietly starts discarding session data under memory pressure.

Caching is worth doing. It is also the component most often added without deciding anything, and the decisions are what determine whether it helps.

First, check whether the query was the problem

The honest first step, and the one that is skipped.

A cache in front of a query doing a sequential scan on a large table hides the problem at the cost of correctness risk. Adding the index makes the uncached query fast enough that the cache may be unnecessary, and it helps every other query touching that table. Pull the slow query list first, as described in tuning managed PostgreSQL.

Cache when the work is genuinely expensive and genuinely repeated: an aggregation over a large dataset, a call to a slow third-party API, a rendered fragment assembled from several sources, a computed permission set. Not when the work is a badly written query.

Invalidation, and why a time to live is usually right

There are three approaches and the industry has largely converged on the least clever one.

Expiry by time to live. Set a duration, accept that data can be stale for that long, move on. It is self-healing: a bug that leaves a wrong value in the cache fixes itself, and nothing has to be coordinated. This is the correct default for the overwhelming majority of cases, and the design work is choosing the duration from how stale the data may acceptably be, not from how long it takes to compute.

Explicit invalidation on write. Delete or update the key when the underlying data changes. Precise, and it requires every write path to know about every cache key derived from that data. It works well when one service owns both. It fails when a second service, a batch job or a manual database fix changes the data without going through the write path, and that failure is silent.

Write-through. Update the cache and the store together. Consistent within one process and does not survive a partial failure between the two writes without more machinery than most teams want.

Our default: time to live everywhere, plus explicit invalidation on the small number of keys where staleness is genuinely unacceptable, and never explicit invalidation alone.

Two supporting habits. Version the key when the shape of the cached value changes, rather than trying to purge, because a deploy that changes a serialisation format against a populated cache produces errors that are hard to attribute. And add jitter to the duration so that keys populated together do not all expire together.

The stampede, and the three fixes

A popular key expires. Two hundred concurrent requests miss, all of them run the expensive computation, and the database that the cache was protecting receives two hundred simultaneous queries. This is the cache making the outage worse rather than preventing it, and it happens at the worst moment by definition.

Three mitigations, in increasing order of effort:

• Jitter on the expiry, which spreads the misses but does not help when one key is genuinely hot.
• A lock so only one request recomputes, while the others wait briefly or serve the stale value. The stale-while-revalidate shape is usually the best behaviour: return the old value immediately and refresh in the background.
• Proactive refresh of known-hot keys before they expire, which requires knowing which keys are hot.

Implement at least jitter and the single-flight lock on anything expensive. This is not a theoretical problem; it is the second most common cache-related incident after staleness.

Redis or Valkey, after the licence change

Redis changed its licence away from a permissive open source model, which prompted the Linux Foundation to adopt a fork, Valkey, continuing under the original permissive terms. Redis has since also made a permissive licence available again for recent versions, so the situation is less clear-cut than it was during the initial split.

For most teams this changes very little. The protocol is the same, clients work with both, and the cloud providers offer managed services for one or both, with several having moved their managed offering to the fork.

Decide on the same basis as the equivalent infrastructure question in OpenTofu or Terraform: if you use a managed service, use what your provider offers and do not manufacture a problem. If you self-host and licence terms matter to your organisation or your product, the fork removes the question. If you depend on specific Redis modules, check availability before switching, because that is where the divergence actually bites.

Memory, eviction and the day it fills

A cache has bounded memory, and what happens at the boundary is a policy you either chose or inherited.

Set the eviction policy deliberately. Evicting the least recently used key among those with an expiry set is the sensible default for a cache. A policy that never evicts returns errors on write when memory is full, which turns a cache into an outage. A policy that evicts any key, including ones without an expiry, will happily discard the session data you also put in there.

Which raises the more important point: do not mix a cache and a durable store in one instance. If some keys must not be lost, they belong somewhere with different persistence guarantees, or at minimum in a separate instance with a different policy.

Monitor the hit rate, evictions and memory fragmentation. A hit rate that is low means the cache is costing latency and money without helping, and a cache with a high eviction rate is thrashing, which is worse than not caching at all.

What not to cache

• Data the user just wrote. Read-after-write through a cache produces the same class of bug as replica lag.
• Anything personal without thinking about retention. A cache holding personal data is a datastore for privacy purposes, with the deletion obligations described in GDPR as engineering controls. A time to live is at least a retention rule, which is one reason to prefer it.
• Permission decisions, for long. A cached authorisation that outlives a revoked access is a security finding.
• Values that are cheap to compute. The network round trip to the cache is not free, and for a fast query it can be slower than the query.

The things people forget

• The cache is a dependency now. What happens when it is unreachable? The correct answer is usually to degrade to the source, with a circuit breaker so a slow cache does not add latency to every request.
• Serialisation is a compatibility surface. Two application versions reading each other's cached objects during a rolling deploy is a real failure mode.
• A single instance is a single point of failure. Decide deliberately whether you need replication, and remember that failover takes time during which the source sees full load.
• Connection limits apply here too. A serverless function fanning out opens connections to the cache as well as to the database.
• Key expiry is not a scheduler. Relying on expiry callbacks for business logic is fragile.

What to do this week

Look at your cache's hit rate and eviction rate for the last week. If the hit rate is below about eighty percent, ask what the cache is actually buying you, because you are paying a round trip on every miss. Then pick your most expensive cached computation and check whether a stampede is possible when its key expires. Both take twenty minutes and both usually find something. We look at this during the architecture phase of a cloud engagement.