Cache stampede

If sufficiently high load is present, this may by itself be enough to bring about congestion collapse of the system via exhausting shared resources.

Congestion collapse results in preventing the page from ever being completely re-rendered and re-cached, as every attempt to do so times out.

In typical web applications, the function recompute_value() may query a database, access other services, or perform some complicated operation (which is why this particular computation is being cached in the first place).

When the request rate is high, the database (or any other shared resource) will suffer from an overload of requests/queries, which may in turn cause a system collapse.

The following implementation based on an exponential distribution has been shown to be optimal in terms of its effectiveness in preventing stampedes and how early recomputations can happen.

This approach is simple to implement and effectively reduces cache stampedes by automatically favoring early recomputations when the traffic rate increases.