Too Much Cache is Like a Krispy Kreme Burger

Have you ever had a Krispy Kreme Burger? It’s definitely over the top. Too much of a good thing.

If you have read many of my posts about performance tuning, you know that I am a big proponent of caching. “Cache whenever you can” and “RAM trumps disk” are two of my mantras of web performance tuning. So, I wondered…can you over-cache?

Apparently you can – in some situations. As I think about how operating systems use memory, it is obviously a resource to be utilized carefully because it is not an inexhaustible supply. Back in the days (1985) we used to watch memory consumption like a hawk on our VAX 11-780. Even more so on those PDP machines. But as RAM got bigger and operating systems got more sophisticated in handling paging, I stopped worrying about memory much.

I found a couple of articles about Windows using too much cache and The Memory Shell Game. While the general principles apply to web applications using too much of a system resource, it doesn’t really address the specifics of how caching can be overdone.

The first article describes how the cache manager will throttle I/O into the cache manager in order to prevent applications from overtaking physical RAM with write I/O. This is called “excessive cached write I/O”.

Additionally, excessive cached read I/O is where an application actively reads many files continuously through the cache manager causing more physical pages to be consumed. When the cache manager grows too big, other processes will get paged out to disk. This can have a catastrophic affect on performance.

Both of these cache problems are more prevalent on 64 bit systems, so I would conclude that web developers will be seeing more of this issue in the future.

Now that I work in web performance field, memory has again become important to me. As load increases, it is a key performance indicator to measure memory consumption.

Even so, rarely have I ever seen caching have a negative performance impact on web applications. These articles do show a downside to caching that can affect the overall performance of a server. That is a lesson to learn. But I just don’t see a way to make the jump from the data in these articles to over-caching in a web system.

Caching is good. Caching is a performance engineer’s friend.