Debugging a MongoDB/WiredTiger Cache Pressure Issue in Production

The Problem #

The API would intermittently slow to a crawl for a minute or two, several times a day, with no obvious pattern — not tied to a specific route, not correlated with traffic spikes I could see in the app's own logs.

Context #

The Mongo instance was self-hosted, sharing a small VM with the API process itself, which turned out to matter more than I initially gave it credit for.

What I Tried #

Started where I assumed the problem was: added indexes to the slowest-looking queries in mongosh's explain() output, on the theory that something was doing a collection scan under load.

What Went Wrong #

The indexes helped the specific queries I profiled, but the intermittent slowdowns kept happening anyway — because they weren't caused by a slow query at all. explain() on an individual query looked fine; the problem only showed up as aggregate latency across everything at once.

The Solution #

Checked db.serverStatus().wiredTiger.cache during a slowdown and found the cache was consistently near its configured limit, forcing WiredTiger into aggressive eviction — which blocks writes while it happens. The VM's total RAM was undersized for both Mongo's default cache sizing and the Node process running alongside it.

// mongosh, during a slowdown
db.serverStatus().wiredTiger.cache["bytes currently in the cache"]
db.serverStatus().wiredTiger.cache["tracked dirty bytes in the cache"]

Set wiredTigerCacheSizeGB explicitly instead of letting it default to roughly half of total system RAM, leaving enough headroom for the API process to not starve Mongo of memory under load.

Why It Works #

WiredTiger's default cache sizing assumes it can have about half the box's RAM to itself. On a shared VM where another process is also competing for memory, that assumption is wrong, and the database spends CPU evicting pages under memory pressure instead of serving queries — invisible to any single query's explain() output because it's a resource contention problem, not a query plan problem.

Lessons Learned #

An intermittent, traffic-uncorrelated slowdown that doesn't show up in query-level profiling is a signal to look at the database's resource metrics, not to keep tuning individual queries that already look fine in isolation.

What I Would Do Differently #

I'd check serverStatus() cache metrics as a first step for any "slow sometimes, for no obvious reason" report, before spending an afternoon adding indexes that didn't address the actual bottleneck.

Related Concepts #

WiredTiger cache eviction, resource contention on shared hosts, db.serverStatus() diagnostics.