Your Node.js Backend Is Slow. Here's the 4-Layer Caching Strategy I Used to Handle 600K Concurrent Users.

Every few weeks, a founder or CTO reaches out to me with the same problem: "Our backend was fast when we launched. Now it's painfully slow, our cloud bill is climbing, and my team keeps saying we need to 'rewrite everything.'"

I've heard this story dozens of times. And in almost every case, a full rewrite isn't what they need.

Let me share what actually works — drawn from a real project where I took a system drowning under 600,000 concurrent users and not only made it fast but reduced the infrastructure from 12 virtual machines down to 4. No rewrite. No new framework. Just a systematic caching strategy.

The Real Problem Isn't Your Code — It's Your Data Flow

Here's what most development teams get wrong about performance: they look at the code first. They refactor functions, optimize loops, switch to the latest framework. The result? Maybe a 5-10% improvement.

The reason is simple. In most high-traffic applications, 80% or more of the latency comes from how data moves through your system — database queries, API calls, network hops — not from the application logic itself.

When I was brought in to optimize a large-scale media platform serving billions of page views per month, the team had already spent months "fixing the code." Response times were still averaging 800ms+. The bottleneck was obvious: the same database queries were being executed millions of times per hour, returning the same results, because nothing was cached at any meaningful layer.

The 4-Layer Caching Architecture

Layer 1: Application-Level Caching with Redis

Tiered TTL Strategy — Not all data is equal:

• User session data: 5 minutes TTL
• Category pages and navigation: 30 minutes TTL
• Trending content: 2 minutes TTL
• Static reference data (city lists, categories): 24 hours TTL

Cache Warming via Cron Jobs — Instead of waiting for the first user to hit a cold cache, background jobs pre-populated the cache for the top 500 most accessed pages every 90 seconds.

Redis Cluster for Horizontal Scaling — A single Redis instance becomes a bottleneck fast. Deploy a Redis cluster with read replicas so cache reads are distributed across multiple nodes.

Result at this layer: Average API response time dropped from 800ms to ~200ms for cached endpoints.

Layer 2: HTTP Reverse Proxy Caching with Varnish

Redis handles the application layer, but thousands of requests still hit the Node.js process just to check the cache. At 600K concurrent users, even that overhead adds up.

Varnish sits in front of your application servers and caches entire HTTP responses. When a request comes in for a page Varnish has cached, it serves the response directly from memory — the request never even reaches Node.js.

Configuration highlights:

• URL-based cache rules: Homepage and top 50 category pages cached aggressively with 60-second TTL; article pages get 5 minutes
• Grace mode for stale content: Varnish serves the stale version immediately while asynchronously refreshing in the background — users never see a slow response
• Cache invalidation via purge endpoints: When editors published new content, the CMS sent a purge request to Varnish for affected URLs

Combined result after Layers 1 & 2: 70% of all traffic was served directly from Varnish without touching application servers at all.

Layer 3: Database Query Optimization and Pre-Processing

For the 30% of requests that do reach the database, the biggest win wasn't indexing — it was pre-processing.

Instead of running complex aggregation queries in real-time (like "top 10 articles in each city for the last 24 hours"), I moved these computations to scheduled background jobs. A cron job ran every 2 minutes, computed the results, and stored them in a pre-computed results collection.

The API then simply read from this collection instead of running the aggregation.

• Before: 3-5 second aggregation queries under load
• After: Under 10 milliseconds on pre-computed reads

I also implemented read replicas for MongoDB, directing all read-heavy API traffic to secondary nodes while keeping writes on the primary.

Layer 4: CDN and Edge Caching

The final layer pushes static and semi-static content physically closer to users.

The less obvious move: caching certain API responses at the CDN level. For publicly accessible, non-personalized endpoints like "latest headlines" or "weather for Delhi," I set CDN cache headers with 30-60 second TTLs. This meant users in different geographies were served from their nearest CDN node, and the origin server was never hit for these high-frequency requests.

The Results: 12 VMs Down to 4

| Metric | Before | After | |---|---|---| | Average response time (cached) | 800ms+ | Under 50ms | | Average response time (DB-hitting) | 800ms+ | 100-150ms | | Infrastructure | 12 VMs | 4 VMs | | Monthly infrastructure cost | baseline | 60% reduction | | Traffic spike resilience | Cascading failures | Handled 2-3x normal traffic without degradation |

When Not to Cache: The Traps

• Caching user-specific data globally — If personalized content leaks into a shared cache, one user sees another user's data. Always separate personalized endpoints from cacheable ones.
• No cache invalidation strategy — Every cache layer needs an explicit invalidation mechanism — TTL, purge API, event-driven invalidation, or a combination.
• Over-caching during development — Your staging environment should have caching disabled or set to very low TTLs.

What This Means For Your Product

If your backend is slow and your instinct is to rewrite it — pause. Map your data flow first. Identify where the same data is being fetched repeatedly. Then work through the layers: application cache, reverse proxy, database pre-processing, and edge caching.

A well-implemented caching strategy can buy you 12-18 months of runway before you need to consider more invasive changes like a full rewrite or migration to microservices.

Originally published on LinkedIn