How We Cut P95 API Latency by 40% Without a Rewrite
When a client's API p95 latency started creeping past 800ms, the first instinct on the team was to blame the framework and start planning a rewrite. We asked them to hold off for two weeks and look at what was actually slow first.
The real bottleneck wasn't the request layer — it was three specific database queries running on every request that could have used an existing index but didn't, plus a connection pool sized for a traffic pattern the service had long outgrown. Fixing the indexes and resizing the pool closed most of the gap on their own.
The remaining latency came from a handful of hot reads that were safe to cache for a few seconds without users noticing stale data. Adding a short-lived cache in front of those specific reads — not a blanket caching layer — brought p95 down from 800ms to under 480ms.
None of this required touching the framework or the service's architecture. The lesson we keep relearning on engagements like this: measure before you rewrite, because the actual bottleneck is rarely where the team assumes it is.
Back to all articlesRelated Articles
A Practical Guide to RAG Pipeline Evaluation
The metrics that actually predict whether your RAG system will hold up in production.
Core Web Vitals in 2026: What Actually Moves the Needle for Enterprise Web Apps
Most Core Web Vitals advice targets marketing sites — enterprise web apps have a different bottleneck.
Fine-Tuning vs. RAG vs. Prompt Engineering: A Practical Decision Framework
Three ways to make an LLM know about your business, and how to tell which one actually fits your problem.