Why the Average Response Time Lies

Drag the sliders and watch the average drift away from what users actually experience, while p95 and p99 tell the truth.

Try a scenario:
Average
p50 (median)
p95
p99
Max
1,000 simulated requests. Each bar is how many requests took that long.

What you're looking at

The chart shows 1,000 simulated requests. Most of them take around the "typical response time". Some share of them, controlled by the second slider, hit a slow path instead: a slow database query, a cache miss, a timeout. That's a very common shape for real systems, and it's exactly the shape the average is worst at describing.

Pick the "Slow database query" scenario and look at where the average lands. It sits in a gap where almost no actual requests are. Nobody had that experience. Most users got something much faster, and a minority got something much slower.

Which number should I report?

  • p50 (the median) is the typical experience. Half of requests were faster, half slower.
  • p95 is the "bad but not unusual" experience: 1 in 20 requests were at least this slow. This is the number I'd put front and centre in most reports and thresholds.
  • p99 is the tail. If you have a lot of traffic, 1% is still a lot of people, so it's worth tracking for anything important.
  • Max is a single request. It's worth a glance, but it's often caused by something unrelated (a garbage collection pause, a cold cache), so I wouldn't set pass/fail criteria on it.

Why the tail matters more than it looks

The last slider is the one that surprises people. A single page view usually makes several API calls, and the page is only as fast as the slowest one. If each call has a 5% chance of being slower than the p95, then a page making 10 calls has about a 40% chance of hitting at least one of them.

So "only 5% of requests are slow" can easily mean "a big chunk of page views feel slow". That's why the p95 and p99 deserve more attention than they usually get.

Getting percentiles out of k6

k6 shows p(90) and p(95) in its end-of-test summary by default. You can choose which stats it shows with summaryTrendStats:

export const options = {
  summaryTrendStats: ['med', 'p(90)', 'p(95)', 'p(99)', 'max'],
};

And to make the test actually fail when the tail gets too slow, add a threshold:

export const options = {
  thresholds: {
    http_req_duration: ['p(95)<500', 'p(99)<1500'],
  },
};

There's more on this in Stop Reporting the Average Response Time. If you're planning a test, the k6 VU calculator will tell you how many virtual users you need to hit a target load.