Adrian Cockcroft: why P99 fails for modern services, and the AI-built tools that help

Performance engineer Adrian Cockcroft, whose career spans Sun Microsystems, Netflix, eBay and Amazon, joined RedMonk analyst Rachel Stephens at P99 CONF to discuss how AI is changing performance engineering. The conversation covered his early kernel work at Sun, the limits of P99, and the analysis tools he now builds with large language models.

From kernel code to AI-built tools

At Sun, Cockcroft recalled, engineers looked at system metrics such as vmstat output and guessed what the numbers meant; the documentation was vague. So he went to the source itself: he read the kernel code, worked out exactly where each number came from and which ones were approximations, and wrote it all down. That effort produced two books: "Sun Performance and Tuning" and "Resource Management".

Four decades later, tracing tools are everywhere, but Cockcroft's interest lies in what they do not show. Systems can look fine in dashboards while behaving badly, he said, so he hunts for new ways to read the data: finer grain, deeper levels, distributions instead of averages. Today he builds them by vibe coding with an LLM. "My speedup is infinite, because this code would never exist without these tools. I wouldn't have the time to build them."

Peaks, not percentiles

Cockcroft has argued for years that a single P99 number cannot describe the latency of a modern web service. A percentile cannot tell you whether the underlying distribution has one peak or several; once there is more than one peak, the mean, the standard deviation and even the P99 itself lose most of their meaning.

His favourite example is a two-peak histogram: a fast peak from cache hits and a slow one from misses that need real work. When the cache hit rate changes, each peak stays in place and only its height moves, while the average and the P99 swing widely. After ChatGPT arrived, Cockcroft automated the statistical approach he had already worked out: the open-source tool, written in R, detects an arbitrary number of peaks and tracks how they fluctuate over time.

Advice: start wide, then zoom in

Asked what he would tell teams working on high-performance systems, Cockcroft advised starting with the macro view to find what is interesting, then digging deeper until you are inspecting individual slow requests end to end. He compared it to the microscope you got as a kid: focus at 10x first, click to 100x to look at a single speck, then to 1,000x once that is sharp.

P99 CONF 2026, a free virtual conference about performance, runs on October 21-22.