An edge application can feel fast during a demonstration and inconsistent under real traffic. Most requests finish promptly, while a smaller group waits for a cold dependency, a busy queue, or a distant service. A single average compresses those different experiences into one number.

Tracking p95 and p99 latency helps make slower requests visible. The numbers become useful when they describe a clearly defined request population, use trustworthy measurements, and connect to a decision the team can make. Without that context, a percentile is another dashboard label.

Understand what a percentile describes

P95 is the 95th percentile of an observed response-time distribution; p99 is the 99th. Informally, about 95 percent and 99 percent of observations lie at or below the respective values. The precise result can depend on the estimator and, for histograms, the available buckets. Neither value is the maximum, and neither promises that every user receives that experience.

Consider an illustrative set of 1,000 requests: 990 finish in 100 milliseconds and ten finish in five seconds. The arithmetic mean is 149 milliseconds. That average describes the arithmetic correctly while obscuring the ten very different experiences. A p99 near the boundary may also fail to describe how far the slowest observations extend.

Report the distribution alongside request counts, failures, and useful thresholds. Read how to interpret CDN analytics for the surrounding delivery signals that help explain a change.

Use tail measurements to ask better questions

Google's SRE guidance on monitoring distributed systems explains why averages can conceal slow requests and recommends collecting latency distributions. It also distinguishes successful request latency from failed request latency, and places latency alongside traffic, errors, and saturation. These signals help separate a slower service from one that fails quickly or becomes overloaded.

Apply that perspective to a concrete user action. Ask whether a slower tail affects initial page readiness, an API operation, a model download, or an inference result. Set a measurement boundary and a target that correspond to that action. Two requests with different purposes may require different expectations.

A percentile chart should prompt investigation rather than assign blame. A rising p99 may justify examining queue depth, dependency calls, regional traffic, or recent releases. Establish which additional evidence would support each explanation before selecting a fix.

Define the beginning and end of the clock

Measure the experience the user waits for

Choose whether a measurement starts before connection setup, when the edge receives the request, or when application processing begins. Likewise, distinguish the first response byte from the complete response. Each boundary answers a different question; name it clearly in dashboards and reports.

For streaming inference, consider separate measurements for the first useful output, gaps during delivery, and total completion. An early first token does not by itself describe the remainder of a long response. For model delivery, measure when the artifact becomes usable if that is the actual product milestone.

Keep stage timing connected to complete requests

Instrument connection, routing, queueing, cache lookup, origin work, and transfer stages where they apply. Preserve a shared request identifier in traces. Do not add stage p99 values and label the sum as end-to-end p99: the slowest observation in each stage may belong to a different request. Measure complete requests directly and use stage data for diagnosis.

Aggregate observations before calculating percentiles

Averaging regional p95 or p99 values does not produce the percentile of all requests. Percentiles describe positions in distributions, so a percentile value alone loses the observations needed to reconstruct a combined distribution. Weighting those regional percentile values by traffic does not repair that loss.

Instead, combine raw observations or compatible histogram distributions for the population and interval you intend to describe, then calculate the desired percentile. Check units, bucket boundaries, metric definitions, and the aggregation behavior of the monitoring system. Bucket-based percentiles are estimates, so retain enough resolution around the response-time ranges that matter.

For a threshold objective, a separate calculation may be more direct: total qualifying requests divided by total eligible requests. Sum those counts across regions using the same definition. This avoids trying to infer the proportion meeting a threshold from an already summarized percentile.

The analytics measurement guide helps connect these definitions to an operational dashboard.

Compare traffic populations deliberately

A global request-weighted metric answers what happened across the observed requests. A regional metric answers what happened within that region. Both are useful. A high-volume region can dominate the global result, so keep regional views when geography matters to the application.

Compare like workloads: cached and uncached requests, short and long outputs, authenticated and public routes, successful and failed operations. Start with a few meaningful dimensions. Excessive segmentation can leave each chart with too little traffic to interpret, while a single blended chart can hide a specific regression.

Request weighting also differs from user weighting. A highly active client contributes more observations than someone who makes one request. If the goal concerns the proportion of users affected, define a user-level measure separately. Do not describe a request percentile as a statement about the percentage of people having a satisfactory experience.

Account for sample size and missing observations

Display the observation count and reporting window beside high percentiles. In a small sample, very few observations determine the upper tail. A sudden movement may reflect a handful of requests, a changed traffic mix, or a genuine regression. Examine those possibilities before attributing the change to a release.

Sampling deserves the same scrutiny. A diagnostic trace collection designed to retain errors and slow requests is valuable for investigation, but its unweighted latency distribution does not represent ordinary traffic. Understand what was included, what was excluded, and whether the metric uses all requests or an appropriate estimation method.

Keep timeouts and cancellations visible. Dropping them from the latency view can make the remaining completed requests appear healthier as user experience worsens. Report their counts and define how they affect objectives. When a timeout hides the eventual completion time, do not invent an exact duration for the unseen work.

Also distinguish a logical operation from its individual attempts. If an application retries a failed call, an attempt-level chart and a user-operation chart describe different populations. Measure the full wait experienced by the caller, and retain attempt details for diagnosis. Decide how a cancelled retry is recorded before comparing two routing policies.

Investigate the slow path before changing routing

Select a representative group of slow requests and compare them with ordinary requests from the same route and period. Look for recurring queue delays, connection setup, retries, cache misses, model initialization, payload size, or dependency waits. Treat each pattern as a hypothesis to verify.

The edge network guide explains where delivery components fit, while the LLM request routing guide explores routing choices. Moving work closer to a user is useful only when it addresses an important part of the measured delay. A nearby endpoint can still spend most of its time waiting elsewhere.

Test one bounded change against a comparable baseline. Record workload, concurrency, response quality where relevant, error behavior, and the complete latency distribution. Include cold starts, bursts, and steady demand. Watch for work that gets displaced to another stage or a different group of users.

Set explicit stop conditions for an experiment. A better p99 accompanied by more failures, missing results, or degraded output may not meet the application's objective.

Write the conclusion with its scope: which route improved, under what demand, for which locations, and over what period. Preserve unresolved cases. That record helps the next investigation distinguish a verified improvement from an assumption that all traffic benefited equally.

Conclusion: make the tail an operational decision

P95 and p99 help describe the slower portion of a request distribution. Their value depends on clear timing boundaries, appropriate aggregation, meaningful traffic groups, and visibility into errors and missing observations.

Begin with one important user action. Define its eligible requests, measure its complete duration, retain a distribution, and connect slow examples to traces. Use that evidence to choose a focused improvement and evaluate its consequences. The goal is a more dependable experience whose behavior the team can explain.