Read the signals behind every request.

Start with an operational question

Are users waiting longer? Is the origin doing more work? Did a model release change successful outcomes? Each question needs a defined population, a time window, and a comparison. A large collection of charts cannot compensate for an unclear denominator.

Choose a small set of signals that connect an application result to the path that produced it. Add detail when an investigation requires it, and remove dimensions that are expensive or sensitive without improving decisions.

Measure request and byte efficiency

Request cache hit ratio describes how many measured requests were served as hits. Byte hit ratio describes the proportion of measured response bytes served from cache. Document the provider’s cache-status definitions and state whether the calculation excludes ineligible traffic.

A large number of small cached responses can make a request hit ratio look healthy while a few large misses dominate origin transfer. Compare both measures with origin traffic and object size before changing the policy.

Separate the stages of waiting

Connection time, cache lookup, origin work, model execution, and result transfer answer different questions. Record a consistent end-to-end boundary alongside useful internal stages. Track errors separately so fast failures cannot make a latency chart appear improved.

Use latency distributions to understand the slower experiences that an average can hide. Keep the aggregation method and traffic population explicit, especially when comparing regions or releases. The p95 and p99 latency guide explains the common pitfalls.

Keep dimensions intentional

Useful dimensions might include a bounded route category, model release, or region. Unbounded identifiers can make a metric system difficult to operate. Keep detailed request identifiers in an appropriate investigation path instead of adding every identifier to every metric.

Connect metrics to investigations

Metrics help detect changes. Logs can explain individual events. Traces can connect stages in a distributed request. Establish a shared vocabulary so a team can move between those views without guessing whether similarly named measurements mean the same thing.

OpenTelemetry’s overview describes its role in generating and exporting telemetry signals. Choosing an instrumentation framework is only one part of the plan; storage, access, retention, and interpretation remain operational decisions.

Include the model’s outcome

For an AI workload, a successful HTTP response does not necessarily mean a usable answer. Define a task-specific success check, such as a validated output structure or a measured evaluation result. Connect fallback attempts and compute consumption to that useful outcome.

• Write down the exact numerator and denominator for every ratio.
• Record which events are sampled and which counters are complete.
• Protect sensitive request content and limit access to diagnostic detail.
• Compare a release with a relevant baseline before attributing changes to it.

Continue with the CDN analytics playbook, then connect its findings to cache experiments and the cost of the full request path. A signal is most useful when someone knows what action it can justify.