Kvíz spojený s průvodcem · Střední úroveň
Latence ocasu v kvízu modelování
Interpretujte latenci p95 a p99 a vyhodnoťte dávkování, řazení do fronty, časové limity a zajištění bez vytváření dodatečné zátěže nebo duplicitních vedlejších účinků.
Otázka 1 z 8
What does p99 latency mean for a measured request sample?
Náhled každé otázky8 otázky
Odpovědi a vysvětlení se zobrazují, jak budete v kvízu.
- What does p99 latency mean for a measured request sample?
- The single slowest request
- The average latency of all requests
- 99 percent of requests completed at or below that value
- 99 requests per second
- Why can mean latency hide a service problem?
- A low mean proves every request met its latency objective
- A slow tail can affect a smaller share of requests without shifting the average much
- Averages always equal p99
- The mean uses the p99 value as its input
- What tradeoff can dynamic batching introduce?
- Lower throughput with guaranteed lower latency
- More model accuracy without extra work
- Reduced memory with no scheduling effects
- Higher throughput but additional queueing delay
- When is hedged inference safest to consider?
- For idempotent requests with controlled duplicate work
- For non-repeatable actions such as charging a customer
- When the service is already overloaded with no limits
- For requests that modify shared state
- How can hedged requests make a latency incident worse?
- They add concurrent attempts and more load when the fleet is already congested
- They convert the tail distribution into a fixed upper bound
- They eliminate slowdowns when replicas share a saturated backend
- They serialize each duplicate so only one request consumes resources
- Why measure end-to-end latency rather than only model kernel time?
- Kernel time always equals request time
- Queueing, preprocessing and network time also affect user response
- Preprocessing cannot add delay
- Network time is included in model FLOPs
- What additional signal should be examined alongside latency percentiles?
- Only the model parameter count
- Only average training loss
- The checkpoint file name
- Timeout and failure rates
- Why should latency percentiles include a stated measurement window and population?
- The estimate depends on which requests and time period were sampled
- Percentiles are fixed constants for a model
- The window determines the model weights
- It removes all sampling variability