Monitorowanie ML za pomocą Prometheusa i Grafana Quiz

Odpowiedzi i wyjaśnienia pojawiają się w miarę rozwiązywania quizu.

1. How does Prometheus commonly collect metrics from a model service?
   • Scrape a metrics endpoint exposed by the service
   • Read every user's prediction database directly
   • Retrain the model at each interval
   • Inspect dashboard screenshots
2. Which metric type is suited to cumulative request counts?
   • Gauge
   • Counter
   • Histogram bucket label only
   • Dashboard annotation
3. Why avoid a user ID as a Prometheus label?
   • It prevents latency measurement
   • It changes the model weights
   • It can create an unbounded number of time series
   • It makes the metric name invalid
4. What can a latency histogram provide?
   • A direct causal explanation
   • A guarantee of model accuracy
   • A raw prompt transcript
   • Aggregatable bucket counts for latency distributions
5. What should a Grafana dashboard show for an inference service?
   • Only model training accuracy
   • Service latency, errors, traffic and relevant model signals
   • Every raw user prompt
   • Only the most recent prediction
6. Why separate service health alerts from model-quality monitoring?
   • Quality never changes after deployment
   • Prometheus cannot store model metrics
   • A service can be available while predictions are poor, or vice versa
   • Latency is the only useful metric
7. Where should raw request IDs or sensitive prompt text generally be handled?
   • Public dashboard titles
   • Unbounded Prometheus labels
   • Metric names
   • Controlled logs or traces with access and retention policies
8. What does a pending period in an alert rule help with?
   • Avoiding alerts on brief transient metric spikes
   • Reducing metric cardinality
   • Improving model calibration
   • Changing histogram buckets after collection