引導式測驗 · 中等 等級
模型實驗中的護欄指標測驗
選擇可行的護欄,定義非劣效裕度並考慮不確定性、多個指標和延遲損害。
問題 1 的 8
What role does a guardrail metric play in an experiment?
預覽每個問題8 問題
當您參加測驗時,就會出現答案和解釋。
- What role does a guardrail metric play in an experiment?
- It monitors an important outcome that should not degrade beyond an accepted limit
- It replaces the primary metric after results are known
- It guarantees every subgroup benefits
- It measures only model training time
- Why predefine a non-inferiority margin?
- It guarantees a zero-effect test
- It states the largest acceptable decline before seeing the result
- It ensures all metrics have equal importance
- It converts a guardrail into an SLO automatically
- Why may a rare safety incident be hard to evaluate in a short experiment?
- The primary metric removes safety outcomes
- Rare events have no user impact
- Too few events may provide insufficient statistical power
- Confidence intervals are always zero
- What should a team do if a guardrail estimate is inconclusive?
- Claim a primary-metric gain resolves the uncertainty
- Treat it as proof of no harm
- Ignore the metric permanently
- Consider uncertainty and predefined escalation rules
- Why can many guardrail metrics complicate interpretation?
- More metrics remove uncertainty
- Multiple comparisons and noisy signals can create false alarms
- Guardrails are independent by definition
- They prevent subgroup analysis
- Which metric is a plausible operational guardrail for an inference service?
- Number of optimizer parameters
- Training set alphabetic order
- p99 latency
- The model's file size alone
- Why define an owner and response action for each guardrail?
- It guarantees the model improves
- It makes the metric statistically independent
- It avoids choosing thresholds
- A metric must lead to an appropriate decision when it changes
- What should be reported with guardrail results?
- Metric definition, uncertainty and sample coverage
- Only a green or red icon
- The model's training loss only
- A claim of universal safety