引導式測驗 · 中等 等級

提示版本控制和回歸測試測驗

測試版本化提示配置中的內容、回歸套件如何處理不確定性,以及為什麼模型快照和 LLM 法官需要固定。

問題 1 的 8

哪一組項目應作為一個提示配置一起進行版本控制?

預覽每個問題8 問題

當您參加測驗時,就會出現答案和解釋。

  1. 哪一組項目應作為一個提示配置一起進行版本控制?
    • Only the prompt wording
    • 僅型號名稱
    • Prompt template, model identifier, sampling parameters and tool schemas
    • Only the user inputs from production
  2. 為什麼要固定過時的模型快照而不是通用的最新別名?
    • So model changes happen only when you deliberately switch and re-test
    • Because aliases are always slower
    • Because snapshots are free
    • Because aliases do not support system prompts
  3. 指南对温度 0 有何说明?
    • It guarantees identical outputs every time
    • It makes regression testing unnecessary
    • It only affects the length of outputs
    • It reduces randomness but does not guarantee identical outputs across runs
  4. 回归套件应如何处理不确定性输出?
    • Require exact string matches
    • Use thresholds and repeated samples instead of exact equality
    • Ignore any failing cases
    • Only test with a single example
  5. 黃金組新測試案例的強大來源是什麼?
    • Past production failures turned into test cases
    • Random text from the web
    • Only cases the current prompt already passes
    • Outputs copied from a competitor
  6. If an LLM judge grades outputs, what should you do?
    • Let the judge model change freely
    • Trust every verdict without review
    • Pin the judge's model and prompt and check it against human labels
    • Use the same prompt under test as the judge
  7. Why log the prompt version ID with each production request?
    • To reduce token costs
    • To improve model accuracy
    • To satisfy the model provider's terms
    • To tie problems to a specific deploy and roll back quickly
  8. Why list individual cases that flipped from pass to fail rather than only the average score?
    • Averages are always wrong
    • An unchanged average can hide new breakage offset by gains elsewhere
    • Flipped cases reduce API costs
    • Only flipped cases are graded