Quiz połączony z przewodnikiem · Średni Poziom

Quiz dotyczący szybkiego wersjonowania i testowania regresji

Testuje, co należy do wersjonowanej konfiguracji podpowiedzi, jak zestawy regresji radzą sobie z niedeterminizmem i dlaczego migawki modeli i oceny LLM wymagają przypinania.

Powiązane ścieżki prowadząceSzybkie wersjonowanie i testowanie regresyjne
Pytanie 1 z 8

Which set of items should be versioned together as one prompt configuration?

Podgląd każdego pytania8 pytania

Odpowiedzi i wyjaśnienia pojawiają się w miarę rozwiązywania quizu.

  1. Which set of items should be versioned together as one prompt configuration?
    • Only the prompt wording
    • Only the model name
    • Prompt template, model identifier, sampling parameters and tool schemas
    • Only the user inputs from production
  2. Why pin a dated model snapshot instead of a generic latest alias?
    • So model changes happen only when you deliberately switch and re-test
    • Because aliases are always slower
    • Because snapshots are free
    • Because aliases do not support system prompts
  3. What does the guide say about temperature 0?
    • It guarantees identical outputs every time
    • It makes regression testing unnecessary
    • It only affects the length of outputs
    • It reduces randomness but does not guarantee identical outputs across runs
  4. How should a regression suite handle nondeterministic outputs?
    • Require exact string matches
    • Use thresholds and repeated samples instead of exact equality
    • Ignore any failing cases
    • Only test with a single example
  5. What is a strong source of new test cases for a golden set?
    • Past production failures turned into test cases
    • Random text from the web
    • Only cases the current prompt already passes
    • Outputs copied from a competitor
  6. If an LLM judge grades outputs, what should you do?
    • Let the judge model change freely
    • Trust every verdict without review
    • Pin the judge's model and prompt and check it against human labels
    • Use the same prompt under test as the judge
  7. Why log the prompt version ID with each production request?
    • To reduce token costs
    • To improve model accuracy
    • To satisfy the model provider's terms
    • To tie problems to a specific deploy and roll back quickly
  8. Why list individual cases that flipped from pass to fail rather than only the average score?
    • Averages are always wrong
    • An unchanged average can hide new breakage offset by gains elsewhere
    • Flipped cases reduce API costs
    • Only flipped cases are graded