Quiz collegato alla guida · Medio Livello

Quiz rapido sul controllo delle versioni e sulla regressione

Verifica cosa appartiene a una configurazione di prompt con versione, come le suite di regressione gestiscono il non determinismo e perché è necessario bloccare le istantanee del modello e i giudici LLM.

Domanda 1 di 8

Which set of items should be versioned together as one prompt configuration?

Anteprima di ogni domanda8 domande

Le risposte e le spiegazioni vengono visualizzate mentre rispondi al quiz.

  1. Which set of items should be versioned together as one prompt configuration?
    • Only the prompt wording
    • Solo il nome del modello
    • Prompt template, model identifier, sampling parameters and tool schemas
    • Only the user inputs from production
  2. Why pin a dated model snapshot instead of a generic latest alias?
    • So model changes happen only when you deliberately switch and re-test
    • Because aliases are always slower
    • Because snapshots are free
    • Because aliases do not support system prompts
  3. What does the guide say about temperature 0?
    • It guarantees identical outputs every time
    • It makes regression testing unnecessary
    • It only affects the length of outputs
    • It reduces randomness but does not guarantee identical outputs across runs
  4. How should a regression suite handle nondeterministic outputs?
    • Require exact string matches
    • Use thresholds and repeated samples instead of exact equality
    • Ignore any failing cases
    • Only test with a single example
  5. What is a strong source of new test cases for a golden set?
    • Past production failures turned into test cases
    • Random text from the web
    • Only cases the current prompt already passes
    • Outputs copied from a competitor
  6. If an LLM judge grades outputs, what should you do?
    • Let the judge model change freely
    • Trust every verdict without review
    • Pin the judge's model and prompt and check it against human labels
    • Use the same prompt under test as the judge
  7. Why log the prompt version ID with each production request?
    • To reduce token costs
    • To improve model accuracy
    • To satisfy the model provider's terms
    • To tie problems to a specific deploy and roll back quickly
  8. Why list individual cases that flipped from pass to fail rather than only the average score?
    • Averages are always wrong
    • An unchanged average can hide new breakage offset by gains elsewhere
    • Flipped cases reduce API costs
    • Only flipped cases are graded