引導式測驗 · 中等 等級

提示詞評估需要多少測試案例測驗

圍繞決策、可檢測效果、結果可變性和代表性抽樣而不是固定計數計劃即時評估。

問題 1 的 8

Why is there no single sample size that is always enough for prompt testing?

預覽每個問題8 問題

當您參加測驗時,就會出現答案和解釋。

  1. Why is there no single sample size that is always enough for prompt testing?
    • It depends on variability, effect size and the statistical design
    • Every prompt needs exactly 100 examples
    • It depends only on model parameter count
    • Confidence intervals eliminate case selection
  2. What does a confidence interval add to a measured pass rate?
    • A guarantee the next version will be better
    • An estimate of uncertainty around the measured rate
    • A list of training examples
    • A truth score for each answer
  3. Why compare versions on the same representative cases when possible?
    • To remove all output randomness
    • To make every case equally difficult
    • To enable paired comparison accounting for case-level differences
    • To increase the number of independent cases automatically
  4. What does repeating a prompt on one input measure most directly?
    • Population diversity
    • Coverage of the task distribution
    • Independent case count
    • Run-to-run variation for that input under those settings
  5. If a team oversamples rare safety failures, what should it avoid?
    • Reviewing those cases separately
    • Using them to discover failure modes
    • Treating the raw failure fraction as population prevalence without weighting
    • Labeling the sample as targeted
  6. Which inputs support a statistical sample-size plan?
    • Prompt word count only
    • Model training-set size
    • Number of evaluators
    • Target effect, outcome variability, confidence/significance and desired power
  7. What should a team define before comparing prompts?
    • The best early pilot outcome
    • Only which prompt looks polished
    • Sample size after seeing every result
    • Metrics, sampling frame, grading rules and decision threshold
  8. Given a small evaluation with a wide interval, how strong a conclusion is justified?
    • The prompt is proven to improve quality
    • Ignore the interval if the mean is higher
    • The result is uncertain and more representative evidence may be needed
    • Ship because no regression was observed