引導式測驗 · 中等 等級
提示詞評估需要多少測試案例測驗
圍繞決策、可檢測效果、結果可變性和代表性抽樣而不是固定計數計劃即時評估。
問題 1 的 8
Why is there no single sample size that is always enough for prompt testing?
預覽每個問題8 問題
當您參加測驗時,就會出現答案和解釋。
- Why is there no single sample size that is always enough for prompt testing?
- It depends on variability, effect size and the statistical design
- Every prompt needs exactly 100 examples
- It depends only on model parameter count
- Confidence intervals eliminate case selection
- What does a confidence interval add to a measured pass rate?
- A guarantee the next version will be better
- An estimate of uncertainty around the measured rate
- A list of training examples
- A truth score for each answer
- Why compare versions on the same representative cases when possible?
- To remove all output randomness
- To make every case equally difficult
- To enable paired comparison accounting for case-level differences
- To increase the number of independent cases automatically
- What does repeating a prompt on one input measure most directly?
- Population diversity
- Coverage of the task distribution
- Independent case count
- Run-to-run variation for that input under those settings
- If a team oversamples rare safety failures, what should it avoid?
- Reviewing those cases separately
- Using them to discover failure modes
- Treating the raw failure fraction as population prevalence without weighting
- Labeling the sample as targeted
- Which inputs support a statistical sample-size plan?
- Prompt word count only
- Model training-set size
- Number of evaluators
- Target effect, outcome variability, confidence/significance and desired power
- What should a team define before comparing prompts?
- The best early pilot outcome
- Only which prompt looks polished
- Sample size after seeing every result
- Metrics, sampling frame, grading rules and decision threshold
- Given a small evaluation with a wide interval, how strong a conclusion is justified?
- The prompt is proven to improve quality
- Ignore the interval if the mean is higher
- The result is uncertain and more representative evidence may be needed
- Ship because no regression was observed