Ikibazo gihuza ibibazo · Hagati Urwego

Akamaro k'Ibarurishamibare muri LLM Ikosa Ikibazo

Gusobanura amanota atandukanye ukoresheje igishushanyo mbonera cyateganijwe, intera idashidikanywaho, ingano yingaruka, ningirakamaro ifatika kubicuruzwa.

Inzira zijyanye no kuyoboraAkamaro k'Imibare Muri LLM Evals
Ikibazo 1 Bya 8

Ikizamini gisobanura imibare gisobanura iki?

Reba buri kibazo8 ibibazo

Ibisubizo nibisobanuro bigaragara mugihe ufata ikibazo.

  1. Ikizamini gisobanura imibare gisobanura iki?
    • Whether a model is safe in every deployment
    • Whether users prefer a new interface
    • Whether an observed result would be surprising under a specified null hypothesis
    • Whether a score increase is large enough to matter
  2. Kuki ukoresha igereranya mugihe sisitemu zombi zisubiza ibintu bimwe byo gusuzuma?
    • It preserves item-level correspondence when estimating score differences
    • It guarantees the systems have identical scores
    • It makes every example independent
    • It removes all output randomness
  3. When may McNemar’s test be appropriate?
    • For any continuous score without assumptions
    • Only for latency measurements
    • For comparing unrelated samples
    • For paired binary outcomes on the same items
  4. What should be reported alongside a significance result?
    • Only which model won
    • Effect size, confidence interval, sample size, and test method
    • Only a p-value
    • A screenshot of the leaderboard
  5. Ni ukubera iki gusubiramo icyitegererezo bishobora kuba ingirakamaro?
    • They guarantee deterministic output
    • They replace a representative test set
    • They estimate run-to-run variability separately from variation across test examples
    • They always increase the mean score
  6. Ubusobanuro bwibarurishamibare bugaragaza inyungu kubakoresha?
    • Yes, by definition
    • No; practical effect size and product goals must also be considered
    • Yes, if the p-value is small
    • No, because statistics cannot inform decisions
  7. What should analysts consider when testing many models and metrics?
    • Multiple-comparison risk or treating exploratory results as provisional
    • Ignoring all confidence intervals
    • Choosing only the largest score after testing
    • Changing the test set after seeing results
  8. Nigute amakipe ashobora kugabanya kubogama kubogamye kumanota make?
    • Ship every statistically higher score
    • Test only on examples used to tune the prompt
    • Ignore subgroup changes
    • Use a predeclared practical threshold and inspect representative held-out examples