quiz bu lëkkale ak tegtal · Diggante Niveau

Laajte bu leer ci metrik Pass@k

Dafay saytu li ñuy woowe pass@k, estimateur bu amul benn werante ci n misaal, wóor pass@k, ak anam yu bari yu pass@k poñ yi di juum wala di gëna yokk.

Yooni njiit yu jëm ciJàpp ci K Metrik
Laaj 1 de 8

Lu pass@k di natt ci benn jafe-jafe?

Xoolaatal laaj bu nekk8 laaj

Sooy def quiz bi, tontu yi ak leeral yi dañuy feeñ.

  1. Lu pass@k di natt ci benn jafe-jafe?
    • fraction k échantillons yi jàll ci test yi
    • Probabilite bi benn ci k jéego yuñ jël ngir jàll test yi
    • Probabilite bi jéem bu njëkk bi di romb ginaaw k jéemaat ci fine-tuning
    • Limu test yi jàll ci k test yi
  2. HumanEval, muy référence biñ dugal ci këyit Codex, lan lay denc?
    • 2,294 jafe-jafe GitHub yu bawoo ci projet Python
    • 1000 jafe-jafe prograam ci C++
    • 164 jafe-jafe prograam Python bind ak loxo ak test unitaire
    • 500 rapoor ci bug yu nit saytu
  3. Lan moo waral këyit Codex defar n misaal yu n ëpp k?
    • Ngir xayma ab xayma bu jaar yoon, bu woyof ci pass@k
    • Ngir may model bi mu jàngee ci ay njuumteem ci diggante misaal yi
    • Ndax test yi deñuy laaj 200 input
    • Ngir defar njariñu model bi
  4. Ak n misaal ak c jub, ban wax mooy estimatëru pass@k bu amul benn par?
    • c xaaj ak n
    • 1 dindi (1 dindi c/n) ci doole k
    • C(c, k) xaaj ak C(n, k)
    • 1 dindi C(n dindi c, k) xaaj C(n, k)
  5. Luy pass^k, buñu fësal ci këyit tau-bench, natt?
    • Chance bi benn ci k test yi di am ndam
    • Chance bi k test independent yépp di am ndam
    • Limu jéego yi ci diggante k jéego yi
    • Njortu yi ginaaw k tour yu jubbanti sa bopp
  6. naka la tamperatiir biy jël misaal di jaxasoo ak k?
    • Tàngoor amul benn njeexital ci pass@k
    • Tàngoor wu woyof moo gën ci bépp k
    • Tàngoor wu gëna rëy dafay jàppale jàll@k ci k bu mag ndax misaal yi ñoo gëna bari
    • Tàngoor wu gëna kawe dafay gëna baaxal pass@1
  7. Lan moo waral poñ yu bari ci pass@100 mën na ëpp njariñ li ci àdduna?
    • Dafay xalaat ni amna lu xam bu baax ban ci misaal yi moo dëggu
    • Dafay lim misaal bi njëkk kese
    • Dafay baña bayyi xel ci test yi
    • Dañu koy xayma saa yu nekk ak dekodaas bu bëgg xaalis
  8. Sudee n = 20 misaal amna ci c = 18 yu jub, lan mooy pass@5?
    • 0.9
    • Lu tollu ci 0.5
    • Mënu ñu ko xayma ndax n dafa tuuti lool
    • ndànk