AMABWIRIZA Yibanze

Ibipimo byanduye

Ibipimo byanduye bibaho mugihe ibibazo cyangwa ibisubizo bivuye mubipimo by'isuzuma bikarangirira kumyitozo yicyitegererezo, bityo amanota menshi arashobora kwerekana gufata mumutwe aho kuba ubushobozi nyabwo.

Umva iki gitaboAI-byavuzwe · ~3 min

Kuriyi page4 min soma
  1. Incamake
  2. Kwibira cyane
  3. Gushyira mu bikorwa Isi
  4. Ejo hazaza h'ibipimo byanduye
  5. Kubishyira mubikorwa
  6. Komeza Ubushakashatsi
  7. Ibibazo bikunze kubazwa

Incamake

Nibyiza kuko nimero yubuyobozi itwara ibyifuzo byubushakashatsi no kugura ibyemezo, kandi amanota yanduye arerekana uburyo icyitegererezo kizakora kubibazo itigeze ibona.

Kwibira cyane

Ururimi runini rwururimi rwahuguwe kumurongo wurubuga rurimo GitHub, arXiv impapuro, amahuriro nurubuga rwigisha. Ibipimo bizwi cyane ni rusange kubishushanyo mbonera, bityo ibibazo byabo, ibisubizo nibiganiro byabo bikopororwa kurubuga kandi bigahita byinjira mumahugurwa. Kwanduza birashobora kuba bitaziguye, aho ikintu cyibizamini kigaragara, cyangwa kitaziguye, binyuze mu magambo ahinnye, ibisobanuro, ibisubizo byanditse, cyangwa amakuru yubukorikori yakozwe nubundi buryo bwabonye igipimo. Irashobora kandi kwinjira mugihe cyo gutondekanya neza niba amabwiriza yamakuru yakusanyirijwe mubipimo-nkomoko. Itandukaniro ryingirakamaro riri hagati yo kubona ikibazo gusa no kubona ikibazo nigisubizo cyacyo. Iya kabiri irangiza cyane, kubera ko icyitegererezo gishobora kwibuka igisubizo aho kugikora. Igisubizo ni amanota yuzuye atimurira mubibazo bishya. Kumenyekanisha laboratoire ntabwo ari shyashya. Impapuro GPT-3 (2020) yakoresheje isesengura rya garama 13 hagati y’ibipimo byayo hamwe n’amakuru y’amahugurwa kandi yemera ko ikosa ryasize aho rihurira. Iyo amakuru yo guhugura arahari, gushakisha birenze ni igenzura ritaziguye. Iyo ataribyo, abashakashatsi bakoresha uburyo butaziguye: kubaza icyitegererezo hamwe no gutangira ikintu cyikizamini kugirango barebe niba cyuzuza amagambo asigaye; uburyo bwo gufata umwanzuro wabanyamuryango nka Min-K% Ikibazo (Shi na bagenzi be, 2023), bareba uburyo icyitegererezo cyerekana neza ibimenyetso byanditse; kugereranya amanota kubintu byumwimerere kurwanya ibyanditswe byanditse cyangwa bishya byanditse; n'ibihe bishingiye kubice bigereranya ibibazo byatewe mbere na nyuma yo guhagarika imyitozo. Ibitekerezo bibiri bitari byo birasanzwe. Ubwa mbere, guhuzagurika ntabwo buri gihe bizamura amanota menshi; ubushakashatsi bumwe busanga ingaruka zoroheje kubikorwa bimwe. Icya kabiri, gusanga ntaho bihuriye ntabwo byerekana ko igipimo gifite isuku, kubera ko kopi zasobanuwe cyangwa zahinduwe zanyerera zinyuze ku murongo. Impinduka zirimo ibizamini byigenga byakozwe, imirongo ya canary, ibipimo bishya buri gihe nka LiveBench na LiveCodeBench, kuyungurura, no gutangaza isesengura ryanduye hamwe nibisubizo.

Gushyira mu bikorwa Isi

Ibipimo byerekana imibare byongeye gushyirwa kumahuriro hamwe nibisubizo byakozwe; urubuga rwikusanyirizo rukusanya izo paji, kandi icyitegererezo cyayitojwe nyuma kigaragaza ibisubizo nyabyo, harimo nigituba mumagambo yumwimerere.

Abashakashatsi bo muri Scale AI banditse GSM1k, ibibazo bishya by'imibare yo mu ishuri-bihuye na GSM8K mu buryo no mu ngorane, basanga moderi zimwe zatanze amanota make ku gipimo gishya, ikimenyetso cyo gukwirakwira ku bipimo rusange.

Igipimo cyerekana code yerekana buri kibazo cyerekana icyitegererezo gikemura ibibazo byinshi byatangajwe mbere yo guhagarika imyitozo kuruta nyuma, aribwo buryo LiveCodeBench yagenewe gushyira ahagaragara.

BIG-intebe yimirimo ikubiyemo umugozi udasanzwe wa canary kuburyo amakipe yamakuru ashobora gushungura inyandiko iyariyo yose hanze yimyitozo ya corpora hanyuma ikareba niba icyitegererezo gishobora kubyara.

Ejo hazaza h'ibipimo byanduye

Imyitozo yo gusuzuma irahindukira yerekeza kubigenga cyangwa igice cyibizamini byigenga, ibipimo byongera ibintu bishya mugihe, hamwe nabandi basuzuma kugenzura amakuru yikizamini. Ibyo bitera impagarara zifatika zifunguye, kubera ko ibipimo rusange byoroshye kubyara no kugenzura. Gukorera mu mucyo kubijyanye namakuru yamahugurwa byatuma igenzura ryanduye ryoroha cyane, ariko abategura icyitegererezo benshi ntibatangaza amakuru yuzuye. Basomyi bagomba gufata amanota ayo ari yo yose ahamye kandi bakareba ibisubizo ku makuru mashya cyangwa yafashwe mbere yo gufata imyanzuro ikomeye.

Kubishyira mubikorwa

Impamvu ari ngombwa

  • Ibyemezo bisobanutse. Iragufasha gutandukanya ibyifuzo bya tekiniki bisobanutse nururimi rwo kwamamaza.
  • Igiciro na bije. Urashobora kubaza ibibazo byiza byo gushyira mubikorwa mbere yo gukoresha amafaranga cyangwa igihe.
  • Itsinda hamwe nakazi. Amakipe asangiye ibitekerezo akora ibicuruzwa byiza, politiki, nibyemezo byo kwiga.

Witondere

  • Amakipe atandukanye arashobora gukoresha ijambo rimwe muburyo butandukanye, sobanura intera hakiri kare.
  • Ibipimo birashobora kugaragara bikomeye mugihe imikorere-yisi-itaringaniye.
  • Kwirengagiza ubuziranenge bwamakuru na gahunda yo gusuzuma akenshi bitanga ibisubizo byoroshye.

Intambwe ikurikira

  1. Tangira nururimi rusobanutse rwibisubizo ukeneye.
  2. Toranya intsinzi imwe hamwe nuburyo bumwe bwo gutsindwa mbere yo kwipimisha.
  3. Koresha umuderevu muto hamwe namakuru ahagarariye, ntabwo ari demo yashizweho.
  4. Inyandiko aho Benchmark Yanduye ifasha kandi nuburyo bworoshye bworoshye.

Komeza Ubushakashatsi

Ibaruwa y'amakuru y'ubuntu

Ibuka amakuru ya buri munsi ya AI

Inkuru eshatu zizewe za AI buri gitondo cy'iminsi y'akazi, zanditse mu rurimi rworoshye. Ubuntu iteka ryose, nta kwamamaza.

Imeri imwe buri munsi y'akazi. Kwiyimura mu buryo bumwe. Ntitwigera tugurisha cyangwa ngo dusangire aderesi yawe.

Isuzume

Take the Benchmark Contamination quiz

Ibisubizo ako kanya kuri buri kibazo, n'icyemezo gishobora gusangirwa gifite ID yagenzurwa umaze gutsinda isomo.

Tangira ikibazo

Fasha uburezi bwa AI bw'ubuntu. AI Understanding ni umuryango utegamiye kuri leta wa 501(c)(3) — nta kwamamaza, nta kwishyura, iteka ryose. Tanga impano

Ibibazo bikunze kubazwa

Ibipimo byanduye ni iki?

Ibipimo byanduye bibaho mugihe ibibazo cyangwa ibisubizo bivuye mubipimo by'isuzuma bikarangirira kumyitozo yicyitegererezo, bityo amanota menshi arashobora kwerekana gufata mumutwe aho kuba ubushobozi nyabwo. Nibyiza kuko nimero yubuyobozi itwara ibyifuzo byubushakashatsi no kugura ibyemezo, kandi amanota yanduye arerekana uburyo icyitegererezo kizakora kubibazo itigeze ibona.

Ibipimo byanduye ni iki?

Kwanduza bivuze ko icyitegererezo gishobora kuba cyarabonye ibikoresho byikizamini mugihe cyamahugurwa, bityo amanota yacyo ashobora kwerekana gufata mumutwe.

Niyihe ntego yumurongo wa kanari ushyizwe mubikorwa BIG-intebe?

Ikiranga kidasanzwe cyoroha kubona no kuvanaho ibipimo ngenderwaho muri corpora no gusuzuma niba umunyamideli yarafashe mu mutwe.

Ubushakashatsi bwa GSM1k bwatanze iki?

Igitonyanga kubibazo bishya bihuye muburyo hamwe ningorabahizi kugirango bikwiranye nigipimo rusange aho kuba ubuhanga rusange bwimibare.

Ni ubuhe buryo bwo gusesengura impapuro GPT-3 yakoze?

Abanditsi ba GPT-3 basuzumye garama 13 zegeranye kandi bemera ikosa ryasize bimwe bidakuweho.

Niki gitekerezo cyibanze cya Min-K% Ikibazo?

Uburyo bugereranije k munsi ya k ijanisha rya token log-probabilite; gufata mu mutwe amanota amanota maremare adasanzwe kuko bike muri byo bitangaza icyitegererezo.