Dzokera kuNhau
ChengetedzoAI Understanding muchidimbu

Kudzidza Kunowana AI Chengetedzo Benchmark Inogona Kushandisa Kure Ishoma Miedzo

Iyo UK AI Security Institute-yakabatana preprint yakaburitsa akati wandei ekuchengetedza-benchmark mhedzisiro ine 97-99% mashoma ekukurudzira, uku yambiro kuti mapfupi bvunzo haaratidze kuchengetedzwa chaiko kwenyika.

5 min readRead the primary source
Primary-source documentKwakanyorwa
Muparidzi
Rivera and colleagues' research paper on arXiv
Source link
arxiv.orghttps://arxiv.org/abs/2608.05086
Source type
Gwaro rekutanga - chiziviso chepamutemo, bepa, faira, kana peji rebato rekutanga ratinoverenga zvakananga.
ContextNzwisisa izvi mumasekonzi makumi matanhatu

Tanga pano

Matemu akakosha

AI Kuchengetedza
Munda wakatarisana nekudzikisira maitiro anokuvadza, kutadza, uye njodzi yekushandisa zvisizvo muAI masisitimu.
API (Application Programming Interface)
Nzira yakarongeka yeimwe software system yekutumira zvikumbiro uye kugamuchira mhinduro kubva kune imwe system.
System Prompt
Murairo wepamusoro-soro unoisa maitiro, mutemo, uye maitiro ekupindura emuenzaniso.
Zviedze iwe pachakoChii chinonzi AI? Quiz

Chii chaitika

Bepa rekutsvagisa rakatumirwa muna Nyamavhuvhu 5 rinoshandisa nzira kubva mukuyedzwa kwedzidzo kuyera izvo AI kuchengetedza mabhenji anotorwa, sarudza madiki madiki ezvinokurudzirwa anobatsira, uye odhita shanduko mumaitiro emuenzaniso.

Vatsvagiri vaviri vakazvimiririra nevatsvaguri vaviri vanodyidzana neUK AI Security Institute vakaongorora 192 chat modhi pamabhenji masere anovhara zvinokuvadza-chikumbiro kuramba, kunyanya-kuramba zvikumbiro zvisina kunaka, kukuvadza mamiriro, uye chokwadi. Iyo yakazara suite yaive ne5,255 yekukurudzira isati yagadziriswa; ongororo yakachengeta 5,067 mushure mekubvisa mhinduro dzisina kunyorwa uye zvinhu zvisina kusiyanisa pakati pemhando dzakaedzwa.

Chikwata chakabata mamodheru seanotora bvunzo uye mabhenji ekumisikidza sezvinhu zvekuyedza, vachishandisa chinhu mhinduro theory kufungidzira kuti ndezvipi zvirevo zvaive zvakaoma uye ndedzipi dzakapatsanurwa zvakanyanya. Muchikamu chayo chikuru chekuongorora, mhinduro yezvitatu-yakapfupikiswa sekuramba kusimba, chokwadi, uye kukuvadza kwemamiriro ezvinhu-yakatsanangura 77% yekusiyana kwemaitiro emuenzaniso, zvichienzaniswa ne47% nokuda kwechinhu chimwe chete.

Pakati pemakumi maviri-yakaitwa ongororo, matatu akagadziriswa makumi maviri neshanu-yekukurumidza bvunzo akadzora izvo zvinhu zvitatu achishandisa isingasviki 2% yesutu. YeHarmBench, SORRY-Bench, uye OR-Bench-Hard, zvingangoita gumi zvakasarudzwa zvinokurudzira zvakaburitsa yakazara-benchmark renji ine corelations ye0.92 kusvika 0.94 uye kucheka nhamba yekukurudzira ne97-99%; mukana wacho wakadzikira sezvo bhajeti rebvunzo raikura.

Iyo compression haisi yekungoverengera sampling. Chidzidzo chemhinduro yechinhu chinofungidzira kuti chirevo chega chega chakaoma sei uye kuti chinopatsanura zvakanaka sei modhi nemaitiro emhinduro dzakasiyana, yozosarudza zvinhu zvinochengetedza chimiro chebvunzo hombe. Vanyori vakafananidza mafomu mapfupi akagadziriswa neanochinjika sarudzo uye akabata ongororo pane kuyera chete data rinoshandiswa kusarudza zvinokurudzira. Iyo dhizaini inotsigira iyo yakatetepa chirevo chekuti mamwe aripo mabhenji masheti anogona kudhindwa zvakanaka; hazviratidze kuti bvunzo yegumi- kana makumi maviri neshanu inogona kuwana nzira nyowani yekutadza.

Kwakabva mashoko: Rivera and colleagues' research paper on arXiv ↗

Nei zvichikosha

Chidzidzo ichi chinopa zano kuti zvirinani-zvakasarudzwa zvinokurudzira zvinogona kuita kuti nguva zhinji AI chengetedzo ichepe isingadhure pasina kunyepedzera kuti benchmark yega yega inoyera imwe yepasirese kuchengetedza mamakisi.

Miedzo yakachipa inogona kuita kuti vanogadzira, marabhoritari madiki, uye vaongorori vakazvimiririra vatarise mamwe mamodheru emhando panguva yekudzidziswa, kunyatso-tuning, quantization, uye system-yekukurumidza shanduko. Mhedzisiro yacho ndeye kuburitsa zviyero zviripo zvebhenji zvakanaka, kwete kuwedzera izvo zvinoonekwa nemabhenji.

The factor analysis zvakare inofumura scoring tradeoff. Mabhenji anopa mubairo kuramba anogona kupindirana, nepo bhenji yekunzvenga kuramba kusiri madikanwa inopa mubairo hunhu hwakapesana. Kuaenzanisa pasina kutsanangura huremu kunogona kuvanza kana modhi yakave yakachengetedzeka, inongo ganhurira zvakanyanya, kana kusanyanya kubatsira pazvikumbiro zvisina kunaka.

Iro bepa rakaedzawo nzira dzekuongorora bhokisi dema pamaune akakurudzira kuita jecha. Cheki yemhinduro-pattern yakabatwa yakasarudzwa kusashanda zvakanaka asi yakapotsa yakawandisa yunifomu isingashande; ichiwedzera cheki yekugona kwenhoroondo yakasimudzwa kusanganisirwa ku94-97% mumatambudziko akateedzerwa. Izvo zvinotsigira ongororo dzakaiswa pane kuvimba nechibodzwa chimwe chete.

Kune vanogadzira, kukosha kwekushanda ndeye frequency. Ongororo ipfupi inogona kudzokororwa mushure mekuchinja kwehurongwa-kukurumidza, quantization pass, fine-tune, chishandiso chekuvandudza, kana kuchengetedza-policy kudzokorora pasina kushandisa bhajeti yebhenji izere nguva imwe neimwe. Asi mhedzisiro inobatsira chete kana zvikwata zvakachengeta yekutanga sutu seongororo yenguva nenguva, tenderedza-yakabuda-yekukurudzira, uye nekuongorora shanduko dzinoshamisa pane kugadzirisa zvakananga bvunzo yakamanikidzwa. Zvikasadaro cheki yakachipa inogona kuve imwe chinangwa chekuwandisa.

Interactive Mechanism

Interactive Mechanism: Iyo Inonyatsoshanda

Ongorora ari pasi tekinoroji kuseri kwekusimudzira uku uchipindirana.

Agent Lifecycle Stage:
1
User Intent & Planning: "Audit customer refund request #4092 and settle payment."
2
Tool Calling: Emits structured JSON call crm_get_transaction(id='4092').
3
Guardrail & Verification:🛡️ Paused: High-value action requires human operator sign-off.
4
Final Settlement: Refund recorded, email receipt dispatched, and audit log stored.
Core takeaway: An AI agent is not just a language model—it is a closed loop of planning, tool invocation, and environment feedback. Production systems require self-healing retries and strict human approval guardrails.
Interactive Concept Check+10 Points
What is AI? Quiz

Which description best fits "narrow AI", the kind of AI in use today?

Zvekutarisa zvinotevera

Tarisa kudzokorora kwakazvimiririra, kusimbiswa kune ramangwana modhi mhuri, uye humbowo hwekuti bvunzo dzakapfupikiswa dzinofanotaura maitiro kunze kwemabhenji avakagadzirirwa kubereka.

Uku kudhindwa kutsva, kwete mutongo wakapedzwa wekuongorora nevezera. Ongororo yemaitiro ekupedzisira akashandisa 134 modhi, iyo vanyori vanocherekedza idiki neyakajairwa psychometric zviyero, uye imwe nzira yekubvisa yakachengeta zvinhu zviviri kwete zvitatu. Mazita emafactory ipfupiso inoyemurika, kwete tsananguro dzakasimbiswa dzekuchengetedza.

Imwe neimwe modhi yakaburitsa mhinduro imwe pane inowanika nekukurumidza kuburikidza neOpenRouter, uye bhenji imwe neimwe yakashandisa yakatemerwa otomatiki mutongi. Miedzo mipfupi inozobuda inogona kugara nhaka kukanganisa mune yekutanga kukurudzira, zvibodzwa mitemo, kuvharwa kwemutauro, uye vatongi; vanyori vanonyatsotaura kuti compression hairatidze kuti bhenji inofanotaura kuchengetedzwa kwekutumirwa kana kuti inoramba ichidzidzisa kune mamwe mamodheru.

Iwo mabheji ejecha bvunzo akashandisa yakakurudzirwa modhi zvipenyu pane mamodheru akakwenenzverwa kunzvenga kuongororwa, izvo zvingave nyore kuona. Iyo API cheki inosimbisa kuenderera kwemaitiro, kwete kuzivikanwa kweakavanzika modhi uremu, uye chidzidzo hachina kuyedza mhandu yakanyatso dzidziswa kukunda nzira dzekuongorora.

Muganhu webepa wakanyanya kukosha uri pakati pekuyera kunyatsoshanda uye vimbiso yekuchengeteka. Chimiro chipfupi chinogona kufungidzira zvinhu zvakavandika zviripo mubhenjimark yepakutanga, asi inotora mutauro weiyo benchmark, mutongi, kukurumidza kugadzira, uye mapofu. Ongororo dzenguva yemberi dzinofanira kuyedza kukurudzira kwemitauro yakawanda, vamiririri vane maturusi, nhaurirano refu, mhando dzakashongedzwa zvakanaka, uye kutonga kwakazvimirira kwevanhu. Vanofanirawo kuzivisa kana chibodzwa chakamanikidzwa chikasagadzikana, nekuti kuwirirana kwakatsetseka pane yakabatwa-kunze data kunogona kuvanza kutadza pane inotevera modhi yemhuri.

Mutemo unoshanda wekurera unotevera kubva kune izvo zvipimo: shandisa bvunzo dzakamanikidzwa sevatariri, kwete mutongo. Matimu anogona kuvamhanyisa kazhinji kuti abate regressions, vobva vakwidza shanduko kune yakazara bhenji uye ongororo yemunhu apo fomu pfupi inofamba zvisingatarisirwi. Humbowo hwechidzidzo chega hunotsigira iyo yakaturikidzana mafambiro nekuti chimiro chechinhu chakatorwa kubva kune zvimwe zvinokurudzira, vatongi, uye modhi zvinobuda. Kutsikisa zvinhu zvakasarudzwa, kodhi yekusarudza, mhedzisiro, uye nhoroondo yeshanduro yaizotendera vaongorori vakazvimiririra vatarise kana bvunzo pfupi dzichiramba dzichidzidzisa mushure mekunge vanogadzira vazviona uye mushure memhando itsva mhuri dzachinja kugoverwa kwemhinduro.

Kujeka kwakafanana kunokosha kana modhi yakagadziridzwa. Muedzo mupfupi wakarongedzerwa pachizvarwa chimwe unogona kuve nyore, kutetepa, kana kuenderana netsaona nehurongwa hutsva. Matimu anofanirwa kuchengetedza shanduro dzekare, kuburitsa kana chinhu kana mutongi achinja, uye kushuma nguva dzekusavimbika pachinzvimbo chekupa chinzvimbo chakamisikidzwa sechiratidzo chekuchengetedza. Iwo maitiro anoshandura mhedzisiro yepepa kuita chirongwa chekutarisa chinotarisika uku uchichengeta bhenji rekutanga riripo kuti riongororwe zvakadzama.

Related guides & Quizzes

Chii chinonzi AI?ChatGPT neLLMsTsika dzeAIEdza zvaunoziva - edza yemahara AI quizTarisa kumusoro izwi reAI mune yedu glossaryTevedza iyo AI regulation tracker
Wakawana izvi zvinobatsira?