Kwenzekeni
Iphepha locwaningo elithunyelwe ngo-Agasti 5 lisebenzisa indlela esuka ekuhlolweni kwezemfundo ukukala ukuthi izilinganiso zokuphepha ze-AI zithwebula ini, khetha amasethi amancane okwaziswa okuwusizo, kanye nezinguquko zokuhlola ekuziphatheni okuyimodeli.
Abacwaningi ababili abazimele kanye nabacwaningi ababili abaxhumene ne-UK AI Security Institute bahlole amamodeli engxoxo angu-192 kumabhentshimakhi ayisishiyagalombili ahlanganisa ukwenqaba kwezicelo eziyingozi, ukwenqaba ngokweqile kwezicelo ezinohlonze, ukulinyazwa komongo, nokuba neqiniso. Uhlu oluphelele luqukethe imiyalo engu-5,255 ngaphambi kokucubungula kusengaphambili; ukuhlaziya kugcine okungu-5,067 ngemva kokukhipha izimpendulo ezingaqinisekisiwe nezinto ezingazange zihlukanise phakathi kwamamodeli ahloliwe.
Ithimba liphathe amamodeli njengabahloli kanye nemiyalelo yebhentshimakhi njengezinto zokuhlola, lisebenzisa ithiyori yempendulo yento ukuze lilinganisele ukuthi iziphi iziyalezo ezazinzima nokuthi yimaphi amamodeli ahlukaniswe kangcono. Ekuhlaziyweni kwaso okuyinhloko, isixazululo sezinto ezintathu—okufingqiwe njengokuqina kokwenqaba, ukuba yiqiniso, kanye nokulimala komongo—sichaze i-77% yokuhlukahluka kwamakhono amamodeli, uma kuqhathaniswa ne-47% yesici esisodwa.
Kuzo zonke izivivinyo ezibanjiwe ezingama-20, izivivinyo ezintathu ezingaguquki ezisheshayo ezingama-25 zibuyise lezo zici ezintathu zisebenzisa ngaphansi kuka-2% we-suite. Okwe-HarmBench, SORRY-Bench, kanye ne-OR-Bench-Hard, cishe ukwaziswa okungu-10 okukhethwe ngokuguquguqukayo kuphinde kwakhiqiza amazinga ebhentshimakhi aphelele ngokuhlotshaniswa kwe-0.92 kuya ku-0.94 futhi yehlisa inani lokwaziswa ngo-97–99%; inzuzo inciphile njengoba isabelomali sokuhlola sikhula.
Ukuminyanisa akukona nje ukusampula okungahleliwe. Ithiyori yempendulo yento ilinganisela ukuthi umlayezo ngamunye unzima kangakanani nokuthi uwahlukanisa kahle kanjani amamodeli anamaphethini okuphendula ahlukene, bese ikhetha izinto ezigcina ukwakheka kohlolo olukhulu. Ababhali baqhathanise amafomu amafushane angaguquki nokukhetha okuguquguqukayo futhi babambe ukuhlaziya kunokulinganisa kuphela idatha esetshenziselwa ukukhetha ukwaziswa. Lowo mklamo usekela isimangalo esincane sokuthi amanye amazinga ebhentshimakhi akhona angenziwa kabusha ngokuphumelelayo; ayibonisi ukuthi ukuhlolwa kwezinto eziyi-10 noma ezingama-25 kungathola imodi yokuhluleka entsha ngokuphelele.
Imininingwane yomthombo: Rivera and colleagues' research paper on arXiv ↗
Kungani kubalulekile
Ucwaningo luphakamisa ukuthi ukwaziswa okukhethwe kangcono kungenza ukuhlola okuvamile kokuphepha kwe-AI kushibhe ngaphandle kokushaya sengathi yonke ibhentshimakhi ikala umphumela owodwa wokuphepha wendawo yonke.
Ukuhlola okushibhile kungavumela onjiniyela, amalabhorethri amancane, nabahloli abazimele ukuthi bahlole ezinye izinguqulo zamamodeli phakathi nokuqeqeshwa, ukulungisa kahle, ukulinganisa, kanye nezinguquko ezisheshayo zesistimu. Umphumela umayelana nokukhiqiza kabusha izilinganiso zebhentshimakhi ezikhona ngokuphumelelayo, hhayi ukwandisa lokho okungatholwa yilawo mabhentshimakhi.
Ukuhlaziywa kwesici futhi kuveza ukuhwebelana kwamaphuzu. Amabhentshimakhi anikeza ukwenqatshwa komklomelo angadlulana, kuyilapho ibhentshimakhi yokugwema ukwenqaba okungadingekile ivuza ukuziphatha okuphambene. Ukuzilinganisa ngaphandle kokuchaza izisindo kungafihla ukuthi imodeli iphephe kakhudlwana, ikhawule kakhulu, noma isebenziseke kancane ezicelweni ezinobulungiswa.
Leli phepha liphinde lahlola izindlela zokuhlola ibhokisi elimnyama ekuziphatheni okugqugquzelwe ngamabomu. Isheke lephethini yokuphendula libanjwe ukusebenza ngaphansi okukhethiwe kodwa lageja ukusebenza okungaphansi komfaniswano omningi; ukwengeza ukuhlola ikhono lomlando kukhuphule ukutholwa okuhlanganisiwe kwaba ngu-94–97% kuzo zonke izimo ezilingisiwe. Lokho kusekela ukuhlola okunengqimba kunokuthembela kumphumela owodwa.
Konjiniyela, inani lokusebenza imvamisa. Ukuhlola okufushane kungenziwa kabusha ngemva koshintsho lokwaziswa kwesistimu, iphasi yokulinganisa, ukushuna kahle, isibuyekezo samathuluzi, noma ukubuyekezwa kwenqubomgomo yokuphepha ngaphandle kokusebenzisa isabelomali sebhentshimark egcwele isikhathi ngasinye. Kodwa umphumela uba usizo kuphela lapho amaqembu egcina i-suite yasekuqaleni njengocwaningo lwezikhathi ezithile, ezungeza imiyalo ebanjiwe, futhi ephenya izinguquko ezimangalisayo esikhundleni sokulungiselela ukuhlolwa okucindezelweyo. Uma kungenjalo isheke elishibhile lingaba elinye impokophelo yokugcwala ngokweqile.
I-Interactive Mechanism: Indlela Esebenza Ngayo Ngempela
Hlola ubuchwepheshe obuyisisekelo ngemuva kwalokhu kuthuthukiswa ngokuhlanganyela.
crm_get_transaction(id='4092').Which description best fits "narrow AI", the kind of AI in use today?
Ongakubuka ngokulandelayo
Buka ukuphindaphinda okuzimele, ukuqinisekiswa kwemindeni eyimodeli yesikhathi esizayo, nobufakazi bokuthi ukuhlola okufushanisiwe kubikezela ukuziphatha ngaphandle kwamabhentshimakhi abeklanyelwe ukuwakhiqiza.
Lesi ukuphrinta kusengaphambili okusha, akusona isinqumo esiqediwe sokubuyekezwa kontanga. Ukuhlaziywa kwesakhiwo esifihlekile kusebenzise amamodeli we-134, ababhali abaqaphela ukuthi mancane ngezindinganiso ezijwayelekile ze-psychometric, futhi enye indlela yokukhipha igcine izici ezimbili kunezintathu. Amagama wesici ayizifinyezo ezisetshenziswayo, hhayi izincazelo eziqinisekisiwe zokuphepha.
Imodeli ngayinye ikhiqize impendulo eyodwa ngesikhathi ngasinye esitholakalayo nge-OpenRouter, futhi ibhentshimakhi ngayinye yasebenzisa ijaji layo elizenzakalelayo elinqunyiwe. Uhlolo olufushane oluwumphumela lungazuza njengefa amaphutha ekwazisweni kwasekuqaleni, imithetho yokufaka amaphuzu, ukusabalala kolimi, namajaji; ababhali basho ngokusobala ukuthi ukucindezela akubonisi ukuthi ibhentshimakhi ibikezela ukuphepha kokuthunyelwa noma ihlala inolwazi ngamamodeli azayo.
Ukuhlolwa kwe-sandbagging kusebenzise amamodeli ezinto eziphilayo aziswayo kunamamodeli ashunwe kahle ukuze agweme ukuhlolwa, okungaba lula ukukubona. Ukuhlolwa kwe-API kusungula ukuqhubeka kokuziphatha, hhayi ubunikazi bemodeli yesisindo efihliwe, futhi ucwaningo aluzange luhlole isitha esiqeqeshwe ngokukhethekile ukuze sinqobe izindlela zokucwaninga.
Umngcele wephepha obaluleke kakhulu uphakathi kokusebenza kahle kokulinganisa kanye nesiqiniseko sokuphepha. Ifomu elifushane lingalinganisa izici ezicashile ezikhona kubhentshimakhi yoqobo, kodwa lizuza njengefa lolimi lwebhentshimakhi, ijaji, uzimele osheshayo, namabala angaboni. Ukuhlola okuzayo kufanele kuhlole imiyalo yezilimi eziningi, ama-ejenti anikwe amandla ngamathuluzi, izingxoxo ezinde, amamodeli ashunwe kahle aphikisayo, nokwahlulela kwabantu abazimele. Kufanele futhi babike lapho isikolo esicindezelwe singazinzi, ngoba ukuhlobana okuhle kwedatha ebanjiwe kungafihla ukwehluleka kumodeli olandelayo womndeni.
Umthetho wokutholwa osebenzayo ulandela kuleyo mikhawulo: sebenzisa izivivinyo ezicindezelwe njengabalindi, hhayi izinqumo. Amaqembu angawagijimisa njalo ukuze abambe ukuhlehla, bese ekhuphula ushintsho kubhentshimakhi ephelele kanye nokubuyekezwa komuntu lapho ifomu elifushane lihamba ngokungalindelekile. Ubufakazi bocwaningo uqobo busekela lokho kugeleza komsebenzi okunezingqimba ngoba ukwakheka kwesici kuthathwe ekwazisweni okuthile, amajaji, kanye nemiphumela yemodeli. Ukushicilela izinto ezikhethiwe, ikhodi yokukhetha, imiphumela ebanjiwe, nomlando wenguqulo kuzovumela abahloli abazimele ukuthi bahlole ukuthi ukuhlola okufushane kuhlala kunolwazi yini ngemva kokuba onjiniyela bekubona nangemva kokuthi imindeni yamamodeli amasha iguqule ukusatshalaliswa kwezimpendulo.
Ukubonisa ngale okufanayo kubalulekile lapho imodeli ibuyekezwa. Uhlolo olufushane olulinganiselwe esizukulwaneni esisodwa lungaba lula kakhulu, lube mncane kakhulu, noma luqondaniswe ngephutha nomyalo wesistimu entsha. Amaqembu kufanele alondoloze izinguqulo zangaphambili, adalule lapho into noma ijaji lishintsha, futhi abike izikhathi zokuzithemba esikhundleni sokwethula izinga elicindezelwe njengomphumela wokuphepha onembile. Lezo zinqubo zishintsha umphumela wephepha ube uhlelo lokuqapha olufundekayo kuyilapho kugcinwa ibhentshimakhi yasekuqaleni itholakala ukuze ibuyekezwe ngokujulile.