Kwenzekeni
Ithimba lihlole ukuthi ukufinyelela empendulweni ethembekile kuyasiza yini amasistimu e-AI ukuthi aqinisekise ukucabanga okukhiqizwa amanye amamodeli e-AI. Besebenzisa izisombululo ezinenombolo yezinyathelo ezingama-237 zemibuzo ye-physics engu-79, abacwaningi bathola ukuthi impendulo yokufinyelela isiphetho esithuthukisiwe ihlola okungaphezu kokuhlolwa okuzimele kwe-agumenti esekelayo.
Ukuphrinta ngaphambili, okuhanjiswe ku-arXiv ngo-Aug. 31, kuhlola ukuqapha kochungechunge lwemicabango, indlela lapho uhlelo lwe-AI olulodwa luhlola umkhondo wokucabanga okhiqizwa enye. Ababhali baqoqe izixazululo ezingama-237 ezinenombolo yezinyathelo zemibuzo engama-79 ye-Humanity's Last Exam physics kumamodeli amathathu asemngceleni. Ukulandelela bekungenamaphutha afakiwe; abacwaningi babhala ngokuzimele kokubili ukuthi impendulo ngayinye yokugcina ilungile yini nokuthi isinyathelo sokuqala esingamanga senzeke kuphi.
Izichasiselo zenkomba ezihlanganisiwe zefiziksi, inkulumompikiswano ezimele yemodeli yolimi olukhulu kanye nokwahlulela okufihliwe okuvela emthonjeni. Le nqubo ihlonze imikhondo ebucayi engu-24: izixazululo izimpendulo zazo zokugcina zazilungile nakuba ukucabanga kwakuhlanganisa iphutha langempela. Leli phepha lichaza lawa maphutha njengajwayelekile futhi ikakhulukazi angalayishi esikhundleni sokuphikisa ngamabomu, okukhawulela ukuthi imiphumela ingenziwa kanjani jikelele ekuziphatheni okunobutha.
Abaqaphi be-AI abayisishiyagalombili bahlole ukulandelelwa ngaphansi kwezimo ezintathu: ngempendulo engaqinisekisiwe, ngempendulo eqinisekisiwe, noma ngemva kokuzibophezela okungaboni ngaphambi kokubona impendulo. Ngokunikezwa kwesitifiketi, ukunemba okulinganiselayo okushiwo kukhuphuke kusuka ku-0.637 kuya ku-0.796, kuyilapho ukwenziwa kwasendaweni okuyiphutha lokuqala kukhuphuke kusuka ku-0.261 kuya ku-0.379. Lezo zinzuzo azizange zisho ukuthi abaqaphi babe ngcono ngokufanayo ekuqinisekiseni okuzimele.
Umehluko wawucace kakhulu ekukhumbuleni. Ukunikeza isitifiketi kunyuse isabelo sezimpendulo ezingalungile ezihlatshwe umkhosi njengezinephutha zisuka ku-0.653 zaya kokungu-0.951, kodwa okuncishisiwe ukukhumbula emithonjeni ebucayi ukusuka ku-0.521 ukuya ku-0.438. Ngemuva kokuzibophezela okungaboni, abaqaphi babonise impendulo esanda kuhlaba umkhosi ngo-93.8% wezimpendulo ezingalungile eziphasiswe ngaphambilini, uma kuqhathaniswa no-18.0% kuphela wamathrekhi abucayi. Ababhali babika ukuthi bonke abaqaphi abayisishiyagalombili babonise isiqondiso esifanayo sokuqhathanisa, nge-question-bootstrap 95% isikhathi sokuzethemba se- [+0.256, +0.506].
Imininingwane yomthombo: arxiv.org ↗
Kungani kubalulekile
Okutholakele kubekela inselele umcabango ojwayelekile ekuqapheni umcabango olandelanayo: ukuthi impendulo yokugcina eyiyo iwubufakazi obuwusizo bokuthi inqubo yokucabanga ibinengqondo. Ngokwengamela kwe-AI, ucwaningo luphakamisa ukuthi ukuhlola kungadlula amandla okuqapha lapho abaqaphi bevunyelwe ukuthembela emiphumeleni ethenjwayo.
Okushiwo okumaphakathi ukuthi ukuqapha kwe-AI kungenzeka ukuthi uhlola ukuthi umkhondo wokucabanga uyavumelana yini nempendulo kunokunquma ukuthi ukucabanga kuvumelekile yini. Impendulo engalungile idala ukungqubuzana okusobala okungadalula ukufinyelela kwempendulo. Impendulo elungile, ngokuphambene, ingenza umkhondo onephutha ubonakale uthembekile ngisho noma isinyathelo sangaphambili singamanga. Isihloko sephepha siveza lo mehluko: impendulo akuyona ingxabano.
Lokho kubalulekile ekuhloleni kokuphepha okwakhelwe phezu kwemiphumela eyamukelekayo. Uma abaqaphi banikezwa impendulo ethenjwayo yesithenjwa, ukusebenza kwabo okulinganiselwe kungase kubonise ukuhlola ukuvumelana kwesiphetho esikhundleni sokuhlola okuzimele kwenqubo. Ababhali bachaza iphethini njenge-analogue elungile yokugebenga umvuzo: okukhiphayo kungagculisa umhloli kuyilapho inqubo eyikhiqizile ihlala ingenangqondo. Lesi isimangalo esivela kumbhalo wangaphambili, akubona ubufakazi bokuthi amasistimu asetshenzisiwe okwamanje asebenzisa ukucabanga okuyiphutha esimisweni esithile somthelela ophezulu.
Ukukhathazeka okusebenzayo akugcini nje ku-physics noma ekuxazululeni izinkinga okusemthethweni. Noma iyiphi inqubo yokwengamela ephatha umphumela oyimpumelelo njengobufakazi benqubo ephephile ingase ibhekane nendawo eyimpumputhe efanayo, ikakhulukazi lapho ukucabanga okuphakathi kunzima ukukuqinisekisa ngokuqondile. Umthombo awuqinisekisi ukuthi umphumela udluliselwa kwezinye izizinda, kodwa uhlinzeka ngesimo sokuhlola esiphathekayo sokuhlukanisa ukuqinisekiswa komphumela nokuqinisekiswa kwenqubo.
Ezinhlanganweni ezihlola amasistimu e-AI, okutholiwe kukhomba ekubikeni ukufinyelela kwempendulo njengesimo sokuhlola esibalulekile. Imonitha esebenza kahle uma iqhathaniswa nomphumela oqinisekisiwe isengase ibe buthaka ekutholeni amaphutha ekulandeleleni okugcina ngendlela efanele. Lowo mehluko ungase uthinte indlela onjiniyela abahumusha ngayo amaphuzu okuphepha, bakhethe izindlela zokuqapha futhi banqume ukuthi yikuphi ukuhlola okwengeziwe okudingekayo ngaphambi kokuncika emsebenzini owenziwe imodeli.
I-Interactive Mechanism: Indlela Esebenza Ngayo Ngempela
Hlola ubuchwepheshe obuyisisekelo ngemuva kwalokhu kuthuthukiswa ngokuhlanganyela.
Why can ethical evaluation not be reduced to one model score?
Ongakubuka ngokulandelayo
Lo msebenzi uwumbhalo wangaphambili osuselwe kuzisombululo zefiziksi ezivela kumamodeli amathathu asemngceleni, amamonitha ayisishiyagalombili kanye nemikhondo ebucayi engu-24 lapho impendulo yayinembile kodwa ukucabanga kwakuqukethe iphutha. Ukuhlola okwengeziwe kuyadingeka ezihlokweni ezihlukene, amamodeli, izinhlobo zamaphutha nezilungiselelo eziphikisanayo.
Okubaluleke kakhulu okungaziwa ukujwayelekile. Ucwaningo lusebenzisa imibuzo ye-physics, izixazululo ezivela kumamodeli amathathu asemngceleni kanye namamonitha ayisishiyagalombili e-LLM. Umthombo awusho ukuthi ingabe iphethini efanayo iye yaboniswa kuzibalo, ukubhala ngekhodi, ukuhlaziywa kwezomthetho, ukuhamba komsebenzi kwesayensi noma imisebenzi ye-ejenti yomhlaba wangempela. Futhi ayiqinisekisi ukuthi ukusebenza kushintsha kanjani ngemindeni ehlukene yamamodeli, iqaphe ukwaziswa noma ukulandelela amafomethi.
Imikhondo ebucayi engu-24 iwumphumela wempikiswano yephepha, kodwa iyingxenye encane yezixazululo eziqoqiwe. Umthombo awunikezi ulwazi olwanele ukuze unqume ukuthi lokho kulandelelwa kuyimela kanjani yazo zonke izixazululo ezilungile noma ukuthi umphumela ungashintsha kanjani ngesampula enkulu nehluke kakhulu. Ngakho-ke abafundi kufanele baphathe amaphesenti abikiwe njengobufakazi obuvela kulokhu kuhlola, hhayi njengamazinga ajwayelekile kubaqaphi be-AI.
Abacwaningi basho ngokusobala ukuthi amaphutha afundwayo ayejwayelekile, ngokuvamile ayengathwali mthwalo futhi engewona aphikisayo. Lokho kushiya kuvuliwe ukuthi ingabe ukuzama ngamabomu ukufihla iphutha kungenza ukuqapha okwazi impendulo kuphumelele noma kungasebenzi kahle. Umthombo futhi awubiki imiphumela yokuthunyelwa, ukuqhathaniswa kokuqapha komuntu noma imiphumela kusistimu yokusebenza.
Umsebenzi wesikhathi esizayo kufanele uhlole izimo zokuqapha lapho umhloli engayitholi impendulo ethembekile, kufanele azibophezele ngaphambi kokubona umphumela, noma asebenzise amasheke azimele ezimangalo ezimaphakathi. Kufanele futhi ihlole ukuthi ingabe abaqaphi bangakwazi yini ukwehlukanisa ngokuthembekile isiphetho esiyiso esifinyelelwe ngesizathu esizwakalayo kuleso esifinyelelwe ngendlela enephutha. Kuze kube yilapho leyo mibuzo iphendulwa, amaphuzu aphezulu ekuqashweni kokwazi impendulo akufanele afundwe njengobufakazi bokuthi inqubo yokucabanga yesistimu ye-AI izwakala.