Kwenzekeni
Iphepha le-arXiv lethula i-HealthBench-Psych ne-HealthBench-Psych-Hard, izinsiza ezimbili okuhloswe ngazo ukuhlukanisa indlela amamodeli olimi asebenza ngayo ezingxoxweni eziphathelene nempilo yengqondo. Ababhali bahlole izingxoxo zodokotela-rubrikhi ezingu-5,000 ezivela ku-HealthBench, baqinisekisa isigatshana esifanelekile ngokusebenzisa imizuliswano emibili yokubuyekezwa kodokotela abangaboni, futhi bakhombe izingxoxo ze-610. Bahlole amamodeli angu-20 emngceleni navulekile besebenzisa amajaji amathathu e-LLM, babika iqoqo lemingcele eliboshwe ngokwezibalo, ukuziphatha okungenakulinganiswa kokunqaba kumamodeli amabili kanye namazinga acishe afane kuwo wonke amajaji.
Iphepha, elithunyelwe ku-arXiv ngo-Agasti 25, 2026, lethula i-HealthBench-Psych njengengxenye yezempilo yengqondo ye-OpenAI's broader HealthBench. Ababhali bathi amabhentshimakhi ezempilo ajwayelekile awahlali ehlukanisa imiphumela ngokukhethekile komtholampilo, okwenza kube nzima ukuhlukanisa ukusebenza empilweni yengqondo. Impendulo yabo iyisisetshenziswa sokuhlola esikhethekile kunemodeli entsha yolimi noma ukwethulwa komkhiqizo. Umthombo uchaza insiza yesibili, i-HealthBench-Psych-Hard, kodwa i-abstract ayichazi ukuthi ihluke kanjani ku-subset eyinhloko noma ukuthi yakhiwe kanjani.
Ababhali bahlole izingxoxo zerubhrikhi kadokotela ezingu-5,000 zokuhambisana nempilo yengqondo nalokho abakuchaza njengerubrikhi esobala esetshenziswa yi-LLM. Babe sebefaka into ekhethiwe emizuliswaneni emibili yokubuyekezwa kodokotela abangaboni. Ukubuyekeza kufaka phakathi izilawuli ezifihliwe ezaziwayo-azifaki, ezivezwa umthombo njengengxenye yenqubo yokuqinisekisa. Lokhu kukhiqize izingxoxo ezingama-610, noma u-12.2% wekhophasi yasekuqaleni. I-abstract ayisho ukuthi yiziphi izimo, izimpawu, amaqembu abasebenzisi, izilimi noma izinhlobo zezicelo ezivela kulezo zingxoxo, ngakho-ke ukumbozwa kwesethi engaphansi akukwazi ukuhlolwa kusukela emthonjeni wodwa.
Abacwaningi bahlole umngcele we-20 namamodeli avulekile ngephaneli yomthengisi owela amajaji amathathu e-LLM. Babika iqoqo lomngcele eliboshwe ngokwezibalo, okusho ukuthi i-abstract awethuli owinile oyedwa ocacile phakathi kwamasistimu ahamba phambili ngaphansi kwalokhu kuhlola. Baphinde babike ukuziphatha kokwenqaba okulinganisekayo kumamodeli amabili, kodwa abacacisi ukuthi yimaphi amamodeli anqabile, iziphi izinhlobo zezaziso ezibangele ukwenqaba noma ukuthi lokho kunqaba kuthathwe njengokufanelekile. Izinga elibikiwe licishe lifane kuwo wonke amajaji amathathu, kanti i-tau ka-Kendall yayiku-0.92 noma ngaphezulu. Leli phepha lithi likhipha i-subset, ipayipi lokuhlola, izimpendulo zemodeli, amabanga kanye nekhodi yokuhlaziya njengesisetshenziswa esisebenziseka kabusha. Lokho kukhululwa kungavumela abanye abacwaningi ukuthi bakhiqize kabusha izingxenye zokuhlola futhi baqhathanise amasistimu engeziwe. Kodwa-ke, umbhalo owumthombo awunikezi izixhumanisi ezingahlolwa lapha, futhi awuchazi imikhawulo yokufinyelela, amalayisense, ukuvikelwa kobumfihlo noma noma yiziphi izingxoxo ezakhiwe ngokuzenzakalelayo, azikhonjwanga noma zithathwe kubantu abathile. Leyo mininingwane ibalulekile ngoba idatha yokuhlola impilo yengqondo ingabandakanya okuqukethwe okubucayi.
Imininingwane yomthombo: arxiv.org ↗
Kungani kubalulekile
Ukusebenza kwempilo yengqondo kungafihlwa lapho amamodeli ehlolwa kuphela ngezilinganiso ezibanzi zezokwelapha. Idathasethi egxile okukhethekile, esebenziseka kabusha ingenza ukuqhathanisa kucace kakhudlwana futhi kusize abacwaningi namasistimu onjiniyela okuhlola ahloselwe ukwesekwa kwengqondo. Ukukhishwa nakho kuyaphawuleka ngoba kuhlanganisa isethi engaphansi, ipayipi lokuhlola, izimpendulo zemodeli, amabanga nekhodi yokuhlaziya kunokubika kuphela okutholiwe okuhlanganisiwe. Iphepha aliqinisekisi ukuthi noma iyiphi imodeli ehloliwe iphephile noma iyasebenza ukuze isetshenziswe emtholampilo.
Inani elimaphakathi lomsebenzi isilinganiso. Amabhentshimakhi wezokwelapha abanzi angaveza amaphuzu aphelele ngenkathi efihla ukuhluka okunengqondo phakathi kwezinto ezikhethekile. Ngokuhlukanisa izingxoxo ezihlobene nempilo yengqondo, i-HealthBench-Psych inganikeza abathuthukisi nabahloli indlela egxile kakhulu yokubuza ukuthi ingabe imodeli iphendula ngokufanelekile yini ezicelweni zokusekelwa kwengqondo. Lokho akwenzi ibhentshimakhi ibe ukuhlolwa komtholampilo, kodwa kungase kuthuthukise ukubonakala kube ingxenye yemodeli yokuziphatha ehlanganiswa nemisebenzi yezokwelapha engahlobene. Insiza ingase futhi ibe usizo ngoba yakhelwe ukuhlanganiswa ekugelezeni komsebenzi kanjiniyela.
Ababhali abakhiphi izingxoxo kuphela kodwa futhi nenqubo yokuhlola, izimpendulo eziyimodeli, amamaki kanye nekhodi yokuhlaziya. Uma lezo zinto zibhalwe ngokwanele futhi zikwazi ukukhiqizwa kabusha, abacwaningi bangasebenzisa ukusetha okufanayo kokuhlola kuwo wonke amamodeli ahlukene noma izinguqulo zamamodeli. Ibhentshimakhi eyabiwe ingenza izinguquko kube lula ukuqhathanisa, nakuba umthombo ungaqinisekisi ukuthi ukukhishwa kwamukelwe kabanzi kangakanani noma ukuthi amaqembu angaphandle akhiqize kabusha imiphumela yawo. Ukutholwa kweqoqo lomngcele oboshelwe ngokwezibalo kuyisexwayiso ngokumelene nokuphatha isikolo sebhentshimakhi esisodwa njengobufakazi bokuthi imodeli eyodwa ehamba phambili ingcono ngokwezigaba ukusebenzisana nempilo yengqondo. Umphumela ungase ubonise ukuthi amasistimu emingcele ahloliwe asebenze ngendlela efanayo ngaphansi kwalesi sethi yengxoxo ethile, irubrikhi nokuhlelwa kokwahlulela.
Amazinga acishe afane kuwo wonke amajaji aqinisa ukuvumelana koku-oda okubikiwe, kodwa awafakazi ngokuzimele ukuthi izindinganiso zamajaji zivumelekile ngokomtholampilo noma ukuthi amazinga abikezela imiphumela yabantu abafuna ukusekelwa. Ukuziphatha okubikiwe kokwenqaba nakho kunokubaluleka okungokoqobo kodwa kusalokhu kungacaciswanga. Ukwenqaba kungase kufaneleke kwezinye izicelo futhi kungabi usizo kwabanye, ikakhulukazi ezingxoxweni ezibucayi lapho uhlelo kufanele luhlukanise phakathi kokusekelwa okujwayelekile ngokomzwelo, ingozi ephuthumayo kanye nezicelo ezidinga ukunakekelwa kochwepheshe. Umthombo uthi amamodeli amabili abonise impatho yokwenqaba elinganisekayo, kodwa awusho ukuthi ukwenqaba kuthuthukisa ukuphepha, ukunciphisa ukusetshenziswa noma ukwahlulela okuhambisanayo kodokotela. Akukho emthonjeni okubonisa ukuthi noma iyiphi imodeli kufanele isetshenziswe esikhundleni sikachwepheshe oqeqeshiwe wezempilo yengqondo.
I-Interactive Mechanism: Indlela Esebenza Ngayo Ngempela
Hlola ubuchwepheshe obuyisisekelo ngemuva kwalokhu kuthuthukiswa ngokuhlanganyela.
Which component of an AI application is the machine-learning model itself?
Ongakubuka ngokulandelayo
Isinyathelo esilandelayo esibaluleke kakhulu ukuhlola okuzimele kanye nokusetshenziswa kabusha kwezinto ezikhishiwe. Abafundi bazodinga amaphuzu emodeli ngemodeli, izihloko ezithile zempilo yengqondo ezimelwe, irubhrikhi nokwaziswa kwejaji, nobufakazi bokuthi ibhentshimark ikala ukuziphatha okunenjongo ngokomtholampilo. Umthombo awuwahlonzi amamodeli angu-20, ubike amaphuzu anemininingwane noma uchaze ukubaluleka kokutholwa kokunqaba. Ukuhlola okwengeziwe kufanele kuhlole ukuthi ingabe amazinga ayaqhubeka yini kuzo zonke izinguqulo zamamodeli, izilimi, inani labantu kanye nezingxoxo zomhlaba wangempela.
Abasebenzisi abazimele bokukhishwa kufanele baqale bahlole ukwakheka kwezingxoxo ezingu-610 kanye nendlela yokunquma esetshenziselwa ukuzihlukanisa njengezifanele. Imibuzo ebalulekile ihlanganisa ukuthi ingabe izibonelo zifaka izimo ezibucayi, ukwesekwa ngokomzwelo okujwayelekile, imibuzo ehlobene nokuxilongwa, imibuzo yokwelashwa kanye nolimi oluhlukene ngokwamasiko. Umthombo awuyiphenduli le mibuzo. Ngaphandle kwalolo lwazi, umphumela ophezulu noma ophansi ungase ubonise inhlanganisela yesihloko sebhentshimakhi kunekhono elibanzi lempilo yengqondo.
Ukuhlola kufanele futhi kubhekwe ukuthi imodeli kanye nokungafihli lutho kwejaji. Amagama angabonakali akuwona amamodeli ahloliwe angu-20 noma izinguqulo zawo, futhi awanikezi amaphuzu angawodwana, izikhawu zokuzethemba, ukwaziswa kwejaji noma inqubo yezibalo ngemuva kweqoqo lomngcele oboshiwe. Lokho kweqiwa kuvimbela abafundi ekunqumeni ukuthi ngabe umehluko omncane wokusebenza ubunenjongo, noma amamodeli ahlolwe ngaphansi kwezilungiselelo eziqhathanisekayo noma ukuthi amajaji athanda izitayela ezithile zokuphendula. Ubumfihlo bokukhishwa kanye nemininingwane yokuphatha idinga ukunakwa. Ngenxa yokuthi ibhentshimakhi iphathelene nempilo yengqondo, abasebenzisi bazodinga ukwazi ukuthi izingxoxo zithathwe kanjani, noma ngabe ziqukethe ulwazi oluhlonza umuntu siqu, ukuthi izinto ezibucayi zasingathwa kanjani nokuthi imigomo yokusebenzisa kabusha ivumela ini. Umthombo uqinisekisa ukuthi izimpendulo zemodeli, amamaki kanye nekhodi yokuhlaziya ziyakhululwa, kodwa azichazi izivikelo noma imikhawulo. Leyo mininingwane kufanele ibe khona ngaphambi kokuba izinhlangano zisebenzise ibhentshimakhi ukwenza izinqumo zokusatshalaliswa noma zokuphepha.
Ucwaningo lwesikhathi esizayo kufanele luhlole ukuthi ingabe okutholakele okubikiwe kubambe ngaphandle kwale dathasethi kanye nephaneli yokwahlulela. Izandiso eziwusizo zizofaka ukuhlolwa kodokotela abazimele, ukuhlolwa kwabantu okuqondile, imindeni yamamodeli eyengeziwe, izilimi eziningi kanye nokuhlolwa kwe-longitudinal kuzo zonke izibuyekezo zemodeli. Umthombo usungula insiza entsha yokuma futhi ubike imiphumela yokuqala; ayisunguli ukusebenza kahle komtholampilo komhlaba wangempela, imiphumela ethuthukisiwe yomsebenzisi noma ukuphepha kokusebenzisa noma iyiphi imodeli kumasevisi ezempilo yengqondo.