Kwenzekeni
Iphepha le-arXiv lethula i-TrustDABench, ibhentshimakhi yokuhlola ukuthi amamodeli ezilimi ezinkulu angakwazi yini ukuhlaziya idatha ehlelekile efana namaspredishithi namafayela e-CSV. Ababhali badale izimo eziphazamisekile eziqinisekiswe abantu ezingu-2,340 ezinhlotsheni eziyi-19 zezinguquko futhi bahlola ama-LLM ayisishiyagalombili. Iphepha libika ukuthi amamodeli ngokuvamile akhiqiza izimpendulo ezizwakalayo kodwa ezingasekelwe, aphuthelwe ubufakazi obungqubuzanayo futhi ashintsha iziphetho zawo lapho ukwakheka kwethebula noma ubudlelwano obuphakathi kwetafula bushintshwa.
Isimangalo esiyinhloko sephepha siwukuthi ukuhlaziywa kwedatha ehlelekile ebukeka kahle kudinga okungaphezu kokwenza ukulandelana kwemisebenzi. Ichaza ukwethembeka endleleni evumelekile ukusuka embuzweni womsebenzisi ukuya ebufakazini bedatha obufanele. Lokho kwenza uhlaka kuholela ekuhlolweni okubili: ukuthi i-LLM iyenqaba yini ukuphendula noma icela ukucaciselwa lapho ingekho indlela yobufakazi obusebenzayo, nokuthi igcina ukuhlaziya okulungile lapho ubufakazi obufanayo buvezwa kwelinye ifomu lethebula. Umthombo wethula lezi zivivinyo njengezilinganiso zebhentshimakhi zokwethembeka nokuqina.
I-TrustDABench yakhiwe ngokuqala ngokubuka indlela yobufakazi nokusebenzisa ama-opharetha aphazamisayo angu-19. Ababhali bathi labo opharetha baqiniswa ngohlaka lwesizukulwane olususelwa ku-Agentic-LLM, lwakhiqiza izehlakalo ezingama-2,340 eziqinisekiswe abantu. I-abstract ayifaki ohlwini lwazo zonke opharetha abangu-19 noma ichaza imithetho yokuqinisekisa ngokuningiliziwe. Iyaqinisekisa ukuthi ibhentshimakhi yayiklanyelwe ukwethula izinguquko ezilawulwayo emisebenzini yedatha ehlelekile kunokumane iqoqe imibuzo nezimpendulo ezijwayelekile. Ababhali bahlole ama-LLM amele ama-LLM ayisishiyagalombili, kodwa i-abstract ayiwahlonzi wonke amasistimu ayisishiyagalombili noma ichaze ukwaziswa kwawo, ukufinyelela kwamathuluzi noma izindawo zokwenza.
Okutholwe yisihloko kungukusebenza okubuthakathaka kuzo zombili izinhlangothi. Umphumela obikwe kahle kakhulu wokuthembeka ube yisilinganiso se-MRS esingu-24.21%, esitholwe yi-GPT-5.5. Umphumela wokuqina okubikiwe ube yi-ASR eyisilinganiso esingu-9.10%, etholwe yi-Claude-Sonnet-5. Umthombo awuwachazi ama-acronyms ku-abstract, ngakho-ke ukubala kwawo okuqondile kanye nesiqondiso kufanele kuqinisekiswe ephepheni eligcwele ngaphambi kokuqhathanisa izinombolo namanye amabhentshimakhi. Ababhali baveza ukwehluleka njengokuhlelekile: amamodeli awavamisile ukubona ubufakazi obungqubuzanayo, ngokuvamile aqhubeka ngezindlela zokuhlaziya ezisebenzisekayo kodwa ezingasekelwe, futhi ahlala ezwela izinguquko ezibandakanya imingcele yokubuka noma ubudlelwano obuphakathi kwetafula.
Imininingwane yomthombo: arxiv.org ↗
Kungani kubalulekile
Ama-LLM asesetshenziswa kakhulu ukuhumusha idatha yebhizinisi, yesayensi neyokuphatha, lapho impendulo eshelelayo ingafihla indlela yokuhlaziya engavumelekile. I-TrustDABench igxile ekutheni imodeli iyazi yini lapho ubufakazi bunganele nokuthi ingabe ifinyelela esiphethweni esifanayo lapho ulwazi olulinganayo lumelelwe ngendlela ehlukile. Imiphumela ebikiwe iphakamisa ukuthi ukudlulisa imisebenzi evamile yokuhlaziya idatha kungase kungabonisi ukucabanga okuthembekile phezu kwamathebula.
Inkinga engokoqobo igebe phakathi kwempendulo engakhiqizwa kanye nempendulo esekelwa idatha. Imodeli ingase ikwazi ukulandela ukulandelana kwe-spreadsheet noma ukusebenza kwethebula kuyilapho iqala emigqeni engalungile, indiva ukuphikisana noma ukweqa ubudlelwano ubufakazi obungathetheleli. Kulezo zimo, ukubulawa ngempumelelo akuqinisekisi isiphetho esivumelekile. I-TrustDABench ilandelana ngoba iqondise lowo mehluko ngokuqondile esikhundleni sokuphatha noma iyiphi impendulo efundekayo noma eyakhiwe ngokwezinombolo njengobufakazi bokwethembeka.
Lokhu kubalulekile noma yikuphi lapho abantu basebenzisa khona ama-LLM ukuze bafingqe noma bahlaziye ulwazi oluhlelekile. Umthombo uqamba ngokuqondile amaspredishithi, amafayela e-CSV nenye idatha ehlelekile, kodwa awukubhali ukuthunyelwa ezimbonini ezithile noma ubike ukulimala komhlaba wangempela. Ubuthakathaka obubikiwe bebhentshimakhi nokho bukhomba inkinga yokubuyekezwa okusebenzayo: abasebenzisi bangase badinge hhayi kuphela inombolo yokugcina noma incazelo, kodwa nokuthi imaphi amarekhodi, imingcele yethebula kanye nobudlelwano bemodeli esetshenzisiwe. Okutholwe yiphepha kusekela leso sexwayiso njengesiphetho socwaningo, hhayi njengesilinganiso esiqinisekisiwe esizimele sawo wonke amasistimu asetshenzisiwe.
Imiphumela iphinda inselele ukuqagela okulula kwemodeli. I-GPT-5.5 ithole amaphuzu aqine kakhulu okuthembeka okubikiwe, kuyilapho i-Claude-Sonnet-5 ithole amaphuzu aqine kakhulu okubikiwe; i-abstract ayisho ukuthi noma iyiphi isistimu ibihamba phambili kuwo wonke umsebenzi noma ukuthi amamethrikhi akala ukuziphatha okufanayo. Ngakho-ke imodeli ingabonakala iqinile kuhlangothi olulodwa kuyilapho ihlale isengozini kobunye. Incazelo ebanzi, ngokusho kwephepha, ukuthi ukuhlaziywa kwedatha ehlelekile okuthembekile kudinga kokubili ukuqashelwa komngcele wobufakazi kanye nokubonisa-okungaguquki kokucabanga, hhayi kuphela amandla amakhulu emisebenzini ejwayelekile.
I-Interactive Mechanism: Indlela Esebenza Ngayo Ngempela
Hlola ubuchwepheshe obuyisisekelo ngemuva kwalokhu kuthuthukiswa ngokuhlanganyela.
What is a common training objective for an autoregressive language model?
Ongakubuka ngokulandelayo
Leli phepha likhomba izinto ezimbili ezibalulekile: ukuqashelwa kangcono kwemingcele yobufakazi kanye nokucabanga okuhlala kuzinzile kuzo zonke izethulo ezihlukene zamathebula. Umthombo uthi ikhodi nedatha kuyatholakala, kodwa awuqinisekisi ukuthi ingabe ibhentshimakhi ibonisa amasethi edatha omhlaba wangempela, ukuthi amamodeli ayisishiyagalombili aqhathaniseka kanjani kuyo yonke imisebenzi ngayinye, noma ukuthi ubuthakathaka obubikiwe buyaqhubeka yini ngemva kokuthuthukiswa okuhlosiwe. Umsebenzi wokulandelela kufanele uhlole idathasethi yangaphandle, ukuhamba komsebenzi kanye nokuziphatha okuyimodeli ngemva kokunciphisa.
Into yokuqala okufanele uyibuke ukuthi ingabe ikhodi yebhentshimakhi kanye nedatha kunika amandla ukukhiqiza kabusha okuzimele. Umthombo uthi “Ikhodi&Data” ziyatholakala, kodwa umbhalo onikeziwe awubandakanyi isixhumanisi sekhosombe, ilayisense yokuma, amafomethi emisebenzi noma izincazelo zamagoli. Leyo mininingwane izonquma ukuthi abacwaningi bangakwazi kanjani ukuhlola kalula iziphazamiso, baqinisekise ukuhlola komuntu futhi baqonde ukuthi izilinganiso ze-MRS ne-ASR zilinganisa ini. Kuze kube lolo lwazi seluhlolisiswa, amaphesenti abikiwe kufanele athathwe njengezimangalo ezisuka kulesi sinyatheliso sangaphambili esisodwa kunezilinganiso zokusebenza eziqediwe.
Umbuzo wesibili ubuqiniso bangaphandle. I-TrustDABench iqukethe izimo eziphazamisekile eziqinisekiswe umuntu, kodwa i-abstract ayisho ukuthi lezo zimo zifana kangakanani namaspredishithi enhlangano angcolile, amapayipi edatha ashintshayo noma ukuhlaziya okwenziwa ngamathuluzi angaphandle. Futhi ayibiki ukuthi imiphumela iyahluka ngosayizi wedatha, isizinda, izinga lokungaqondakali noma uhlobo lokuhlobana kwethebula. Ukuhlola kokulandelela kumasethi edatha azimele kanye nokugeleza komsebenzi okungokoqobo kungasiza ukuthola ukuthi amaphethini okuhluleka okubikiwe aqondene nebhentshimark noma avamile ekusetshenzisweni okungokoqobo.
Okokugcina, abacwaningi nabaqambi kufanele bahlole ukunciphisa kunokwengeza kuphela amaphuzu ebhentshimakhi engeziwe. Iphepha lihlonza ukucaciswa, ukwenqaba, ukutholwa kokungqubuzana nokuzinza kuwo wonke amafomu ethebula njengokuziphatha okubalulekile, kodwa umthombo awubiki ukungenelela okukuthuthukisayo. Izinyathelo ezilandelayo eziwusizo zingabandakanya ukuhlola umkhondo wobufakazi obusobala, ukuhlola okuhlelekile okuphakathi, ukwaziswa kokuphikiswa nokubuyekezwa komuntu ngokumelene neziphazamiso ezifanayo. Umthombo uphinde ushiye kuvuliwe indlela amamodeli aziphatha ngayo lapho idatha ngempela ingenayo impendulo, lapho ukutolika okuningi kubonakala, noma lapho umsebenzisi ecindezela isistimu ukuthi iqhubeke naphezu kokuntuleka kosekelo.