Kwenzekeni
Ukuphrinta kusengaphambili ku-arXiv ngo-Agasti 24 kwethula ama-Credal Large Language Models, noma ama-CLLM, asebenzisa iqoqo lama-adaptha e-LoRA ukuze amele uhla lokuqagela okubambekayo esikhundleni sokusatshalaliswa kwamathuba okukodwa. Ababhali bathola izikolo zokuzibophezela zeleveli yethokheni kanye neleveli ye-semantic futhi bawahlolele ukuphendula imibuzo, ukulinganisa, ukubikezela okukhethiwe, ukutholwa kokubona izinto ezingekho, nokucabanga.
Leli phepha lichaza umkhawulo ngendlela evamile amamodeli olimi amelela ngayo ukungaqiniseki: imodeli evamile ikhiqiza ukusabalalisa okuqagelayo okukodwa, ababhali abathi kungahlanganisa ukungazi nokungaqondakali kwangempela. I-CLLM yabo ehlongozwayo esikhundleni salokho isebenzisa inhlanganisela yama-adaptha e-LoRA ukwenza lokho iphepha elikubiza ngokuthi isethi yobufakazi. Ngamagama asebenzayo, indlela ihloselwe ukulondoloza ukungaboni ngaso linye noma ukusabalala phakathi kokusabalalisa okuqagelayo okuzwakalayo kunokucindezela konke ukungaqiniseki kokuphumayo okukodwa kwe-softmax. Umthombo awusho ukuthi lesi sifanekiso senza imodeli ilungile; ithi ingenza izinga lokuzibophezela kwemodeli libe nolwazi oluthe xaxa.
Ababhali bethula izinyathelo ezimbili ezihlobene. I-Credal Token Commitment, noma i-CTC, isebenza endaweni yamathokheni futhi ihlanganisa ukusekela okunesibopho esiphansi, ububanzi bekhredithi, kanye ne-entropy ye-intersection. I-abstract ithi i-CTC ingenziwa ikhompuyutha ngaphandle kwesizukulwane esengeziwe, okungenzeka sibalulekile ezinhlelweni lapho amasampula aphindaphindiwe angangeza ukubambezeleka noma izindleko. I-Semantic Commitment Consistency, noma i-SCC, inweba umqondo endaweni ye-semantic kusetshenziswa ukuqedwa okuyisampula. Leli phepha liphinde lichaze i-SCC-Gap ukukala ukungafani phakathi kokusekelwa kweleveli yamathokheni kanye nokusekelwa kwezinga le-semantic. Lawa maphuzu ethulwa njengamathuluzi okuhlonza lapho ukuzethemba kweleveli ephezulu kwemodeli kungase kungahambisani nobubanzi bezincazelo ezivezwa izimpendulo zayo ezingaba khona.
Ukuhlola kufaka i-Gemma-2-9B, Llama-3.1-8B, ne-Qwen2.5-7B ku-OpenBookQA, CoQA, TriviaQA, kanye ne-ARC-Challenge. Ngokusho kwe-abstract, i-CLLM iyindlela esebenza kahle kakhulu ekuphenduleni imibuzo ngokunemba ngenkathi igcina iphutha lokulinganisa elilindelekile lokuncintisana. Ababhali baphinde babike ukuthi i-CTC ifika phakathi kwamaphesenti angu-1.5 wendawo engcono kakhulu yokuthola i- ngaphansi kwejika lesici sokusebenza komamukeli kuzilungiselelo eziningi, ngaphandle kwesizukulwane esengeziwe. Ekuqaguleni okukhethiwe ekufakweni okungu-80%, i-abstract ibika ukunemba okungu-99.0% ku-OpenBookQA ye-CLLM ne-SCC. Iqala ukusho umphumela we-ARC-Challenge ye-CLLM ngokuzethemba kwe-Csem kodwa iyancishiswa ngaphambi kokunikeza umphumela, ukuze leso simangalo singakwazi ukuhlaziya emthonjeni onikeziwe.
Imininingwane yomthombo: arxiv.org ↗
Kungani kubalulekile
Amamodeli olimi angaveza izimpendulo ezishelelayo ngokuzethemba okungadingekile. Uma imiphumela ebikiwe ibambezela, ukulinganisa ukungaqiniseki kuzo zonke izibikezelo ezisabalalisa eziningi ezizwakalayo kungasiza amasistimu akhombe izimpendulo ezidinga ukuqinisekiswa, ukuziyeka, noma ukubuyekezwa komuntu ngaphandle kokudinga ukukhiqiza okwengeziwe ezimeni eziningi.
Inkinga esemqoka akuyona nje ukuthi imodeli yolimi ingawuphendula umbuzo, kodwa ukuthi ingakwazi yini ukwehlukanisa ulwazi nokungaqiniseki. Impendulo eshelelayo kodwa engalungile ingaba yingozi kakhulu kunokwenqaba okusobala lapho abasebenzisi bephatha ukuzethemba njengobufakazi. Isethulo esihlongozwayo sobufakazi bephepha silungisa leyo nkinga ngokugcina ukungaboni ngaso linye phakathi kwezibikezelo ezisekelwe ku-adaptha. Uma indlela ijwayele, inganikeza onjiniyela isignali eseceleni yemodeli yokunquma ukuthi baphendule nini ngokuqondile, bacele ukuqinisekiswa, bahambise umbuzo kwenye isistimu, noma babandakanye umuntu.
Umphumela obikiwe wokuqagela okukhethiwe ubaluleke kakhulu ekusetshenzisweni ngoba amasistimu akhethiwe awadingi ukuphendula yonke imibuzo. Isistimu engagcina ukunemba okuphezulu kuyilapho ihlanganisa kuphela izimo ecabanga ukuthi zisekelwe ngokwanele ingase ibe usizo kakhulu kuzilungiselelo lapho amaphutha ethwala izindleko ezizwakalayo. Ukunemba okubikwayo okungu-99.0% ekufakweni okungu-80% ku-OpenBookQA kuyakhuthaza kuleyo dathasethi nokucushwa, kodwa kufanele kuqondwe njengomphumela wephepha kunobufakazi bokuthi isistimu esetshenzisiwe izozuza ukusebenza okufanayo. I-abstract ayicacisi inani lezibonelo, izindlela zokuqhathanisa, noma incazelo yokusebenza yekhava.
Isimangalo sesizukulwane esingangeziwe se-CTC singaba nendaba ekwakhiweni kwemibono. Amasu amaningi okungaqiniseki ancike ekukhiqizeni ukuqedwa okuningi, okungakhuphula ukubala nokubambezeleka. Umthombo uthi i-CTC ihlanganisa amanani amaningana ahlobene nokungaqiniseki ngaphandle kokukhiqiza okwengeziwe, kuyilapho i-SCC isebenzisa ngokusobala ukuqedwa okuyisampula. Lowo mehluko unikeza iphepha i-engeli ekhonkolo yobunjiniyela: amaphuzu athile angase ashibhe ukuwasebenzisa, kuyilapho elinye lingase lithwebule ukungezwani kwe-semantic ngokuqondile. I-abstract ayizibali izindleko ze-LoRA ensemble ngokwayo, nokho, ngakho-ke inani eliphelele lokuthengiselana alikaziwa.
I-Interactive Mechanism: Indlela Esebenza Ngayo Ngempela
Hlola ubuchwepheshe obuyisisekelo ngemuva kwalokhu kuthuthukiswa ngokuhlanganyela.
What is a common training objective for an autoregressive language model?
Ongakubuka ngokulandelayo
Umphumela okwamanje uwukuhlola kokuphrinta kwangaphambili okubikwe ngumlobi, hhayi ukutholwa kokusebenza okusungulwe ngokuzimele. Imininingwane ebalulekile isalokhu ingatholakali ku-abstract enikeziwe, okuhlanganisa umphumela ogcwele we-ARC-Challenge, izisekelo eziqondile, i-computitional overhead, nokuthi indlela idluliswa kahle kangakanani ngale kwamamodeli ahloliwe namasethi edatha.
Umbuzo wokuqala ukuthi ingabe izinzuzo ezibikiwe ziyasinda yini ekuphindaphindeni okuzimele. Umthombo uyi-arXiv ephrintiwe ngaphambilini ethunyelwe ngo-Agasti 24, 2026, futhi okokusebenza okunikeziwe kuqukethe kuphela i-abstract. Ngakho-ke imiphumela izimangalo ezenziwe ababhali, hhayi amaqiniso aqinisekisiwe ngokuzimela. Ukucutshungulwa kokulandelela kufanele kuhlole amathebula aphelele, izisekelo, izincazelo zokuzethemba, ukuhlukahluka kwezibalo, nokuthi ingabe izinzuzo ezibikiwe ziyahambisana yini kuwo wonke amasethi wedatha kanye nayo yomithathu imindeni eyimodeli.
Umusho we-ARC-Challenge we-abstract awuphelele: uthi i-CLLM enokuzithemba kwe-Csem ithola umphumela kodwa ayinikezi inani. Lolo lwazi olungekho lukhawulela ukuqhathaniswa nesimangalo esiphelele se-OpenBookQA futhi luvimbela ukuhlolwa okugcwele kokusebenza kwendlela yokucabanga. Iphepha liphinde libike umphumela wokutholwa kombono ongekho mayelana nokuba phakathi kwamaphesenti angu-1.5 amaphuzu e-AUROC ehamba phambili kuzilungiselelo eziningi, kodwa umthombo onikeziwe awukhombi amaphuzu aphelele, izindlela eziqhudelana kahle kakhulu, noma okuhlukile.
Imibuzo yokusatshalaliswa ibalulekile ngokufanayo. Iphepha lisebenzisa iqoqo lama-adaptha e-LoRA, futhi i-abstract ayisho ukuthi mangaki ama-adaptha adingekayo, aqeqeshwe kanjani, noma inkumbulo nesikhathi esingakanani abengezayo. Iphinda ihlole amamodeli ezilimi amathathu kuphela aqanjwe amagama kanye namasethi edatha okuphendula imibuzo emine. Akwaziwa ukuthi noma amaphuzu asebenzela izingxoxo ezinde, isizukulwane esivulekile, okokufaka kwezilimi eziningi, imisebenzi eqondene nesizinda esithile, noma amamodeli angaphandle kosayizi ohloliwe nebanga lezakhiwo. Kuze kube yilapho leyo mibuzo iphendulwa, i-CLLM iphathwa kangcono njengendlela yocwaningo ethembisayo yokulinganisa ukungaqiniseki kunesivikelo esiqinisekisiwe sokusetshenziswa kwezigxobo eziphezulu.