Chii chaitika
Preprint yakaunzwa kuarXiv musi wa24 Nyamavhuvhu inosuma maCredal Large Language Models, kana maCLLM, ayo anoshandisa ensemble yeLoRA adapter kumiririra huwandu hwefungidziro inonzwisisika pane imwe chete mukana wekugovera. Vanyori vanowana token-level uye semantic-level kuzvipira zvibodzwa uye vanovaongorora kuti vapindure mibvunzo, calibration, sarudzo yekufungidzira, kuona kwekufungidzira, uye kufunga.
Iro bepa rinotsanangura kuganhurirwa nenzira yakajairwa mamodheru emitauro anomiririra kusava nechokwadi: modhi yakajairwa inoburitsa imwe fungidziro yekugovera, iyo vanyori vanoti inogona kubatanidza kusaziva nekusajeka kwechokwadi. Yavo yakatsanangurwa CLLM pachinzvimbo inoshandisa ensemble yeLoRA adapter kugadzira iyo inonzi bepa seti yekredhi. Mukutaura kwekuita, nzira yacho inoitirwa kuchengetedza kusawirirana kana kupararira pakati pezvinogoneka kufanotaura kugovera pane kumanikidza kusavimbika kwese mune imwe softmax kubuda. Kwakabva hakutauri kuti chinomiririra ichi chinoita kuti modhi ive yakarurama; inotaura kuti inogona kuita kuti dhigirii yekuzvipira iwedzere ruzivo.
Vanyori vanosuma matanho maviri anoenderana. Credal Token Commitment, kana CTC, inoshanda munzvimbo yechiratidzo uye inosanganisa yakaderera-yakasungwa rutsigiro, credal upamhi, uye mharadzano entropy. Iyo abstract inoti CTC inogona kuverengerwa pasina chimwe chizvarwa, icho chinogona kukosha kune masisitimu uko kudzokororwa sampling kungawedzera latency kana mutengo. Semantic Commitment Consistency, kana SCC, inotambanudzira iyo pfungwa kune semantic nzvimbo uchishandisa sampled kupedzisa. Iyo bepa zvakare inotsanangura SCC-Gap kuyera kusawirirana pakati pechiratidzo-chikamu chetsigiro uye semantic-level rutsigiro. Izvi zvibodzwa zvinounzwa sematurusi ekuona kana iyo modhi-yepamusoro-level chivimbo ichigona kusaenderana nemhando yezvirevo zvinoratidzwa nemhinduro dzayo.
Ongororo iyi inosanganisira Gemma-2-9B, Llama-3.1-8B, uye Qwen2.5-7B paOpenBookQA, CoQA, TriviaQA, uye ARC-Challenge. Zvinoenderana neabstract, CLLM ndiyo nzira inonyatsoita pabvunzo-mhinduro chaiyo uchichengetedza makwikwi anotarisirwa kukanganisa kukanganisa. Vanyori vanoshumawo kuti CTC inouya mukati me1.5 muzana yemapoinzi enzvimbo yakanakisa yekuona-yekuona pasi peanogamuchira anoshanda maitiro curve mune akawanda marongero, pasina chimwe chizvarwa. Pakufanotaura kwakasarudzika pa80% yekuvhara, iyo abstract inoshuma 99.0% kunyatsoita paOpenBookQA yeCLLM ine SCC. Inotanga kutaura mhedzisiro yeARC-Challenge yeCLLM ine chivimbo cheCsem asi inodimburwa isati yapa mhedzisiro, kuitira kuti kudai hakugone kuongororwa kubva kwawakapihwa.
Nei zvichikosha
Mienzaniso yemitauro inogona kuburitsa mhinduro dzakatsetseka nechivimbo chisina kufanira. Kana zvakashumwa zvikaramba zviripo, kuyera kusavimbika pane dzakawanda zvinogoneka kufanotaura kugovera kunogona kubatsira masisitimu kuona mhinduro dzinoda kuongororwa, kurega, kana kuongororwa kwevanhu pasina kuda kuwedzera chizvarwa muzviitiko zvakawanda.
Nyaya yepakati haizi yekungoti chimiro chemutauro chinogona kupindura mubvunzo, asi kuti chinogona kusiyanisa ruzivo nekusava nechokwadi. Mhinduro yakatsetseka asi isiriyo inogona kuva nengozi kupfuura kurambwa kuri pachena apo vashandisi vanobata chivimbo seumboo. Bepa rinotsanangurwa credal chinomiririra dambudziko iro nekuchengetedza kusawirirana pakati peiyo adapta-yakavakirwa kufanotaura. Kana iyo nzira ikawedzera, inogona kupa vanogadzira chiratidzo-padivi chiratidzo chekusarudza nguva yekupindura zvakananga, kukumbira kuongororwa, kuendesa mubvunzo kune imwe system, kana kubatanidza munhu.
Mhedzisiro yesarudzo-yekufanotaura yakataurwa inonyanya kukosha pakutumirwa nekuti masisitimu akasarudzika haadi kupindura mubvunzo wese. Sisitimu inokwanisa kuchengetedza kurongeka kwepamusoro ichivharira chete nyaya yainoti inotsigirwa zvakakwana inogona kubatsira zvakanyanya muzvirongwa umo zvikanganiso zvinotakura zvinodhura. Iyo yakashumwa 99.0% yechokwadi pa80% yekuvhara paOpenBookQA inokurudzira mukati meiyo dataset nekugadzirisa, asi inofanirwa kunzwisiswa semhedzisiro yebepa kwete humbowo hwekuti system yakatumirwa ichaita zvakafanana. Iyo abstract haitauri huwandu hwemienzaniso, nzira dzekuenzanisa, kana tsananguro yekushanda yekuvhara.
Iyo isina-yekuwedzera-chizvarwa kudai yeCTC inogona zvakare kukosha kune inference dhizaini. Mazhinji maitiro ekusava nechokwadi anovimba nekugadzira akawanda kupedzisa, izvo zvinogona kuwedzera computation uye kunonoka. Tsime rinoti CTC inosanganisa akati wandei ane hukama-ane hukama pasina chizvarwa chekuwedzera, nepo SCC ichishandisa zvakajeka kupedzisa. Musiyano iwoyo unopa bepa kongiri yeinjiniya kona: chimwe chibodzwa chingave chakachipa kushandisa, nepo imwe yacho ichigona kubata kusawirirana semantic zvakananga. Iyo abstract haienzanise mutengo weLoRA ensemble pachayo, zvisinei, saka iyo yese yekushandira yekutengesa-off inoramba isingazivikanwe.
Interactive Mechanism: Iyo Inonyatsoshanda
Ongorora ari pasi tekinoroji kuseri kwekusimudzira uku uchipindirana.
What is a common training objective for an autoregressive language model?
Zvekutarisa zvinotevera
Mhedzisiro parizvino ndeye munyori-yakashumwa preprint evaluation, kwete yakazvimiririra yakazvimiririra yemaitiro ekutsvaga. Ruzivo rwakakosha rwunoramba rwuripo muchidimbu chakapihwa, kusanganisira iyo yakazara ARC-Challenge mhedzisiro, chaiwo mabhesi, computational pamusoro, uye nemafambisirwo enzira kupfuura mamodheru akaedzwa nemaseti.
Mubvunzo wekutanga ndewekuti izvo zvakaburitswa zvinopona zvakazvimiririra kudzokorora. Sosi yacho ndeye arXiv preprint yakaunzwa muna Nyamavhuvhu 24, 2026, uye zvinhu zvakapihwa zvinongori neabstract. Migumisiro saka zvichemo zvakaitwa nevanyori, kwete zvakazvimiririra zvakasimbiswa chokwadi. Ongororo yekutevera inofanirwa kuongorora matafura akazara, nheyo, tsananguro dzekuvimba, kusiyanisa kwenhamba, uye kana zvakanakira zvakashumwa zvinopindirana pamaseti mana ese uye mhuri dzese dzemhando nhatu.
Iyo abstract's ARC-Challenge mutsara haina kukwana: inotaura kuti CLLM ine Csem chivimbo inowana mhedzisiro asi isingapi kukosha. Irwo ruzivo rwakashaikwa runoganhura kuenzanisa neyakazara OpenBookQA kudai uye inodzivirira kuongororwa kwakazara kwemaitiro ekufunga kwemaitiro. Iri bepa zvakare rinoshuma mhedzisiro-yekuona mhedzisiro maererano nekuve mukati me1.5 muzana yemapoinzi eakanakisa AUROC mune akawanda marongero, asi rakapihwa rinoratidza zvibodzwa zvakakwana, nzira dzakanakisa dzekukwikwidza, kana kusarudzika.
Mibvunzo yekutumirwa yakakosha zvakaenzana. Iro bepa rinoshandisa ensemble yeLoRA adapters, uye abstract haitaure kuti mangani maadapta anodiwa, madzidzisirwo aanoitwa, kana kuti yakawanda sei ndangariro uye inference nguva yavanowedzera. Iyo zvakare inoongorora matatu chete ane mazita emitauro mamodheru uye ina mibvunzo-mhinduro dhatasethi. Izvo zvinoramba zvisingazivikanwe kana zvibodzwa zvinoshandira hurukuro refu, chizvarwa chakavhurika, mapindiro emitauro yakawanda, madomasi-chaiwo mabasa, kana mamodheru ari kunze kwesaizi yakaedzwa uye dhizaini yezvivakwa. Kusvika iyo mibvunzo yapindurwa, CLLM inobatwa zvakanyanya senzira inovimbisa yekutsvagisa kuyerwa kwekusava nechokwadi kwete chengetedzo yakasimbiswa yekushandiswa kwepamusoro.