Kwenzekeni
Abacwaningi u-Bingqi Shan, u-Zhehao Yu, u-Kenhong Lin kanye no-Baoquan Zhang bathumele umbhalo wangaphambili onamakhasi ayi-11 ku-arXiv ngomhla ka-18 Agasti 2026 echaza i-HB-SJD, inguqulo ehlanganisiwe yokuqagela kwe-Jacobi decoding esetshenziswa njengesipele sesampula sesizukulwane ngaphakathi kwe-visual on-policy distill. Indlela ishintsha kuphela indlela imodeli yomfundi ekhiqiza ngayo idatha yayo yokuqeqeshwa, futhi ababhali babika ukuqeqeshwa okusheshayo ku-LlamaGen ngaphandle kokulahlekelwa okushiwo kukhwalithi yokuphumayo.
Ukuphrinta ngaphambili okunesihloko esithi "Accelerating Visual on-Policy Distillation with Batched Speculative Jacobi Rollouts" kwahanjiswa ku-arXiv ngomhla ka-18 Agasti 2026 futhi kufakwe kukhathalogi ngaphansi kokufunda komshini (cs.LG) njenge-arXiv:2608.18183. Ukufakwa ohlwini kunikeza ababhali abane - u-Bingqi Shan, u-Zhehao Yu, u-Kenhong Lin no-Baoquan Zhang - futhi uchaza idokhumenti enamakhasi ayi-11 enezibalo ezi-4. Ikhasi le-arXiv libonisa inguqulo eyodwa, i-v1, futhi ayikho ijenali noma inkomba yenkomfa. Yonke into engezansi ivela kumethadatha engabonakali nesohlwini; umbhalo ogcwele ubungeyona ingxenye yomthombo obuyekezwe lapha.
Inkinga ababhali abaqondise kuyo ingaphakathi kwendlela yokuqeqesha kunomkhiqizo osetshenzisiwe. Ku-visual on-policy distillation, imodeli encane yokukhiqiza isithombe "yomfundi" iqeqeshwa kusetshenziswa amasampula umfundi uqobo awakhiqizayo, "nothisha" omkhulu ohlinzeka ngesignali yokufunda. Ngenxa yokuthi umfundi uyimodeli ye-autoregressive, lawo masampuli akhiwa ithokheni eyodwa ngesikhathi. Uhlaka lwephepha ukuthi lesi sinyathelo sokukhiqiza esilandelanayo - ukukhishwa - sengeza izindleko ezinkulu esinyathelweni ngasinye sokuqeqeshwa, ngakho-ke iluphu yokuqeqeshwa ikhokha ibhili yokukhipha amakhodi ngokuphindaphindiwe kunokukodwa.
Ikhambi elihlongozwayo liboleka i-Speculative Jacobi Decoding, indlela ebikezela futhi iqinisekise amathokheni amaningana ngokuhambisana ngaphandle kokudinga imodeli yokusalungiswa encane ehlukile. Ababhali bayaqaphela ukuthi indlela yasekuqaleni yayibhalelwe ukukhiqiza ukulandelana okukodwa ngesikhathi, okungafaneleki kahle ku-loop yokuqeqesha ekhiqiza izithombe eziningi ngesikhathi esisodwa. Umnikelo wabo, i-HB-SJD, uvumela isithombe ngasinye sibe yinqwaba ngokuya ngenqubekelaphambili yaso yokuqopha kuyilapho izithombe ezihlezi ezindaweni ezihlukene ngokulandelana kwazo zisaqinisekiswa ndawonye ekudluleleni phambili okuhlanganisiwe kumodeli. Njengoba izithombe ziqeda, isistimu iyashintsha phakathi kwalokho ababhali abakubiza ngokuthi ukuqaliswa Okugcwele Nokuhlangene ukuze kwehliswe izindleko zomjikelezo wokukhishwa wakamuva, ongenasici.
Ababhali bakubeka obala mayelana nobubanzi: I-HB-SJD ingena esikhundleni se-backend kuphela yokukhishwa kwabafundi futhi ishiya uthisha, inhloso yokukhishwa kwe-distillation kanye nenqubo yokwenza kahle ingashintshiwe. Lokho kufakwa kohlaka kuyethula njengokuyeka ubunjiniyela esikhundleni seresiphi entsha yokuqeqeshwa. Ukuze uthole ubufakazi, i-abstract icaphuna ukuhlolwa nge-LlamaGen, umndeni wemodeli yesizukulwane esizenzakalelayo, futhi ithi i-HB-SJD inciphisa kakhulu kokubili isikhathi sokukhishwa kanye nesikhathi sokuqeqesha esisuka ekupheleni ngenkathi ilondoloza ikhwalithi yokukhiqiza yomfundi ofakwe emanzini.
Izinto ezimbalwa ezingabonakali ezingabonakali kufanele zichazwe ngokucacile. Ayinikezi isici sokusheshisa, azikho izibalo zewashi lasodongeni, azikho incazelo yezingxenyekazi zekhompuyutha, azikho osayizi benqwaba noma ukulungiswa kwesithombe, futhi azikho i-metric yekhwalithi eshiwo ngemuva kwesimangalo sokuthi ikhwalithi iyagcinwa. Ayisho ukuthi i-HB-SJD ikhiqiza okukhiphayo okufana nokukhishwa kwekhodi kwethokheni ngethokheni noma okuqhathanisekayo. Uhlu lwe-arXiv alukhombisi ukukhishwa kwekhodi, futhi njengokuphrinta kusengaphambili umsebenzi awunawo ukubuyekezwa kontanga. Akukho lutho lapha oluphinde lwakhiqizwa ngokuzimela.
Imininingwane yomthombo: arxiv.org ↗
Kungani kubalulekile
Izindlela zokuqeqesha ezisekelwe kunqubomgomo ziyabiza kakhulu ngoba imodeli kufanele ikhiqize amasampula amasha ithokheni elilodwa ngesikhathi esinyathelweni ngasinye. Ukuhambisa indlela yokukhipha amakhodi ngokuhambisanayo isuka ekucabangeni iye ku-loop yoqeqesho kuhlasela izindleko ezikhawulelwe ukuthi ubani ongakwazi ukuqeqesha amamodeli akhiqizayo ahlangene, nakuba usayizi wenzuzo awukwazi ukubhekwa ku-abstract kuphela.
Izindleko zokuqeqesha zezindlela zenqubomgomo zilawulwa isinyathelo okunzima ukusifanisa. Ngokungafani nokuqeqeshwa okujwayelekile okugadiwe, lapho idatha isivele ikhona, ukuqeqeshwa kwenqubomgomo kufanele kukhiqize idatha ngesikhathi sokugijima, futhi isizukulwane esizenzakalelayo silandelana ngokwemvelo: ithokheni ngalinye lilinda ngaphambi kwalo. Lokho kwenza ukukhishwa kube intela eqhubekayo kunezindleko zokusetha kanye nje, futhi yisizathu sokuthi izindlela zenqubomgomo zingabiza kakhulu ngeyunithi ngayinye yokufunda kunezinye izindlela ezingaxhunyiwe ku-inthanethi.
Umsebenzi omningi oshicilelwe wokukhipha amakhodi okuhambisanayo ubekwe ekusetshenzisweni - ukunciphisa ukubambezeleka noma izindleko zabasebenzisi ababuza imodeli esetshenzisiwe. Ukukhomba isigaba esifanayo sesu ku-loop yokuqeqesha kuqondise isabelomali esihlukile nendawo ehlukile: abacwaningi namathimba amancane anquma ukuthi ukuhlolwa kungathengeka nhlobo. Uma ukonga okubikiwe kumile, umthelela ongokoqobo uwukuqeqeshelwa kwenqubomgomo okuqhutshwa ngeyunithi ngayinye yokubala, hhayi ukukhiqiza isithombe esisheshayo kubasebenzisi bokugcina.
Umnikelo othize yingxenye engathandeki, futhi ngokungangabazeki yiyona phuzu. Ukuqoshwa kwe-Jacobi okuqagelayo kwase kukhona kakade; obekungekho inguqulo eziphatha ngendlela enengqondo lapho ukulandelana okuningi kuthuthuka ngamanani angalingani. Amaqoqo lapho ezinye izithombe ziqedile kanti ezinye zingathandi ukumosha izingxenyekazi zekhompuyutha emsebenzini ongenzi lutho, kanye neswishi ye-Full-to-Compact echazwe ku-abstract iqondiswe ngokulinganayo kuleyo mfucuza. Ngenxa yokuthi uguquko lupheleliselwe ekukhishweni okungemuva, ukulamukela ngeke kudinge ukuphinda kushunwe ukusethwa kwe-distillation - ukucatshangelwa okunengqondo kunoma ubani osevele uqinisekise iresiphi yokuqeqeshwa.
Isihloko esingaphansi komfula siphinde sibe nesizotha ngomthelela womphakathi. I-distillation yalolu hlobo ikhona ukuze kukhiqizwe amamodeli esizukulwane sesithombe angasebenza kuzilungiselelo eziboshiwe, okuhlanganisa namadivayisi omuntu siqu. Ukuqeqeshwa okushibhile kwamamodeli anjalo empeleni kunganciphisa ikhompuyutha namandla adingekayo ukuwakhiqiza. Lolo chungechunge lwemiphumela luyakholeka kodwa alunakulinganiswa: ukuphrinta kwangaphambili, njengoba kufingqiwe embhalweni wayo ongekho emthethweni, kubika ukuncishiswa kwesikhathi sokuqeqeshwa kumndeni owodwa oyimodeli, hhayi ukubalwa kwamandla noma izilinganiso zezindleko.
Uma kuqhathaniswa nalokho, isisekelo sobufakazi sincane ngezindlela okufanele zithukuthelise noma yisiphi isiphetho. Umndeni oyimodeli eyodwa, azikho izinombolo ezibikiwe ku-abstract, futhi akukho ukubuyekezwa kontanga ndawonye kusho ukuthi isimangalo okwamanje siwukugomela komcwaningi kunomphumela omisiwe. Izindlela zokuqopha eziqagelayo nazo zizwela ukuthi ukuqagela kwazo okuhambisanayo kwamukelwa kaningi kangakanani, futhi ukuziphatha kokwamukela kungashintsha ngosayizi weqoqo, ubude bokulandelana kanye nesikali semodeli - akukho okuphawulwa yi-abstract.
I-Interactive Mechanism: Indlela Esebenza Ngayo Ngempela
Hlola ubuchwepheshe obuyisisekelo ngemuva kwalokhu kuthuthukiswa ngokuhlanganyela.
In AI training, what is an "epoch"?
Ongakubuka ngokulandelayo
Ukuthi iphepha eligcwele nanoma iyiphi ikhodi ekhishiwe ibika izici ezibambekayo zokusheshisa, ihadiwe, osayizi beqoqo kanye nezilinganiso zekhwalithi yesithombe; noma ngabe indlela idlulela ngale kwe-LlamaGen nangale kokukhishwa kwe-distillation kokunye ukuqeqeshwa okuphathelene nenqubomgomo; kanye nokuthi ukuphrinta kwangaphambili kuyasinda yini ekubuyekezweni kontanga kanye nokuphindaphinda okuzimele.
Into yokuqala okufanele uyihlole yi-PDF egcwele nanoma iyiphi ikhodi ehambisana nayo. Izimangalo ezinokhonkolo - ukuthi ukukhishwa kwaba ngokushesha kangakanani, kuma-GPUs, ngamuphi usayizi weqoqo, futhi kukalwa ngokuphikisana nesizinda sokuqalwa kwekhodi ukuqaliswa - kunquma ukuthi ingabe lokhu kuwumphumela wobunjiniyela oseceleni noma ophusile. Kanjalo nobufakazi bekhwalithi: noma ngabe ababhali babika isilinganiso esijwayelekile sekhwalithi yesithombe ngokuhambisana neziqhathaniso zesampula, nokuthi ingabe imiphumela yomfundi ocwebile iyaboniswa yini ukufana naleyo ekhiqizwe ngaphansi kokukhishwa okuvamile okuzenzakalelayo.
Okwesibili, bheka indlela indlela eziphatha ngayo njengoba iqoqo likhula. Izindlela ezichazwe zokwenza Okugcwele Nokuhlangene zisikisela ukuhwebelana phakathi kokusetshenziswa kwenkumbulo nokumosha ukubala, futhi izindlela zokuqagela ngokuvamile zilahlekelwa inzuzo yazo lapho ukwehluleka kokuqinisekisa kuphakama. Ukulinganisa okuzimele kumasayizi weqoqo ahlukene nobude bokulandelana kuzobonisa ukuthi izinzuzo ezibikiwe ziyingxenye yendlela noma yokucushwa okukodwa okuhle.
Okwesithathu, i-generalization ingumbuzo ovulekile. Ukuhlolwa okuqanjwe ku-abstract kuhilela i-LlamaGen kuphela. Ukuthi i-HB-SJD isiza amanye amamodeli abonakalayo azenzakalelayo, nokuthi ingabe iyasebenza ekuqeqeshweni kwenqubomgomo ngale kwe-distillation - ukulungiswa kahle kwesitayela sokufunda-sokuqinisa amamodeli wezithombe, isibonelo, noma ukuqeqeshwa okusindayo kwamamodeli ombhalo - akunakwa. I-backend esebenza kuphela esakhiweni esisodwa siwumnikelo omncane kunalowo ongene kuzinhlaka ezijwayelekile zokuqeqesha.
Okokugcina, bukela omaka abajwayelekile besisindo sesayensi: ukuthi ukuphrinta kwangaphambili kuyamukelwa yini endaweni, noma ngabe amanye amaqembu akhiqiza kabusha izimangalo zesikhathi, nokuthi indlela yobuciko ilandwa yini ezitaki zokuqeqeshwa zomthombo ovulekile lapho umphumela wayo kumaphrojekthi wangempela uzobonakala. Ingekho lezo zimpawu, ukufundwa okufanele ukuthi abacwaningi abane bahlongoze futhi bazibika ngokwabo ukuthuthukiswa kokuqeqeshwa kokuqeqeshwa, nokuthi ukuthuthukiswa akukakahlolisiswa omunye umuntu.