Byagenze bite
Itsinda ry’ubushakashatsi ryerekanye SHIFT-LLM, uburyo bwo gukosora amahugurwa adafite amahugurwa yerekana imiterere nini yindimi nini zahinduwe zahinduwe kugirango zigabanye ibiciro. Uburyo bwinjiza umurongo usigaranye Adaptor kuri buri kibanza cyo gutema hanyuma ukayihindura hamwe ifunze-ifunze byibuze-kare kwisubiraho ukoresheje dataset ntoya. Uru rupapuro rutangaza isuzuma mu miryango itanu yicyitegererezo, ibipimo bitandatu byo gutoranya ibice hamwe n’ibipimo birindwi bya zeru, hamwe no kugarura ukuri mu bikoresho byinshi byageragejwe.
Uru rupapuro rusobanura gutema uburebure nkuburyo bwo kugabanya igiciro cyurugero rwururimi runini ukuraho ibice byose bya Transformer. Nkuko abanditsi babivuga, ibi birema gukwirakwiza: leta zihishe zakozwe nyuma yo gukurwaho ntikigihuye nogusaranganya biteganijwe kumurongo wo hasi. Ibidahuye bishobora kuvamo igihombo cyukuri nubwo moderi isigaye iracyafite ibyinshi muburyo bwumwimerere. SHIFT-LLM itangwa nkurwego rwo gukosora iki kibazo cyihariye, bigatuma imikoranire hagati yo gukuraho ibice hamwe nimyitwarire yicyitegererezo yo hasi yibikorwa byingenzi byakazi.
Ibice byateganijwe ni Umurongo usigaranye Adaptor, cyangwa LRA, ushyirwa kuri buri mwanya waciwe. Adaptor irinda inzira iranga umwimerere usigaye wongeyeho kandi ikongeramo yoroheje ya affine isigaye ikosorwa. Abanditsi bavuga ko uku gukosora gushyirwaho binyuze mu gufunga-gufunga byibuze-kwaduka kwisubiraho ku kintu gito gifashwe na kalibrasi. Ntabwo bisaba kubara buhoro buhoro. Mu bisobanuro by'impapuro, LRA igamije kugereranya ivugurura risigaye ryaba ryarakozwe na blok yakuweho, mugihe twirinze kwitabwaho no kugaburira-imbere kubara bifitanye isano n'iryo tsinda.
Inkomoko ivuga ko ubwo buryo bwageragejwe ku miryango itanu y’icyitegererezo, ingingo esheshatu zo guhitamo ibice byo gukuraho hamwe n’ibipimo birindwi bya zeru. Ivuga ko SHIFT-LLM idahwema kugarura ubunyangamugayo bwatakaye binyuze mu gutema ubujyakuzimu mu bice byinshi, hamwe no kuzamura amanota 15.7 ku ijana kuri Llama-3.1-8B-Amabwiriza. Inkomoko ntabwo itanga amazina yerekana ibipimo, amanota y'ibanze, ibipimo byo gutema, gupima ibyuma cyangwa ibisubizo kuri buri cyitegererezo mubisobanuro byatanzwe. Ntabwo ivuga kandi ko uburyo bwabyaye ubwigenge cyangwa bwinjijwe mubicuruzwa byubucuruzi.
Ibisobanuro birambuye: arxiv.org ↗
Impamvu ari ngombwa
Kuraho ibice byose birashobora kugabanya ikiguzi cyo gukora LLM ariko birashobora no guhagarika imbere imbere byateganijwe nyuma. SHIFT-LLM yashizweho kugirango ikemure ubwo bucuruzi hamwe no gukosora byoroheje aho kugarura ibitekerezo byakuweho no kugaburira imbere. Niba ibisubizo byatangajwe muri rusange, inzira irashobora gutuma imiterere yururimi yifunitse ikoreshwa neza aho igiciro cyo gufata umwanzuro, kwibuka cyangwa ubukererwe bibujijwe.
Ikibazo gifatika cyakemuwe nimpapuro nikimenyane cyo guhagarika ibicuruzwa: moderi ntoya cyangwa ntoya irashobora kuba ihendutse gukora, ariko ubunyangamugayo bwayo burashobora kugabanuka kuko gukuraho ibice bihindura ibimenyetso byimbere byanyuze kumurongo. Umusanzu wimpapuro nugushaka uburyo bwo gutsindwa muburyo butaziguye aho gufata icyitegererezo cyaciwe nkicyitegererezo gisanzwe. Ibi bifite akamaro kuko igiciro cyo gufata umwanzuro ntigiterwa gusa no kubara ibipimo ahubwo binaterwa numubare wokwitabwaho no kugaburira-imbere kubara byakozwe kuri buri cyinjijwe.
SHIFT-LLM ivugwa muburyo bwa kalibrasi irashobora kuba ingirakamaro kuko idashingiye kumyitozo ngororamubiri. Abanditsi bavuga ko hakenewe urugero rwa kalibrasi magana make gusa kandi ko adapteri zishobora gushyirwaho uburyo bwo gufunga. Ibyo birashobora kugabanya amakuru, kwibuka hamwe nubuhanga bwubwubatsi bujyanye no kugarura ubuziranenge nyuma yo gutema. Icyifuzo cyo hasi-fonctionnement no guhuza neza murwego rwikurikiranya rwaciwe nabyo byerekanwe nkuburyo bwo kongeramo compression. Ibi nibisabwa kubijyanye nigishushanyo mbonera hamwe nubushakashatsi bwakozwe, ntabwo ari gihamya ko buri gikorwa kizagera ku kuzigama kimwe.
Igikorwa kirerekana kandi impamvu compression yicyitegererezo idashobora gusuzumwa gusa kubara ibipimo byakuweho cyangwa ibice. Icyitegererezo cyaciwe gishobora kuba gito muburyo bugifite ibibazo byo gukwirakwiza imbere bidahuye nibisohoka. Niba ivugururwa ryavuzwe rikomeye, ibice byoroheje byo gukosora bishobora guha abitezimbere ubundi buryo hagati yo gukora icyitegererezo cyuzuye no kwemera igihombo cyukuri cyo gutema. Ubusobanuro rusange buguma bugarukira, ariko: isoko yatanzwe ntabwo igereranya imikoreshereze yingufu, gutanga ikiguzi, ubukererwe bwanyuma-iherezo cyangwa imikorere mubikorwa byinshi.
Uburyo bukoreshwa: Uburyo bukora
Shakisha ikoranabuhanga ryihishe inyuma yiri terambere.
Which component of an AI application is the machine-learning model itself?
Ibyo kureba
Ibibazo nyamukuru bifunguye nuburyo uburyo bukora hanze y'ibipimo byatangajwe, umubare w'amakuru ya kalibrasi hamwe no kubara bisabwa mu bikorwa, kandi niba inyungu zayo zikomeza imirimo, indimi n'ibidukikije. Inkomoko ni progaramu ya arXiv kandi ntabwo ishyiraho kwigana kwigenga, kuboneka kwumusaruro, ibisubizo birambuye byubukererwe cyangwa ibihe byose byubushakashatsi inyuma yinyungu zavuzwe.
Ikibazo cya mbere cyo kureba ni imyororokere. Inkomoko igaragaza impapuro nka verisiyo ya mbere yoherejwe na arXiv ikavuga muri make ibisubizo byayo, ariko inyandiko yatanzwe ntabwo ikubiyemo imbonerahamwe yibanze, ubushakashatsi bwo gukuraho cyangwa ibisobanuro birambuye. Ibizamini byigenga byakenera gusuzuma niba inyungu zavuzwe zifata munsi yo gutema ibice bitandukanye, kalibrasi-yashizeho ingano nuburyo butandukanye bwo gukuramo ibice. Bakeneye kandi kugereranya SHIFT-LLM no kongera imyitozo, gusibanganya, guhuza neza neza hamwe nubundi buryo bwo guhunika munsi yingengo yimibare ihamye.
Ibipimo byo kohereza bizaba ingenzi. Uru rupapuro ruvuga ko LRAs yirinda kwitabwaho bihenze no kugaburira-kubara mbere yo kubara no gukuraho urwego rwo hasi no guhuza, ariko inkomoko ntabwo itanga ubukererwe nyabwo, gukoresha kwibuka, kwinjiza cyangwa ibisubizo byibikoresho. Wongeyeho adapteri zishobora kugira ingaruka zitandukanye bitewe nurwego rwerekana niba ibikorwa bikurikirana bishobora guhuzwa. Isuzuma rifatika rero rigomba gupima igiciro cyo gutanga cyose, ntabwo ari ugusubirana neza cyangwa umubare wibipimo byavanyweho.
Ingano ya rusange ni iyindi itazwi. Isuzuma ryatangajwe rikubiyemo imiryango itanu yicyitegererezo hamwe n’ibipimo birindwi bya zeru, ariko ibisobanuro byatanzwe ntibigaragaza imirimo, indimi, ingano yicyitegererezo kirenze izina rya Llama-3.1-8B-Igisubizo cyigisha, cyangwa imyitwarire yuburyo bwakoreshejwe amabwiriza akurikira, gukoresha igihe kirekire, gukoresha code cyangwa gusuzuma umutekano. Inkomoko nayo ntisobanura niba kalibrasi yamakuru igomba kumera nkibikorwa byoherejwe. Impapuro zizaza cyangwa code yasohotse irashobora gusobanura izo mbogamizi no kwerekana niba inzira ari ingirakamaro cyane cyangwa cyane cyane muburyo bwububiko bwihariye no gutema.