Byagenze bite
Abashakashatsi bagaragaje uburyo icyitegererezo cyururimi rwitwikiriye rwitwara munsi yumutwaro uhuriweho hamwe nibikoresho nyabyo, basanga ubukererwe bwacyo hamwe nogukenera gukenera bitandukanye nubusanzwe imvugo yerekana imvugo.
Urupapuro rwashyikirijwe arXiv ku ya 24 Kanama, rusuzuma imiterere y'ururimi rwitwikiriye, cyangwa dLLMs. Bitandukanye na sisitemu ya autoregressive itanga inyandiko ikurikiranye, dLLMs irashobora kwerekana ibimenyetso byinshi icyarimwe. Abanditsi bavuga ko iri tandukaniro ritera akaga gushushanya sisitemu ya dLLM itanga gusa ibitekerezo bivuye kuri moderi ya autoregressive. Umusanzu wabo wibanze ni ibintu biranga umutwaro uhuriweho aho kuba ibiganiro gusa. Uru rupapuro rero rugaragaza ikibazo cyerekeranye ningaruka zikorwa zuburyo bwibisekuruza. Kugereranya kwayo ni imyitwarire ya sisitemu munsi yumutwaro, hamwe nicyitegererezo gitanga ibimenyetso byinshi binyuze mukwisubiramo aho gukurikira inzira imwe ikurikiranye.
Abashakashatsi bakoresheje LLaDA-8B-Amabwiriza hamwe na Discrete Diffusion Guhatira adapt ya LoRA kuri NVIDIA H200 GPU imwe. Basuzumye ibyashizweho kuri GSM8K na HumanEval. Urupapuro rutangaza ko gusaba ingorane bitagaragara: ibyifuzo biri mubice 11 byagenwe byerekana intambwe. Abanditsi bagerageje niba ikimenyetso icyo ari cyo cyose gishobora guhanura urwego rwicyifuzo mbere yuko ibisekuruza bitangira, ariko raporo nziza ya R2 yari 0.150, byerekana imikorere idahwitse mubushakashatsi bwabo. Ihitamo risobanura urugero rwibipimo. Ubushakashatsi bwatangajwe busobanura ibipimo byageragejwe byerekana icyitegererezo, adapt, GPU, n'ibipimo ngenderwaho, kandi ikizamini cyo guhanura cyatanzwe nkigice kimwe kiranga.
Ubushakashatsi butangaza kandi ko ingengo yimari yigihe gito ishobora guhisha serivisi zitandukanye. Ukurikije impapuro, ingengo yimari iri munsi yikimenyetso 320 irashobora kugabanya ibyifuzo mbere yuko gukwirakwizwa mubukererwe bigaragara. Ku gipimo kimwe-gisabwa, 24% gusa yigihe cyurukuta-isaha yo kubara GPU, mugihe ibisigaye byari byoherejwe na CPU kuruhande. Iyo ibyifuzo byateguwe kugirango pasiporo imwe yimbere isangwe kuri buri ntambwe yerekana, ibyinjijwe byikubye inshuro 16.0 kurwego rwa 16 ugereranije nibisabwa-byoherejwe. Urupapuro rukomeza gukuramo icyiciro-cyigihe cyo gukosora-kuzuza guhuza ibice munsi ya Poisson. Hamwe na hamwe, ibi bipimo bihuza ibyifuzo-urwego rwimyitwarire hamwe na sisitemu-urwego rwateganijwe. Basobanura byombi aho umwanya ukoreshwa nuburyo kugabana akazi mubisabwa bihindura ibyinjijwe byavuzwe, mugihe isesengura ryigihe-cyigihe cyo guhuza nicyitegererezo cyo kuhagera.
Ibisobanuro birambuye: arxiv.org ↗
Impamvu ari ngombwa
Ibyavuye mu bushakashatsi bishobora gufasha injeniyeri gukora ibikorwa remezo bikora neza byerekana imiterere yururimi rushingiye ku gukwirakwiza, mu gihe kandi byerekana ko ibipimo ngufi n'ibitekerezo twarazwe na autoregressive service bishobora guhisha ibiciro byingenzi byo gukora.
Ubusobanuro bufatika nuko serivisi ya dLLM ishobora gusaba kubangikanya kurwego rutandukanye no gukorera autoregressive. Konti yimpapuro, igice cyingenzi cyo kugabana akazi ni buri ntambwe yerekana, ntabwo ari ibyifuzo byose. Ibyo bihindura uburyo sisitemu yo gutanga serivisi igomba gutekereza kubyerekeye kwinjira, guterura, no kwirukanwa mugihe ibyifuzo byinshi bigenda bitera imbere binyuze mubibare bisangiwe. Igisubizo ni isomo rya sisitemu yerekeye guhuza ibikorwa remezo na moderi yimikorere yicyitegererezo. Itandukaniro rigira ingaruka muburyo bwo gutanga ibice bisuzumwa. Kwinjira, guterura, no kwirukanwa ntabwo aribisobanuro birambuye mubikorwa; bagize igice cyo guhuza sisitemu na moderi yerekana inzira.
Ubushakashatsi bwa CPU-hejuru burakenewe cyane cyane muburyo bwo kohereza ubukungu nubuhanga bwo gukora. Niba ari bake gusa basaba urukuta-isaha igihe cyakoreshejwe kuri comptabilite ya GPU muriki gipimo cyageragejwe, wongeyeho ubushobozi bwihuta ntibishobora gukemura ikibazo cyonyine. Ibisubizo byatanzwe byerekana ko guhuza ibyifuzo bishobora kugabanya ibiciro byoherejwe, nubwo ibisubizo byimpapuro bihujwe nuburyo bwihariye, adapt, ibyuma, akazi, hamwe na baseline. Inshingano ni nkenerwa gusuzuma inzira yuzuye kuva gusaba kugera kubikorwa byihuta. Ibipimo by'impapuro bituma iyo nzira igaragara muburyo bwageragejwe, kandi ibisubizo byo guterura byerekana impamvu aho ibintu biri hejuru iyo imikorere isuzumwe.
Uru rupapuro kandi rugaragaza uburyo dLLMs zishobora kugenwa. Ingengo yigihe gito yigihe gishobora gutuma ubukererwe bugaragara nkubwa mbere kuko icyifuzo kirangira mbere yuko itandukaniro ryuzuye muburyo bwo kwerekana intambwe igaragara. Ibyo bifite akamaro kubantu bose bagereranya sisitemu yo gutanga cyangwa kugereranya abakoresha-igihe cyo gusubiza. Abanditsi bavuga ko imiterere y’ibisohoka idakwiye guteshwa agaciro kuko ingano y’icyiciro yiyongera munsi y’ibitekerezo bitatu byavuzwe, ariko raporo y’inkomoko yapimye ukuri kwa GSM8K gusa ku gipimo kimwe gisabwa, aho cyari 74% kugeza 76%. Ibi ntabwo bishyiraho ubuziranenge budahinduka muri buri cyiciro. Niyo mpamvu kandi impapuro zitandukanya ibitekerezo byubaka nibimenyetso byapimwe. Ibitekerezo bishyigikira ibitekerezo byavuzwe nabanditsi, mugihe ibipimo byavuzwe byerekana neza ko bikomeza kugarukira kubisubizo byavuzwe rimwe gusa kandi ntibisubiza ikibazo cyagutse.
Uburyo bukoreshwa: Uburyo bukora
Shakisha ikoranabuhanga ryihishe inyuma yiri terambere.
Which component of an AI application is the machine-learning model itself?
Ibyo kureba
Ubushakashatsi nibiranga hakiri kare bishingiye ku buryo bumwe bw'icyitegererezo, GPU imwe, n'ibipimo bibiri. Ibizamini byigenga kurugero, ibyuma, imizigo yakazi, hamwe nigikorwa cyo gukora bizakenerwa kugirango hamenyekane uburyo ibisubizo bikurikizwa.
Ikintu kinini kitazwi ni rusange. Ubushakashatsi bukoresha imiterere ya masked-diffusion moderi, LLaDA-8B-Amabwiriza hamwe na D2F LoRA adapt, hamwe na NVIDIA H200 GPU. Inkomoko ntisobanura niba urwego 11 rwo kubara intambwe imwe, intege nke zabanjirije iyambere, guhanura kwa CPU-kugeza kuri GPU, cyangwa inyungu ya 16.0x yo kugaragara byagaragaye hamwe nizindi dLLMs, adapteri, yihuta, porogaramu za software, cyangwa gusaba kuvanga. Iyo mipaka ni ngombwa mugihe usobanura ibisubizo. Ibyagaragaye ni gihamya yerekeye igeragezwa ryageragejwe, ntabwo ari ikarita yuzuye ya masked-diffusion ikorera imyitwarire muburyo bwose bushoboka.
Ibindi bikorwa bigomba kugerageza umusaruro nkumuhanda kandi muremure cyangwa byinshi bitandukanye. Uru rupapuro ruraburira cyane ko ingengo yimari iri munsi yikimenyetso 320 ishobora guhisha ubukererwe, bityo isuzuma rigomba kuba ririmo imizigo myinshi bihagije kugirango igaragaze imyitwarire yuzuye. Bizaba ngombwa kandi gupima ubukererwe bwumurizo, kwinjiza, gukoresha kwibuka, hamwe nubwiza hamwe aho gufata igishushanyo kimwe cyinjira nkibimenyetso bihagije byerekana inyungu zoherejwe. Ibipimo nkibi byoroshe gutandukanya iterambere ryikigereranyo cyinjiza niterambere rikomeza kuba ingirakamaro mumodoka nyayo. Bashobora kandi kwerekana niba imyitwarire iteganijwe iteganijwe ikomeza mugihe akazi gakorwa nibisohoka uburebure.
Ikirego cyiza gikomeza kuba itegeko. Abanditsi bavuga ko ubuziranenge butagomba guteshwa agaciro nubunini bwicyiciro munsi yibitekerezo bitatu, ariko inkomoko ntisobanura umurongo mugari wibisubizo byubunini bwibisubizo, ntanubwo itangaza ko umusaruro uhari cyangwa ibyoherejwe nabakoresha. Kwigana kwigenga muri GSM8K, HumanEval, hamwe nindi mirimo bizafasha kumenya niba guhuza guhuza ari ihame ryingirakamaro ryogushushanya cyangwa cyane cyane gutezimbere ubu bushakashatsi. Kugeza ubwo ibyo bizamini biboneka, gusoma birenze urugero birasabwa: guhuza ibice ni ihame ryogushushanya muburyo bwatangajwe, mugihe agaciro kayo koherezwa gasigaye gushirwaho.