HejuruUbuyobozi bukurikira
Umwanya Interpolation yo Kwagura Imirongo
Ururimi AI
Ubuyobozi bwa tekiniki
Umwanya wa Interpolation (PI) nubuhanga bworoshye, bukomeye bwagura idirishya ryimiterere ya Transformer ukanda indangagaciro nshya mumurongo icyitegererezo kimaze kumenya.
Umva iki gitaboAI-byavuzwe · ~2 min
Instead of extrapolating to unseen positions, it interpolates within trained ones, requiring only brief fine-tuning.
Yatangijwe na Meta abashakashatsi (Chen et al.) Mu 2023, Positional Interpolation ikemura ikibazo cyuko moderi hamwe na RoPE zananirana bikabije mugihe zoherejwe mumyanya irenze amahugurwa. Ubushishozi buvuguruzanya: aho gusaba icyitegererezo gukora indangagaciro nini itigeze ibona, PI igabanya ibipimo byinjira byinjira ku gipimo cyerekana igipimo cyerekana uburebure bwa, kuvuga, amakarita 8K asubira mu mwimerere wa 2K. Kuberako icyitegererezo cyatojwe kururwo rwego, kuzunguruka biguma-kugabura. Nyuma yintambwe 1.000 gusa yo gutunganya neza, moderi ya LLaMA yaguye ubu buryo ikemura kugeza 32K. Uru rupapuro rwerekanye ko extrapolation ishobora gutesha agaciro amanota manini, mugihe interpolation ituma imipaka kandi itajegajega, niyo mpamvu interpolation ikora neza cyane kuruta kurenganurwa.
Kwagura moderi ya 2K-ya LLaMA kugirango ikore ibimenyetso 8K-32K hamwe nintambwe zigera ku 1.000
Guhuza uburyo bwo kuganira buriho kubwinyandiko ndende incamake utarinze kwitoza
Gukora nkibisobanuro fatizo NTK-izi gupima na YaRN itezimbere
Gushoboza imirongo miremire cyangwa amategeko-inyandiko yisesengura kuri moderi yabanje gutozwa na Windows ngufi
Umwanya wa Interpolation wabaye ishingiro ryumurongo wo gukurikirana, harimo NTK-uzi gupima na YaRN, ihuza cyane guhitamo kugirango ibungabunge amakuru yaho. Inzira nyabagendwa yerekeza kuburyo bukenera bike cyangwa budahuza neza no guteka ibintu birebire bikoreshwa mukwitegura. PI ikomeza kuba ibyingenzi kandi ikunze guhuzwa na gahunda nshya-yamenyeshejwe gahunda yo kugera kuri 128K-wongeyeho imiterere ya Windows neza.
Ibaruwa y'amakuru y'ubuntu
Inkuru eshatu zizewe za AI buri gitondo cy'iminsi y'akazi, zanditse mu rurimi rworoshye. Ubuntu iteka ryose, nta kwamamaza.
Imeri imwe buri munsi y'akazi. Kwiyimura mu buryo bumwe. Ntitwigera tugurisha cyangwa ngo dusangire aderesi yawe.
Isuzume
Ibisubizo ako kanya kuri buri kibazo, n'icyemezo gishobora gusangirwa gifite ID yagenzurwa umaze gutsinda isomo.
Fasha uburezi bwa AI bw'ubuntu. AI Understanding ni umuryango utegamiye kuri leta wa 501(c)(3) — nta kwamamaza, nta kwishyura, iteka ryose. Tanga impano
Umwanya wa Interpolation (PI) nubuhanga bworoshye, bukomeye bwagura idirishya ryimiterere ya Transformer ukanda indangagaciro nshya mumurongo icyitegererezo kimaze kumenya. Aho gutondekanya imyanya itagaragara, irahuza hagati yabatojwe, bisaba gusa guhuza neza.
PI igabanya ibipimo byerekana umwanya munini kuburyo ikarita ndende ikurikirana ikagaruka mumyitozo yatojwe, igakomeza kuzenguruka mukwirakwiza.
Urupapuro rwa PI rwerekanye imyanya minini itagaragara irashobora gutanga amanota yo gutondekanya ubunini burenze amahugurwa, guhungabanya softmax.
Uru rupapuro rwatangaje ko intambwe zigera ku 1.000 zo gutunganya neza zari zihagije kugira ngo ibintu bigere kuri 32K.
PI ikuraho umwanya m kugeza m / s, ikanda intera nini nini murwego rwambere rwahuguwe.
PI yashyizeho interpolation nkuburyo bwizewe; NTK-izi gupima na YaRN yarayinonosoye muguhuza imirongo inshuro nyinshi.
Komeza wige
Abandi bayobozi batoranijwe kuriyi ngingo
HejuruUbuyobozi bukurikira
Umwanya Interpolation yo Kwagura Imirongo
Ururimi AI