Incamake
YaRN (Nyamara ubundi RoPE extensioN) ni tekinike irambura idirishya rikoreshwa rya transformateur irenze kure ibyo yatojwe, hamwe na tuning-nziza. Nibyiza kuko ireka moderi zihari zikoresha inyandiko ndende cyane zitarinze kwitoza.
YaRN Context Window Scaring ni igice cyururimi-AI igikoresho cyo gusoma, kubyara, gutondekanya, no guhindura inyandiko n'imvugo murwego.
Kwibira cyane
LLMs nyinshi zigezweho zifata imyanya ikoresheje Rotary Position Embeddings (RoPE), ikora neza kugeza muburebure icyitegererezo cyabonye mugihe cy'amahugurwa. Kugaburira murwego rurerure kandi moderi itesha agaciro nabi. YaRN irabikemura mugukuraho imirongo ya RoPE muburyo bwo kumenya inshuro nyinshi: ibipimo byinshyi nyinshi (bifata umubano waho, hafi yubusabane) bisigara ahanini bidakorwaho, mugihe ibipimo bito bito (bifata umwanya muremure) birahujwe. Yongeraho kandi ubushyuhe bwo guhindura kugirango witondere kugirango logit yitware neza kurwego rurerure. Igisubizo, cyerekanwe kuri moderi ya LLaMA, cyagura imiterere kuva kuri 4K kugeza kuri 64K-128K ibimenyetso ukoresheje hafi 0.1% gusa yamakuru yambere yimyitozo nintambwe magana yo gutunganya neza.
Ubushishozi
RoPE izunguruka ikibazo hamwe nurufunguzo rwibanze ku mpande zingana n'umwanya hamwe na buri murongo. Naive umurongo interpolation (Umwanya Interpolation) uhuza imirongo yose uko ingana, byangiza amakuru yaho. YaRN ahubwo ikoresha 'NTK-by-ibice': ihuza gusa ibipimo buke-buke (uburebure-burebure), igasiga imirongo myinshi yonyine, kandi ikagenda hagati yabo. Igipimo cy'ubushyuhe cyitondewe cyishyura entropy ihinduka, ikabika neza uburebure burebure.
Kumenya YaRN Imirongo Idirishya
YaRN (Nyamara ubundi RoPE extensioN) ni tekinike irambura idirishya rikoreshwa rya transformateur irenze kure ibyo yatojwe, hamwe na tuning-nziza. Nibyiza kuko ireka moderi zihari zikoresha inyandiko ndende cyane zitarinze kwitoza. YaRN Context Window Scaring ni igice cyururimi-AI igikoresho cyo gusoma, kubyara, gutondekanya, no guhindura inyandiko n'imvugo murwego. Kugirango wubake byimbitse, fata YaRN Context Window Scaring nkicyitegererezo gikora, ntabwo ari ikintu kimwe: gusobanura ibyagezweho, gusobanura ibitekerezo, no gutandukanya ibyo sisitemu ishobora gukora byizewe nibisaba guca imanza zinzobere.
Mu myitozo, amakipe akomeye akoresha YaRN Context Window Scaring igishushanyo mbonera, kugarura, no gusuzuma imirongo nka sisitemu imwe y'itumanaho ihuriweho. Bandika ibipimo ngenderwaho byerekana intsinzi, bagerageza kurwanya amakuru afatika hamwe nakazi keza, kandi bagasubiramo bashingiye kubikorwa byagaragaye ko batsinzwe aho gutsinda inshuro imwe. Aha niho imyumvire yubumenyi ihinduka mubushobozi burambye kubicuruzwa, politiki, nibikorwa.
Ururimi rwakazi rushobora kugenda byihuse nta gutamba guhuzagurika. Mugihe kimwe, ibintu bya Hallucinated birashobora kwinjiza bucece raporo, gushyigikira imigendekere, cyangwa ibisubizo byubushakashatsi. Uburyo bukomeye cyane ni uguhuza umuvuduko wikigereranyo hamwe na disipuline yimiyoborere: kuyobora abaderevu, gufata ibimenyetso, gutangaza ibyemezo byicyemezo, no gukomeza kuvugurura uburyo bwo kwirinda nkimyitwarire yicyitegererezo, ibyo abakoresha bategereje, nibisabwa n'amategeko bigenda bihinduka.
Ingaruka z'Ingamba
Ururimi rwakazi rushobora kugenda byihuse nta gutamba guhuzagurika.
Ururimi rwakazi rushobora kugenda byihuse nta gutamba guhuzagurika. Mubikorwa byujuje ubuziranenge, ibi bihindurwa mumategeko agenga imikorere, imipaka nyirubwite, hamwe n'imihango yo gusubiramo kenshi kugirango amakipe ashobore kwigirira ikizere aho gupima ibidasobanutse.
Yagura uburyo bwindimi nuburyo bwo gutumanaho.
Yagura uburyo bwindimi nuburyo bwo gutumanaho. Mubikorwa byujuje ubuziranenge, ibi bihindurwa mumategeko agenga imikorere, imipaka nyirubwite, hamwe n'imihango yo gusubiramo kenshi kugirango amakipe ashobore kwigirira ikizere aho gupima ibidasobanutse.
Amakipe arashobora kumara umwanya munini murubanza mugihe automatike ikora gusubiramo.
Amakipe arashobora kumara umwanya munini murubanza mugihe automatike ikora gusubiramo. Mubikorwa byujuje ubuziranenge, ibi bihindurwa mumategeko agenga imikorere, imipaka nyirubwite, hamwe n'imihango yo gusubiramo kenshi kugirango amakipe ashobore kwigirira ikizere aho gupima ibidasobanutse.
Gushyira mu bikorwa-Isi
Kwagura moderi ya LLaMA ifunguye kuva 4K kugeza 128K ibimenyetso kugirango ibashe kwinjiza codebase yose cyangwa amasezerano maremare muri pass imwe
Kureka chatbot igumana amateka maremare y'ibiganiro utagabanije guhinduka mbere
Incamake y'ibitabo birebire by'ibitabo cyangwa inyandiko-mvugo y'amasaha menshi arenga icyitegererezo cy'ibanze kavukire
Guhindura bihendutse moderi yabanje gutozwa kubikorwa birebire-byo kugarura imirimo ukoresheje akantu gato-gahuza neza
Uburyo bwo Gushyira mu bikorwa
YaRN Imirongo Idirishya Gupima mubikorwa
Kwagura moderi ya LLaMA ifunguye kuva 4K kugeza 128K ibimenyetso kugirango ibashe kwinjiza codebase yose cyangwa amasezerano maremare muri pass imwe.
Kwagura moderi ya LLaMA ifunguye kuva kuri 4K kugeza kuri 128K kugirango ishobore kwinjiza codebase yose cyangwa amasezerano maremare muri pass imwe Amakipe akunze kubona ibisubizo byiza mugihe asobanura ibipimo byujuje ubuziranenge imbere, agakomeza inzira yo kuzamuka kwabantu kubibazo bikurikirana, kandi akurikirana inyungu zibyara umusaruro nibiciro byamakosa mugihe.
YaRN Imirongo Idirishya Gupima mubikorwa
Kureka chatbot igumana amateka maremare y'ibiganiro utagabanije guhinduka mbere.
Kureka chatbot ikagumana amateka maremare y'ibiganiro utagabanije guhindukira mbere Amakipe ubusanzwe abona ibisubizo byiza iyo asobanuye ibipimo byujuje ubuziranenge imbere, agakomeza inzira yo kuzamura abantu kubibazo byimbitse, kandi akurikirana inyungu zibyara umusaruro hamwe nibiciro byamakosa mugihe runaka.
YaRN Imirongo Idirishya Gupima mubikorwa
Incamake y'ibitabo birebire by'ibitabo cyangwa inyandiko-mvugo y'amasaha menshi arenga icyitegererezo cy'ibanze kavukire.
Mu ncamake inyandiko-ndende y'ibitabo cyangwa inyandiko-mvugo y'amasaha menshi arenga icyitegererezo cy'ibanze idirishya kavukire Amakipe akunze kubona ibisubizo byiza iyo asobanuye ibipimo ngenderwaho byimbere imbere, agakomeza inzira yo kuzamura abantu kubibazo, kandi akurikirana inyungu zibyara umusaruro hamwe nibiciro byamakosa mugihe.
YaRN Imirongo Idirishya Gupima mubikorwa
Guhindura bihendutse moderi yabanje gutozwa kubikorwa birebire-byo kugarura ibintu ukoresheje akantu gato keza neza.
Guhuza bihendutse icyitegererezo cyateguwe mbere yigihe kirekire cyo kugarura ibintu ukoresheje akantu gato gusa kateguwe neza Amakipe ubusanzwe abona ibisubizo byiza iyo asobanuye ibipimo byujuje ubuziranenge imbere, agakomeza inzira yo kuzamura abantu kubibazo, kandi akurikirana inyungu zibyara umusaruro hamwe nibiciro byamakosa mugihe runaka.
Ingaruka & Kurinda
Ibintu bifatika bishobora kwinjiza bucece raporo, gushyigikira imigendekere, cyangwa ibisubizo byubushakashatsi.
Kwihuta byihuse birashobora gukora ibisubizo bidahuye mubisabwa bisa.
Ibyanditswe byumvikana birashobora kugaragara niba kugenzura kugenzura ari ntege.
Igishushanyo mbonera
Sobanura imiterere isohoka, amajwi, hamwe nubuziranenge mbere yo gutangira.
Sobanura imiterere isohoka, amajwi, hamwe nubuziranenge mbere yo gutangira. Fata buri ntambwe nk irembo ryibimenyetso: niba ibipimo bitujujwe, hagarika kuzenguruka, funga icyuho, hanyuma noneho wagure imikoreshereze.
Ibisubizo byibanze hamwe nisoko yizewe igihe cyose ukuri kwingirakamaro.
Ibisubizo byibanze hamwe nisoko yizewe igihe cyose ukuri kwingirakamaro. Fata buri ntambwe nk irembo ryibimenyetso: niba ibipimo bitujujwe, hagarika kuzenguruka, funga icyuho, hanyuma noneho wagure imikoreshereze.
Komeza kugenzura abantu kugenzura ibisubizo byinshi.
Komeza kugenzura abantu kugenzura ibisubizo byinshi. Fata buri ntambwe nk irembo ryibimenyetso: niba ibipimo bitujujwe, hagarika kuzenguruka, funga icyuho, hanyuma noneho wagure imikoreshereze.
Kurikirana uburyo bwo kunanirwa no kongera imyitozo cyangwa akazi gahoraho.
Kurikirana uburyo bwo kunanirwa no kongera imyitozo cyangwa akazi gahoraho. Fata buri ntambwe nk irembo ryibimenyetso: niba ibipimo bitujujwe, hagarika kuzenguruka, funga icyuho, hanyuma noneho wagure imikoreshereze.