Ƙididdigar Tagar Mahalli na YaRN
YaRN (Har yanzu wani karin ropeN) wata dabara ce wacce ke shimfida tagar mahallin da za a iya amfani da ita fiye da abin da aka horar da shi, tare da mafi karancin daidaitawa.
Dubawa
It matters because it lets existing models handle much longer documents without retraining from scratch.
Zurfafa nutsewa
Yawancin LLM na zamani suna ɓoye matsayi na kalmomi ta amfani da Rotary Position Embeddings (RoPE), wanda ke aiki da kyau har tsawon tsayin samfurin da aka gani yayin horo. Ciyarwa a cikin jerin tsayi mai tsayi kuma ƙirar ta ƙasƙanta da kyau. YaRN yana warware wannan ta hanyar sake daidaita mitocin jujjuyawar RoPE a cikin tafarki mai santsi: maɗaukaki masu girma (waɗanda ke ɗaukar alaƙar gida, da ke kusa) an bar su galibi ba a taɓa su ba, yayin da ƙananan mitoci (wanda ke ɗaukar matsayi mai tsayi) suna tsaka-tsaki. Hakanan yana ƙara daidaita yanayin zafi ga hankali don kiyaye kayan aiki da kyau a dogayen jeri. Sakamakon, wanda aka nuna akan ƙirar LLAMA, ya ƙaddamar da mahallin daga 4K zuwa 64K-128K alamomi ta amfani da kusan 0.1% na ainihin bayanan horo da ƙananan matakai masu kyau na ɗari.
Fahimtar Fasaha
RoPE yana jujjuya tambaya da maɓalli na maɓalli ta kwana daidai da matsayi da mitar kowane girma. Matsakaicin madaidaiciyar madaidaiciya (Matsayi Interpolation) yana lalata kowane mitoci daidai, yana cutar da cikakkun bayanai na gida. YaRN a maimakon haka yana amfani da 'NTK-by-parts': yana shiga tsaka-tsakin ƙananan mitoci (tsawon tsayin tsayi) kawai, yana barin manyan mitoci su kaɗai, kuma yana takure tsakanin su. Ƙwararren zafin hankali yana ramawa canjin entropy, yana kiyaye daidaito a tsayin tsayi.
Dabarun Tasiri
Gudu da sikelin
Gudun aikin harshe na iya tafiya da sauri ba tare da sadaukar da daidaito ba.
Shiga ku isa
Yana faɗaɗa damar shiga cikin harsuna da salon sadarwa.
Shawarwari masu haske
Ƙungiyoyi za su iya ciyar da ƙarin lokaci akan hukunci yayin da aiki da kai ke sarrafa maimaitawa.
Makomar Girman Tagar Yanayin Yanayin YaRN
YaRN-style mitar-sane tsawo ya zama tsoho sinadari don jigilar kayayyaki masu tsayi; bambance-bambancen karatu da magada suna ci gaba da bayyana yayin da labs ke turawa zuwa ga tagogin alamar miliyoyin. Yi tsammanin haɗin kai mai ƙarfi tare da ingantaccen kulawa, KV-cache compression, da haɓaka mai ƙarfi wanda ke daidaitawa akan tashi akan kowane buƙatu. Babban abin da ya fi dacewa shine daidaitawa 'lokacin da aka horar da samfurin' daga 'lokacin da zai iya karantawa da fa'ida,' sanya dogon mahallin ya zama fasalin horarwa mai arha maimakon sadaukarwar gini mai tsada.
Aiwatar da Gaskiyar Duniya
Ƙaddamar da samfurin LLAMA mai buɗewa daga 4K zuwa alamun 128K don haka zai iya shigar da cikakken codebase ko dogon kwangila a cikin fasfo ɗaya.
Barin chatbot ya riƙe dogon tarihin tattaunawa ba tare da yanke juyi da farko ba
Taƙaita takaddun tsayin littafi ko rubutattun sa'o'i da yawa waɗanda suka wuce tagar asali na asali.
Sauƙaƙe daidaita ƙirar da aka riga aka horar don ayyukan dawo da yanayi na dogon lokaci ta amfani da ƙaramar gudu mai kyau kawai.
Hatsari & Tsare-tsare
Abubuwan da aka ruɗe suna iya shigar da rahotanni cikin nutsuwa, kwararar tallafi, ko abubuwan bincike.
Hankali na gaggawa na iya ƙirƙirar sakamako mara daidaituwa a cikin buƙatun iri ɗaya.
Za a iya fallasa bayanan rubutu mai ma'ana idan ikon samun dama yana da rauni.
Taswirar Hanya
Ƙayyade tsarin fitarwa, sautin, da ma'auni masu inganci kafin fitowa.
Amsa a ƙasa tare da amintattun tushe a duk lokacin da daidaito ya shafi mahimmanci.
Ajiye wurin binciken ɗan adam don abubuwan da ake samu masu girma.
Bibiyar tsarin gazawar kuma sake horar da tsokaci ko tafiyar aiki akai-akai.
Ci gaba da Bincike
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the YaRN Context Window Scaling quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Jagora na gaba
Windows mahallin
Tambayoyin da ake yawan yi
What is YaRN Context Window Scaling?
YaRN (Har yanzu wani karin ropeN) wata dabara ce wacce ke shimfida tagar mahallin da za a iya amfani da ita fiye da abin da aka horar da shi, tare da mafi karancin daidaitawa. Yana da mahimmanci saboda yana ƙyale samfuran da ke akwai su sarrafa takardu masu tsayi da yawa ba tare da sake horarwa daga karce ba.
Wanne makircin rufaffen matsayi ne YaRN ke gyara don tsawaita tsawon mahallin?
YaRN yana nufin 'Har yanzu wani karin roPE' kuma yana aiki ta hanyar sake fasalin jujjuyawar jujjuyawar da aka yi amfani da shi a cikin Abubuwan Abubuwan Rotary.
Ta yaya YaRN ke kula da babban mitoci tare da ƙananan mitar RoPE?
Hanyar ''NTK-by-parts'' ta YaRN tana kiyaye girman mita mai girma (na gida) yayin da ke haɗa ƙananan mitoci (mai tsayi) don guje wa cutar da cikakkun bayanai na gida.
Menene mabuɗin ingantaccen fa'idar YaRN akan horar da ƙirar dogon yanayi daga karce?
YaRN na iya tsawaita mahallin ta amfani da kusan 0.1% na ainihin bayanan horo da ƙananan matakan daidaitawa, mai rahusa fiye da sake horarwa.
Bayan sake daidaita mitoci, wane ƙarin gyara ne YaRN ke amfani da shi don kiyaye hankalin dogon zango?
YaRN yana canza yanayin zafin hankali don rama aikin entropy/Logit wanda ke faruwa lokacin da jerin suka yi tsayi sosai.
Wace matsala ce ke faruwa idan kun ciyar da jerin samfurin RoPE tsawon lokaci fiye da yadda aka horar da shi, ba tare da sikeli ba?
RoPE yana gabaɗaya mara kyau zuwa dogayen matsayi maras gani, don haka inganci ya ruguje sai dai idan an sake daidaita mitoci.