Icyerekezo kirekire
Imiterere-ndende yerekana imiterere y'ururimi gusoma no gutekereza hejuru yinyongera nini icyarimwe, kuva kumpapuro amagana kugeza kode yose.
Incamake
It matters because a bigger context window changes what is possible without retrieval, fine-tuning, or splitting documents.
Kwibira cyane
Idirishya ryerekana imiterere ni umubare ntarengwa wibimenyetso ushobora kwitabira muri pass imwe. Icyitegererezo cyambere cyakoresheje ibimenyetso ibihumbi bike; sisitemu igezweho igera ku bihumbi magana cyangwa na miliyoni. Inzitizi nyamukuru ni uko ibiciro byo kwiyitaho bisanzwe bikura inshuro enye hamwe nuburebure bwikurikiranya, bityo gukuba kabiri ibyinjijwe bikubye kane akazi. Ba injeniyeri barwanya ibi hamwe na kodegisi yubwenge nka RoPE nuburyo bwayo bwo gupima, impinduka zokwitabwaho nko kunyerera-idirishya na FlashAttention, hamwe no gucunga neza ubwenge. Ariko idirishya rirerire ntabwo rihita riba ryiza. Ikibazo 'cyatakaye hagati' cyerekana icyitegererezo gikunze kwibutsa amakuru mugitangira no kurangiza kwinjiza birebire byizewe kuruta ibintu byashyinguwe hagati, bityo uburebure buke bugomba guhuzwa no kwibuka kwakoreshwa.
Ubushishozi
Kwiyitaho kugereranya buri kimenyetso nikindi kimenyetso, guha O (n kwaduka) kubara no kwibuka muburebure bwikurikiranya n. Uku kwipimisha kwadratike niyo mpamvu imirongo miremire ihenze. FlashAttention igabanya ububiko bwibuke hamwe na IO-ibizi, iringaniye irinda kwandika matrike yuzuye yo kwibuka kugirango yibuke, mugihe kunyerera-idirishya ryitondewe rigabanya buri kimenyetso kubaturanyi. Umwanya wo guhinduranya (RoPE), akenshi hamwe na interpolation, reka moderi rusange muburebure bwikurikiranya kurenza uko babitojwe.
Ingaruka z'Ingamba
Umuvuduko n'igipimo
Ururimi rwakazi rushobora kugenda byihuse nta gutamba guhuzagurika.
Kugera no kugera
Yagura uburyo bwindimi nuburyo bwo gutumanaho.
Ibyemezo bisobanutse
Amakipe arashobora kumara umwanya munini murubanza mugihe automatike ikora gusubiramo.
Ejo hazaza h'igihe kirekire
Imirongo ya Windows izakomeza gukura, ariko umupaka ugenda uva muburebure bugana kubikoresha neza: ibyiza byo hagati yibutsa, igiciro gito kuri token, hamwe nibitekerezo byizewe mumadirishya yose. Tegereza kwishyira hamwe hamwe no kugarura ibintu bityo moderi ikurura gusa ibyingenzi, wongeyeho guhita wihutira gukoresha imiterere ndende ihendutse mubibazo byinshi. Ubwubatsi buvanga ibitekerezo hamwe na leta-umwanya wicyitegererezo nka Mamba igamije gukemura ibintu birebire cyane hamwe no kugereranya umurongo.
Gushyira mu bikorwa Isi
Kwandika amasezerano yimpapuro 100 yose muburyo bumwe hanyuma ugasaba icyitegererezo gushyira ibendera ingingo zose zivuguruzanya na politiki runaka.
Gupakira codebase yose cyangwa module nini kuburyo moderi ishobora gukurikirana ikosa kuri dosiye nyinshi zidafite intoki za dosiye.
Incamake yigitabo cyuzuye cyangwa inyandiko ndende yinama murupapuro rumwe mugihe ugumya kwerekanwa muri rusange.
Kugaburira amatike menshi yingoboka icyarimwe kugirango moderi isubize itike nshya hamwe namateka yuzuye urebye.
Ingaruka & Kurinda
Ibintu bifatika bishobora kwinjiza bucece raporo, gushyigikira imigendekere, cyangwa ibisubizo byubushakashatsi.
Kwihuta byihuse birashobora gukora ibisubizo bidahuye mubisabwa bisa.
Ibyanditswe byumvikana birashobora kugaragara niba kugenzura kugenzura ari ntege.
Igishushanyo mbonera
Sobanura imiterere isohoka, amajwi, hamwe nubuziranenge mbere yo gutangira.
Ibisubizo byibanze hamwe nisoko yizewe igihe cyose ukuri kwingirakamaro.
Komeza kugenzura abantu kugenzura ibisubizo byinshi.
Kurikirana uburyo bwo kunanirwa no kongera imyitozo cyangwa akazi gahoraho.
Komeza Ubushakashatsi
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the Long-Context Modeling quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Ubuyobozi bukurikira
Umwanya wa Interpolation kumwanya muremure
Ibibazo bikunze kubazwa
What is Long-Context Modeling?
Imiterere-ndende yerekana imiterere y'ururimi gusoma no gutekereza hejuru yinyongera nini icyarimwe, kuva kumpapuro amagana kugeza kode yose. Nibyingenzi kuberako idirishya rinini ryamadirishya rihindura ibishoboka utabanje kugarura, gutunganya neza, cyangwa gutandukanya inyandiko.
Icyitegererezo 'idirishya ry'imiterere' ryerekana iki?
Idirishya ry'imiterere ni ikimenyetso cyerekana ingero icyitegererezo gishobora gusoma no gutekereza icyarimwe icyarimwe imbere.
Ni ukubera iki kwiyitaho bisanzwe bibahenze kubwinyongera ndende?
Kwiyitaho kugereranya buri kimenyetso nikindi kimenyetso, bityo umunzani wigiciro nka O (n kare) muburebure bwikurikiranya n.
Ni ikihe kibazo 'cyatakaye hagati'?
Ubushakashatsi bwerekana kugarura ibintu neza kubintu byashyizwe hagati yigihe kirekire, uburebure bwonyine ntibwemeza kwibuka.
Niki FlashAttention itera imbere cyane?
FlashAttention ibara kwitondera muri tile utabanje kwiyumvisha matrike yuzuye yo kwibuka, koroshya ikiguzi cyo kwibuka cyigihe kirekire.
Ni uruhe ruhare gushyiramo umwanya wo kuzunguruka (RoPE) bigira uruhare muburyo burebure?
RoPE ikubiyemo amakuru ajyanye namakuru kandi irashobora guhuzwa kugirango icyitegererezo gikurikirane igihe kirekire kuruta uburebure bwamahugurwa.