Subira ku makuru
Guhanga udushyaAI Understanding ibisobanuro

Uburyo bwihuse bwa decoding bwibasira ububiko bwibuke muburyo burebure bwa AI

Urupapuro rushya rwa arXiv rwerekana Flash Flash yihuta, uburyo butarangwamo amahugurwa abanditsi bavuga ko bushobora kugabanya ibiciro-byo kubara kubitekerezo byururimi rurerure kandi bikarinda ibipimo ngenderwaho.

Na 4 min readRead the primary source
Source-provided image accompanying Faster decoding method targets the memory bottleneck in long-context AI models
Inyandiko y'ibanzeInkomoko yanditse
Umwanditsi
arxiv.org
Ihuza ry'inkomoko
arxiv.orghttps://arxiv.org/abs/2609.00097
Ubwoko bw'inkomoko
Inyandiko y'ibanze - itangazo ryemewe, impapuro, dosiye, cyangwa urupapuro rwambere-dusoma mu buryo butaziguye.
ImirongoSobanukirwa ibi mumasegonda 60

Tangira hano

Amagambo y'ingenzi

Kwibuka (Memory Memory)
Imiterere yabitswe umukozi wa AI akoresha intambwe cyangwa amasomo kugirango atezimbere.
Umubare
Guhindura uburemere bwikitegererezo kugirango ugabanye imiterere isobanutse nka 8-bit cyangwa 4-bit.
Algorithm
Ibisobanuro bisobanuwe byamategeko cyangwa intambwe mudasobwa ikurikiza kugirango ikemure ikibazo cyangwa irangize umurimo.
IsuzumeModeri ya AI Yasobanuwe Ikibazo

Byagenze bite

Abashakashatsi berekanye uburyo bwihuse bwa Flash Decoding (FFD), ibyuma- bifatanyiriza hamwe kwihutisha imvugo ndende-yerekana imvugo. Uru rupapuro ruvuga ko FFD ihuza guhitamo no kubara mu ntoki imwe, ikoresha umubare muto-muto wo kugereranya ibintu-bisikana, kandi bigashungura mu buryo bworoshye guhuza ibitekerezo binyuze mu ngamba zo hejuru. Abanditsi batanze raporo igera kuri 11,6x yintangiriro-yihuta, inkunga ya 256K-token, hamwe na 2.37x iherezo-iherezo ryinjira.

Uru rupapuro rwashyikirijwe arXiv ku ya 31 Kanama 2026 kandi rugaragazwa ko rwakiriwe muri ICML 2026, ruvuga ibyerekeranye no kwibuka-umurongo mugari hamwe n’igiciro cyo kwitabwaho cya quadratic kijyanye no gushushanya kuva kera. Icyifuzo cyacyo cyihuta cya Flash Decoding ihuza algorithmic sparsity hamwe nibikoresho byuma byahujwe aho gushingira kumurongo wa metadata yo hanze cyangwa icyiciro cyihariye cyo guhuza n'imihindagurikire.

Ukurikije ibisobanuro, FFD ikora scan-ibimenyesha gusikana hamwe na bito bito, hanyuma ikongera ibisubizo byo kubisikana mugihe cyo kubara. Ingamba zayo zo hejuru-delta zungurura ibitekerezo byitondewe ukurikije isaranganya ryagaragaye kandi birinda guhuza isi. Abanditsi basobanura uburyo nkamahugurwa adafite imyitozo na plug-na-gukina. Batanga raporo igera kuri 11,6x yihuta kurwego rwintangiriro, ikagera kuri 256K-token ibice, na 2.37x hejuru yanyuma-iherezo ryinjira. Ibisobanuro bivuga ko isuzuma kuri RULER na LongBench ryakomeje kwerekana urugero rwiza mugihe ritanga umusaruro muke, ariko ntirugaragaza imiterere yapimwe, ibyuma, ibyingenzi, cyangwa ibisubizo nyabyo.

Ibisobanuro birambuye: arxiv.org ↗

Impamvu ari ngombwa

Sisitemu ndende-ya sisitemu ya AI ihenze kuyikoresha kuko decoding isoma inshuro nyinshi amakuru menshi yo kwitondera, bigatuma umurongo wibuke wibintu bifatika. Niba ibyatangajwe byungutse birenze ibizamini byabanditsi, FFD irashobora kugabanya ibyuma nubukererwe bwibisabwa bitunganya inyandiko nini cyane, codebase, cyangwa amateka yo kuganira. Amahugurwa yubusa, gucomeka no gukina birashobora kandi gutuma ibitekerezo byoroha kubyemeza, nubwo inkomoko idashyiraho umusaruro witeguye cyangwa ibikoresho bigari bihuza.

Akazi kagamije ikibazo cyibikorwa remezo kigira ingaruka ku buryo butaziguye ku giciro no gusubiza ibibazo bya AI ndende. Kode yihuta irashobora kugira akamaro kubisesengura ryinyandiko, ububiko bwa software, abafasha-baremereye, hamwe nubundi buryo aho icyitegererezo kigomba kwitabira inshuro nyinshi hejuru yinjiza nini. Kuberako FFD idasaba kongera imyitozo, abashoramari b'icyitegererezo barashobora kuyikoresha mugihe cyagenwe aho kubaka uburemere bw'icyitegererezo cyangwa guhugura ibintu bishya.

Ibisubizo byatangajwe bikomeza kuba ibyavuzwe nabanditsi bivuye mubushakashatsi, ntabwo ari ibisubizo byigenga. Inkomoko ntabwo yerekana ko uburyo bukora neza muburyo bwububiko bwikitegererezo, kwihuta, ingano yicyiciro, cyangwa imirimo-yisi. Ntabwo kandi igereranya kuzigama kwibuka, gukoresha ingufu, igiciro cyose cyo gutanga, cyangwa ibicuruzwa byiza biva hanze.

Interactive Mechanism

Uburyo bukoreshwa: Uburyo bukora

Shakisha ikoranabuhanga ryihishe inyuma yiri terambere.

Document Size:128K tokens
Needle Placement Depth (Location in document):50% into text
Attention Context Buffer Map:
Target Fact (50%)
Equivalent Pages~320Standard book pages
Retrieval Accuracy99.9%Needle recall score
RAM / KV Cache5.1 GBMemory overhead
Prompt CachingActive~80% discount on reuse
Core takeaway: Million-token context windows allow querying whole codebases or legal archives in one prompt. However, KV cache memory scales with context length, making prompt caching crucial for real-time production.
Kugenzura Ibitekerezo Byagenzuwe+10 Points
AI Models Explained Quiz

Which component of an AI application is the machine-learning model itself?

Ibyo kureba

Ibibazo by'ingenzi ni ukumenya niba FFD ibika ukuri kuri moderi nyinshi, imirimo, uburebure bw'imiterere, hamwe n'ibikoresho bigezweho; uko inyungu zayo ziterwa nibice byihariye cyangwa intangiriro; kandi niba code yarekuwe ikoreshwa munsi yimpushya zisobanutse. Inkomoko ntabwo itanga ihuza ryashyizwe mubikorwa, ibikoresho bishyigikiwe, ibisabwa byoherezwa, ibisubizo byingufu, cyangwa ibiciro cyangwa amakuru ahari.

Ubundi igenzura rigomba kwibanda ku mpapuro zuzuye zigerageza gushiraho no gusohora kode: ibyuma bishyigikirwa hamwe nibikoresho bya software, ibipimo fatizo byo kugereranya, ibipimo ntarengwa, ibipimo nyabyo, hamwe nimpushya. Ntibizwi kandi niba FFD ihujwe na moderi ndende ikoreshwa cyane idafite imiterere yihariye yubuhanga, niba inyungu zayo zinjira zikomeza gukoreshwa nabakoresha benshi, kandi niba ibisubizo byatanzwe 256K byerekana ibikorwa bifatika cyangwa ibipimo ngenderwaho bigenzurwa.

Ibijyanye nuyobora & ibibazo

Moderi ya AI YasobanuweAbahinduraAmahugurwa ya AIEjo hazaza ha AIGerageza ibyo uzi - gerageza ikibazo cya AI kubuntuReba ijambo AI mumagambo yacuKurikiza icyerekezo cya AI cyo kurekura
Basanze ari ingirakamaro?