What is Sliding Window Attention? Definition | AI Understanding

Ibisobanuro

Uburyo bwo kwitondera aho buri kimenyetso cyitabira gusa idirishya rinini ryerekana idirishya ryerekana ibimenyetso kugirango ugabanye compte.

Amagambo afitanye isano

Kuraho uburemere buke bwikitegererezo cyangwa neuron kugirango ugabanye ubunini na compte.

Imikoreshereze ishingiye kuri fagitire

Igiciro aho ikiguzi gipima hamwe na API guhamagara, ibimenyetso, igihe cyo gufata umwanzuro, cyangwa compte yakoreshejwe.

Amategeko yo gupima

Isano ifatika yerekana uburyo imikorere itera imbere nubunini bwikitegererezo, amakuru, cyangwa kubara.

Icyitegererezo gike

Icyitegererezo aho ibipimo byinshi ari zeru cyangwa bidakora kugirango bigabanye kubara.

Uburyo bwihuta bwihuse aho umushinga muto wicyitegererezo utanga ibimenyetso byerekana urugero runini rugenzura muburyo bubangikanye.

Umubare ntarengwa winjiza ibimenyetso byururimi rushobora gutunganyirizwa icyarimwe.

Wige byinshi mubuyobozi bwacu

Attention Rollout and Head Pruning Sequence Parallelism and Ring Attention Multi-Head Latent Attention RWKV Linear Attention

See also

ALiBi BPE (Byte Pair Encoding)RoPE (Rotary Positional Embeddings)SentencePiece Positional Encoding ANN (Approximate Nearest Neighbor)Multi-Head Attention HNSW

Reba Amagambo yuzuye ya AI Shakisha ibyerekezo byose bya AI