Baatu Glossaire IA

Luy ALiBi?

Maanaam

Pexem position bias buy yar poñ yi ñuy am ci distance token, di jàppale model yi ñu extrapole ci contexte yu gëna gudd.

Term yu ci méngoo

RoPE (Njëfandikoo position Rotaire)
Pexem kodage positionel buy wëlbati vecteur laaj ak butoŋu ngir kodage position token relatif yi.
Kodage posisioŋ
Leeral yiñ yokk ci token yiñ samp ngir transformatër yi mëna xàmmee xeetu toppalante yi.
Palanteer buy gliise
Modèlu attention boo xamni token bu nekk dafay bàyyi xel ci palanteer bu yam ci token yi ci wetam ngir wàññi calcul bi.
Bias algorithmik
Njuumteg sistemik ci génnug model yi juddoo ci done yu jaarul yoon, xalaat wala tànneef ci modeling.
Mekanism buy bàyyi xel
Benn xeetu komposant buy xoole ci wàll yi am solo ci benn dugal suñuy defar benn génne.
Far
Modèlu njuumte bu dëppoo wala ñàkka yamale ci done wala ci doxalinu model.

Nekk leen ci sunuy tegtal yu amul fayda

ALiBi Position Bias

Xoolal itam

BPE (Byte Pair Encoding)Multi-Head AttentionSentencePieceFlash AttentionANN (Approximate Nearest Neighbor)QLoRAHNSWDPO (Direct Preference Optimization)
Xoolal Glossaire IA bi yéppSaytu bépp tegtal IA bu amul fayda