Jàng Xibaar Jumtukaay yi Liggéey yi Sunu yitte

Baatu Glossaire IA

Lan mooy Jeton?

Maanaam

Benn pàccu mbind buñ toppatoo ci misaali làkk, lu ci melni benn baat wala màndarga.

Xoolal Glossaire IA bi yépp Saytu bépp tegtal IA bu amul fayda

AI bu sosalSistem IA yiy defar ëmbiit yu bees yu melni mbind, nataal, audio, wideo wala kod.Modelu làkk bu mag (LLM)Benn xeetu làkk buñ tàggat ci corpus mbind yu bari ngir sos ak jàngat mbind.Modèle multimodalModèle bu mëna defar wala defar ay xeeti done yu bari lu ci melni bind, nataal wala audio.Piyeesu fraseTokenizer bu xamul làkk, di jàng ay subword ci bind bu ñor te duko njëkka xaaj ci espace bu weex.GSM8KReferansu jafe-jafe baat ci math ci daara yu mag yi ñuy jëfandikoo ngir jàngat xalaat jéego ci jéego ci misaali làkk yi.TokenisasioŋXeetu xaaj mbind ci ay jeton ngir dugal ci model.

Nekk leen ci sunuy tegtal yu amul fayda

SentencePiece Tokenization WordPiece Tokenization Tokenizer-Free Byte-Level Models Speculative Streaming and Multi-Token Prediction

Xoolal itam

Temperature System Prompt Tool Use Synthetic Data Top-k Sampling Supervised Learning Top-p (Nucleus) Sampling Sparse Model