AI Glossary Nguva

Chii chinonzi Kukamura Mazwi?

Tsanangudzo

Maitirwo ekupatsanura mavara kuita tokens yemhando yekuisa.

Bhurawuza yakazara AI GlossaryOngorora ese emahara AI madhairekitori

Mashoko anoenderana

Hwindo reMamiriroIyo yakawanda yehuwandu hwekupinza tokeni iyo modhi yemutauro inogona kugadzirisa kamwechete.BPE (Byte Pair Encoding) (Kukodhiwa KweByte Pair)A subword tokenization algorithm iyo inobatanidza anowanzo hunhu pairs kuita reusable tokens.Encoder (Mutsvagiri)Icho chikamu chemuenzaniso chinoshandura chinopinza kuita chiratidziro chakavanzika.Muenzaniso weMultimodalModhi inogona kugadzirisa kana kugadzira akawanda emhando dzedata senge zvinyorwa, mufananidzo, uye odhiyo.OCR (Kuziva Mavara eMifananidzo)Tekinoroji inoshandura mavara mumifananidzo kana scanner kuita mavara anoverengwa nemuchina.SentencePiece (Chikamu Chemutsara)Mutauro-agnostic tokenizer inodzidza subword mauniti zvakananga kubva kune yakaomeswa zvinyorwa pasina pre-kupatsanurwa pachena.

Dzidza zvakawanda mumadhairekitori edu emahara

SentencePiece TokenizationWordPiece TokenizationTokenizationSubword Tokenization

Onawo

TokenTool UseTemperatureTop-k SamplingSystem PromptTop-p (Nucleus) SamplingSynthetic DataTransfer Learning