quiz bu lëkkale ak tegtal · Diggante Niveau

Laajteb tokenisaasioŋ

Xoolaat taggat SentencePiece ci bind bu ñor, diggante yiñ mëna delloo, algorithm vokabileer yi, ak fallback byte ngir jëfandikoo bind ci làkk yu bari.

Yooni njiit yu jëm ciTokenisaasioŋ ci frase
Laaj 1 de 8

Ban xalaat bu am solo la SentencePiece moytu bi tokenizers yu yàgg yi di wéeru?

Xoolaatal laaj bu nekk8 laaj

Sooy def quiz bi, tontu yi ak leeral yi dañuy feeñ.

  1. Ban xalaat bu am solo la SentencePiece moytu bi tokenizers yu yàgg yi di wéeru?
    • Ni kàddu yi dañu leen tàqale ci ay bayaal
    • Bind boobu dañu ko kode ci UTF-8
    • Ni frase yi di jeexee ci ponksioŋ
    • Ni vocabulaire bi deñu ko njëkka defar
  2. Lan moo waral SentencePiece di wecci barab yi ak màndarga meta buñ mëna gis?
    • Ngir gëna kompresse mbind mi
    • Def tokenization buñu mëna delloosi (amul ñàkk)
    • Ngir dindi bépp ponksioŋ ci saasi
    • Tekki ci diggante làkk yi
  3. Ban ñaari algorithm la SentencePiece gëna jàppale?
    • TF-IDF ak Word2Vec
    • LSTM ak CNN
    • BPE ak xeetu làkku Unigram
    • K-means ak PCA
  4. naka la ko modelu Unigram di tabaxee vocabulaire bi?
    • Soo tàmbalee amul dara ba noppi boole ñaari araf yu bari
    • Sooy kopie ab diksoneer angale
    • Soo jëfandikoo limu 1000 baat
    • Soo tàmbalee ci yu rëy, nga dagg piyees yi gëna néew njariñ jaaraleko ci EM
  5. Ban model moo gëna siiw ci jëfandikoo tokeniseer SentencePiece?
    • T5
    • GPT-2
    • ELMo
    • Gant
  6. Ban njariñ la fallback ci niveau byte di jox SentencePiece?
    • Dafay tax saxaar gi gëna gaaw
    • Dafay garanti ni bépp personage mën nañu ko representé, moytu njuumte ci vocabulaire bi
    • Dina dindi soxla bàyyi xel
    • Dafay yokk vocabulaire bi ci saasi ñaari yoon
  7. What kind of input can SentencePiece use to train a subword model directly?
    • Only text already converted to English word tokens
    • Only a fixed dictionary of complete words
    • Raw sentences, without first splitting them into word sequences
    • Only character-frequency tables with no sentence data
  8. Why is a language-independent tokenizer useful for multilingual text?
    • It translates every input into one language before tokenizing
    • It assigns the same meaning to every token in every language
    • It removes the need for a model vocabulary
    • The same tokenizer pipeline can process scripts without requiring language-specific word segmentation