Ikibazo gihuza ibibazo · Hagati Urwego

BERT na Encoder Models Ikibazo

Ongera usubiremo imiterere ya BERT, kwipfukirana ibipapuro, kodegisi, guhuza n'imihindagurikire, guhagararirwa, guhinduka neza, intego-interuro ikurikira, hamwe no gushyiramo.

Inzira zijyanye no kuyoboraBert na Encoder Model
Ikibazo 1 Bya 8

Ni ubuhe buryo bwo kubona amakuru bwemerera bidirectional attention mu input ya encoder ya BERT y'ibanze?

Reba buri kibazo8 ibibazo

Ibisubizo nibisobanuro bigaragara mugihe ufata ikibazo.

  1. Ni ubuhe buryo bwo kubona amakuru bwemerera bidirectional attention mu input ya encoder ya BERT y'ibanze?
    • Information from both left and right within the allowed input
    • Only the previous token in every layer
    • Only a second unrelated language
    • A guarantee of two correct answers to every question
  2. Ni iki cy'ibanze cya BERT pretraining cyerekana ko token zatoranyijwe zifite umwihariko w'ibyo ziri mu context?
    • Only next-token prediction under a causal mask
    • Only human-written class labels for complete documents
    • Only a fixed image reconstruction objective
    • Kwerekana imiterere y'ururimi
  3. Ni iki gice cy’umuyoboro w’ishusho (transformer) gitanga imiterere ya BERT ya mbere?
    • Only the autoregressive decoder
    • The encoder
    • A recurrent-only sequence network
    • A convolution-only image decoder
  4. Ni iyihe myirondoro y’ibikorwa byo kwigisha ikigaragazwa na BERT mbere yo gukomeza kwigisha ku bikorwa by’inyuma?
    • Always train a completely new network from random parameters for every example
    • Apply the task head before creating any learned representation
    • Learn a corpus-based starting representation, then adapt it to a defined task
    • Guarantee that every target task uses no labeled data
  5. Ni iki gisohoka gishobora gutangwa n’umuyoboro wa BERT wa mbere ku bisobanuro by’ishyirwa mu bikorwa cyangwa ku bisobanuro by’icyiciro?
    • Only a long autoregressive continuation in every task
    • Amajwi mabi
    • Pixel-level image output
    • Contextual text representations
  6. Which comparison accurately distinguishes the parameter-related designs of DistilBERT and ALBERT?
    • DistilBERT uses distillation; ALBERT uses factorized embeddings and cross-layer sharing
    • Both are guarantees of identical speed and accuracy on every task
    • Both replace all text input with image pixels
    • Neither retains any learned encoder representation
  7. Which label distinction is used in the original BERT next-sentence prediction task?
    • A positive versus negative product review
    • A token's verb tense versus noun number
    • An actual next sentence versus a randomly sampled alternative
    • A source sentence's font size versus text color
  8. Which three representations are added in the original BERT input embedding?
    • Only a single corpus-frequency number
    • Token, position, and segment embeddings
    • Decoder probabilities, image pixels, and audio pitch
    • Only one fixed vector shared by every input position