quiz bu lëkkale ak tegtal · Diggante Niveau

Jamba Hybride Transformatër-Mamba

Xoolaat couche hybride Jamba yu njëkk ya, njaxasu-ekspert yi, nekkin buy baaxoo, màggug cache biy bàyyi xel, ak jàngat ci contexte bu yàgg bu lalu ci sargu liggéey.

Yooni njiit yu jëm ciJamba Hybrid Transformateur Médèles Mamba
Laaj 1 de 8

Ban ñaari xeeti core core la Jamba di boole?

Xoolaatal laaj bu nekk8 laaj

Sooy def quiz bi, tontu yi ak leeral yi dañuy feeñ.

  1. Ban ñaari xeeti core core la Jamba di boole?
    • Couche convolutionnelle ak pooling
    • RNN ak CNN
    • Transformateur attention ak Mamba
    • Diffusion ak couches d'attention
  2. Ban beneen pexe la Jamba di jëfandikoo ngir yokk kàttan gi te baña wàññi parametre yi?
    • Njaxasu-ekspert (MoE)
    • Distilaasioŋ xam-xam
    • Taggat ci xam limu
    • Njàngale mu am doole ci feedback nit
  3. How does a Mamba layer's recurrent representation differ from an attention layer's KV cache?
    • It removes every model parameter from memory
    • It skips all tokens beyond the first one
    • It stores an independent full copy of every earlier attention matrix
    • It maintains a fixed-size recurrent state instead of retaining a growing set of past keys and values
  4. Ban palanteer la xeetu Jamba bu njëkk bi jàppale?
    • 4K jetons
    • 256k jetons
    • 32K jetons
    • 8 milioŋ ciy jeton
  5. What is a reason to retain attention components alongside state-space layers in a Jamba-style hybrid?
    • To combine different access and representation mechanisms while evaluating their quality-efficiency tradeoff
    • To guarantee perfect retrieval of every token on every task
    • To make all attention computation independent of sequence length
    • To eliminate the need for any model evaluation
  6. How should Jamba's reported long-context memory benefit be evaluated for deployment?
    • Assume every 256K request fits any single GPU
    • Count only the recurrent state while ignoring weights and attention cache
    • Measure the actual model, precision, batch size, context, and hardware configuration
    • Treat every hybrid model as having zero sequence-state cost
  7. Does original Jamba's use of many Mamba layers make the whole model's sequence memory constant as input length grows?
    • Yes; every hybrid attention cache has a fixed one-token size
    • No; its retained attention layers still store a growing KV cache
    • Yes; model weights disappear once the sequence becomes long
    • Only the tokenizer's file size changes
  8. Why are total MoE parameters and active parameters per token different quantities in Jamba?
    • Every stored expert is necessarily deleted when inactive
    • A token must run all experts regardless of routing
    • The model's supported context directly sets its vocabulary size
    • Routing can activate only a subset of stored experts for each token