Kedis ku xidhan hagaha · Dhexdhexaad Heerka

Kediska Go'aaminta Malaha

Waxay dabooshaa qabyo-oo-xaqiiji dejinta koodka, ilaalinta saxda ah ee qaybinta bartilmaameedka, xadka-heerka aqbalaadda, iyo cabbirada la taaban karo ee xawaaraha.

Wadooyinka haga ee la xidhiidhaDecoding mala awaal ah
Su'aal 1 ee 8

How does speculative decoding speed up generation from a large model?

Horudhac su'aal kasta8 su'aalo

Jawaabaha iyo sharraxaadaha waxay soo baxayaan markaad qaadanayso kediska.

  1. How does speculative decoding speed up generation from a large model?
    • Nashqada qabyada ah ee yar ayaa qiyaaseysa dhowr calaamadood oo qaabka weyni uu si siman u xaqiijiyo
    • Ku cadaadi qaabka nus cabbirkiisa si joogto ah
    • Ka saarida habka dareenka
    • Ku tababar qaabka xog badan
  2. For a large model at low batch size, why can one-token autoregressive decoding be slow?
    • Each token requires internet access
    • Repeated reads of large model weights can make decoding memory-bandwidth bound
    • The model permanently stores a full copy of every possible output
    • The tokenizer performs the model’s matrix multiplications
  3. In greedy speculative decoding, what happens at the first token where draft and target predictions disagree?
    • The whole conversation is thrown away and restarted from scratch
    • Qaabka qabyada ah ayaa isla markiiba dib loo tababaray
    • The matching prefix is kept, the target token is used at the mismatch, and later draft tokens are discarded
    • Tayada wax-soo-saarka ayaa si joogto ah hoos u dhigta
  4. Under exact speculative sampling, how does the output distribution compare with sampling from the target model alone?
    • It is always lower quality but faster
    • It is identical only when the draft model is larger
    • It becomes deterministic regardless of target sampling
    • It is identical in distribution
  5. Maxaa sida tooska ah u go'aamiya xawaaraha ka soo dejinta malo-awaalka ah?
    • Heerka aqbalaadda ee qiyaasaha moodeelka qabyada ah
    • Midabka GPU-da
    • Dhererka magaca tusaalaha
    • Waqtiga maalinta
  6. Maxay Medusa iyo EAGLE ka dhexeeyaan marka la soo dhawaado?
    • Waxay u baahan yihiin hardware quantum
    • Waxay meesha ka saarayaan baahida loo qabo qaab qabyo ah oo gooni ah
    • Waxay tirtiraan qaabka bartilmaameedka
    • Waxay kaliya ku shaqeeyaan sawirada
  7. What should a serving team measure before claiming speculative decoding will speed up its workload?
    • Only the target model’s parameter count
    • Only whether the draft produces fluent text
    • Draft cost, target verification cost, accepted tokens and end-to-end latency under representative traffic
    • The number of tokenizer files
  8. In exact speculative sampling, what happens when a proposed token is rejected?
    • The whole conversation is discarded
    • The target model is permanently changed
    • The rejected token is kept without verification
    • A correction step samples from the distribution needed to preserve the target model’s output distribution