Idanwo ti o sopọ mọ itọsọna · Lile Ipele

Sisanwọle Speculative ati Olona-Tokini Asọtẹlẹ adanwo

Checks draft proposals, target verification, accepted prefixes, multi-token heads and why acceptance rate affects decoding speed.

Jẹmọ itọsọna awọn ọnaSisanwọle Speculative Ati Multi Tokini Asọtẹlẹ
Ibeere 1 ti 8

Kini idi ti iyipada autoregressive boṣewa nigbagbogbo fa fifalẹ lori GPU kan?

Awotẹlẹ gbogbo ibeere8 ibeere

Awọn idahun ati awọn alaye han bi o ṣe n mu ibeere naa.

  1. Kini idi ti iyipada autoregressive boṣewa nigbagbogbo fa fifalẹ lori GPU kan?
    • Awọn ami-ami jẹ iṣelọpọ ọkan ni akoko kan, nlọ GPU ti ko lo ni igbesẹ kan
    • GPU nṣiṣẹ jade ti iranti lẹsẹkẹsẹ
    • O gbọdọ tun ikẹkọ lori kọọkan àmi
    • Awọn fokabulari ti wa ni kekere ju
  2. Kini 'akọsilẹ' ṣe ni iyipada arosọ?
    • O kọ idahun ikẹhin nikan
    • O tanmo orisirisi tani ojo iwaju àmi cheaply
    • O npa awọn ami-iṣeeṣe kekere kuro
    • O compresses awọn iwọn awoṣe
  3. Bawo ni a ṣe tọju didara iṣẹjade ni iyipada arosọ?
    • Nipa yiyi awọn iṣeeṣe si odo
    • Nipa lilo awọn ọrọ ti o tobi ju
    • Nipa ijerisi ti o gba awọn ami-ami nikan ti o baamu pinpin ibi-afẹde
    • Nipa fo ijẹrisi patapata
  4. Kini o ṣe iyatọ asọtẹlẹ-ọpọ-àmi-ara Medusa lati ṣe iyipada arosọ Ayebaye?
    • O nilo awọn awoṣe nla meji lọtọ
    • O ṣiṣẹ nikan fun iran aworan
    • O yọ iwulo fun eyikeyi kọja siwaju
    • O ṣafikun awọn olori asọtẹlẹ iwuwo fẹẹrẹ ni afikun nitorinaa awoṣe awoṣe kan ati rii daju
  5. Kini idi ti ẹrọ oluyipada kan le rii daju ọpọlọpọ awọn ipo oludije ti o fẹrẹ fẹẹrẹ bi ọkan lakoko iyipada?
    • Nitori iyipada jẹ didi bandiwidi iranti-iranti, kii ṣe didi iṣiro
    • Nitori ti o caches gbogbo ayelujara
    • Nitori ijerisi foo awọn ipele akiyesi
    • Nitori iyara aago GPU ilọpo meji
  6. Kini nipataki pinnu iyara lati awọn ọna akiyesi tabi awọn ọna ami-ọpọlọpọ?
    • Awọn awọ ti GPU
    • Awọn apapọ nọmba ti àmi gba fun igbese
    • Awọn ipari ti awọn awoṣe ká orukọ
    • Iwọn disk naa
  7. A draft proposes tokens A, B and C, but target verification first disagrees at B. Which proposal prefix can be accepted?
    • A, B and C
    • B and C only
    • A, the matching prefix before the first mismatch
    • None, even if A matches
  8. Why can speculative decoding preserve the target model’s output distribution?
    • The drafter is guaranteed to be identical to the target
    • The target model is never run
    • All proposed tokens are accepted unconditionally
    • A verification and correction rule accepts or rejects draft tokens according to the target distribution