Èdè AI Itọsọna
Ijerisi Iṣapeye Speculative
Speculative sampling uses a draft model to propose tokens and a target model to verify them.
Gbọ si itọsọna yiiAI-ṣe afihan · ~2 min
Lori iwe yi2 min ka
Akopọ
With the exact rejection-sampling correction, it preserves the target distribution; speedup depends on draft cost, acceptance, target verification and workload.
Jin Dive
Autoregressive generation is slow because each token needs a full forward pass of a huge model. Speculative sampling fixes this by pairing a cheap draft model with the expensive target model. The draft proposes a short run of tokens (say 4-8); the target then scores all of them in one parallel forward pass. A modified rejection-sampling rule accepts the longest prefix that is consistent with the target's own distribution and resamples at the first rejected position. Because acceptance is probabilistic and corrected, the final token stream is provably distributed exactly as if the target had generated alone, no quality loss. Speedups vary with draft cost, acceptance rate, target verification, batching and hardware. A parallel verification pass can confirm multiple tokens, but is not free.
Real-World imuse
Nsin awoṣe iwiregbe 70B pẹlu awoṣe iyaworan 7B lati ge lairi esi ni aijọju ni idaji pẹlu didara iṣelọpọ kanna.
Awọn ori ara-ara Medusa lori awoṣe ẹyọkan ti n sọ asọtẹlẹ ọpọlọpọ awọn ami iwaju iwaju, lẹhinna jẹrisi wọn laisi nẹtiwọọki yiyan yiyan.
Iyipada akiyesi orisun-igi ti o ṣeduro awọn itesiwaju ẹka pupọ ati pe o jẹri gbogbo wọn ni ibi-afẹde ibi-afẹde kan.
Iyara soke koodu-ipari awọn arannilọwọ ibi ti osere awoṣe kapa asotele boilerplate ti awọn ti o tobi awoṣe ni kiakia jerisi.
Ojo iwaju ti Ijerisi Iṣayẹwo Apejuwe
Iyipada asọye ti n di idiwọn ni awọn akopọ itọkasi. Awọn iyatọ tuntun ju awoṣe iyasilẹ lọtọ: akiyesi ara ẹni nlo ijade ni kutukutu tabi awọn olori asọtẹlẹ afikun (Medusa, EAGLE), kikọ ipilẹ-igi jẹri ọpọlọpọ awọn itesiwaju oludije ni ẹẹkan, ati wiwa wiwakọ ṣe afiwe awọn amoro n-gram. Reti isọpọ titọ pẹlu batching ati iṣakoso kaṣe KV, iwọn iyaworan ohun elo, ati lilo gbooro ni awọn ọja ifarabalẹ bii awọn oluranlọwọ iwiregbe ati awọn irinṣẹ ifaminsi nibiti gbogbo awọn iṣẹju-aaya ti ka.
Fifi o sinu iwa
Kini idi ti o ṣe pataki
- Iyara ati iwọn. Ṣiṣan iṣẹ ede le gbe ni iyara laisi irubọ aitasera.
- Wiwọle ati arọwọto. O faagun iraye si kọja awọn ede ati awọn aza ibaraẹnisọrọ.
- Awọn ipinnu diẹ sii. Awọn ẹgbẹ le lo akoko diẹ sii lori idajọ lakoko ti adaṣe n kapa atunwi.
Ṣọra fun
- Awọn otitọ ti a sọ di mimọ le tẹ awọn ijabọ sii ni idakẹjẹ, awọn ṣiṣan atilẹyin, tabi awọn abajade iwadii.
- Ifamọ kiakia le ṣẹda awọn abajade aisedede kọja awọn ibeere ti o jọra.
- Awọn data ọrọ ifarabalẹ le farahan ti awọn idari wiwọle ko lagbara.
Next awọn igbesẹ
- Ṣetumo ọna kika iṣẹjade, ohun orin, ati awọn iṣedede didara ṣaaju ṣiṣejade.
- Awọn idahun ilẹ pẹlu awọn orisun ti o gbẹkẹle nigbakugba ti deede ba ṣe pataki.
- Jeki aaye ayẹwo atunyẹwo eniyan fun awọn abajade ti o ga julọ.
- Tọpinpin awọn ilana ikuna ati tunṣe awọn itọsi tabi ṣiṣan iṣẹ nigbagbogbo.
Tesiwaju Ṣiṣawari
Ìwé ìròyìn ọfẹ
Gba ìkànìyàn AI ojoojúmọ́
Ìtàn AI mẹta tó jẹ́rìí ní gbogbo owurọ́ ọjọ́ iṣẹ́, kọ́wé ní Gẹ̀ẹ́sì tó rọrùn. Ọfẹ́ fún gbogbo àkókò, kò sí ìpolówó.
Ìmeèlì kan ní gbogbo ọjọ́ iṣẹ́. Yọkúrò ní ìkànìyàn ní ìkànìyàn kan. A kò ta tàbí pín àdírẹ́sì rẹ.
Ṣe idanwo fun ara rẹ
Take the Speculative Sampling Verification quiz
Ìdáhùn lẹ́sẹ̀kẹsẹ̀ lórí gbogbo ìdáhùn, àti iwe-ẹ̀rí tó lè pin pẹ̀lú ID tó lè jẹ́rìí lẹ́yìn tí o bá kọ́ ẹ̀kọ́.
Ṣe àtìlẹ́yìn ẹ̀kọ́ AI ọfẹ́. AI Understanding jẹ́ àjọ àìní èrè 501(c)(3) — kò sí ìpolówó, kò sí àkúnya owó, ní gbogbo àkókò. Ṣe ẹbun kan
Awọn ibeere ti a beere nigbagbogbo
Kini Ijerisi Iṣapẹẹrẹ Speculative?
Speculative sampling uses a draft model to propose tokens and a target model to verify them. With the exact rejection-sampling correction, it preserves the target distribution; speedup depends on draft cost, acceptance, target verification and workload.
Kini ero pataki ti o wa lẹhin iṣapẹẹrẹ arosọ?
Awoṣe iyaworan olowo poku ṣe amoro ọpọlọpọ awọn ami iwaju, ati awoṣe ibi-afẹde gbowolori sọwedowo gbogbo wọn ni ọna ti o jọra siwaju nikan.
Kini idi ti iṣayẹwo akiyesi ko dinku didara iṣẹjade?
Atunse ijusile-iṣapẹrẹ ti a tunṣe ṣe iṣeduro awọn ami ti o gba ti pin ni deede bi awoṣe ibi-afẹde yoo ti ṣejade nikan.
In exact speculative sampling, what happens when a drafted token is rejected?
Exact speculative sampling uses a rejection-sampling correction and discards the later draft suffix.
Under exact speculative sampling, what is guaranteed about the output distribution?
The rejection-sampling rule preserves the target distribution; this does not guarantee the same sampled string for each random seed.
What information does the target verification pass provide in exact speculative sampling?
The target pass scores the draft proposals and supplies target probabilities used by the acceptance rule and residual correction; it has a computation cost.
Tesiwaju kikọ
Jẹmọ awọn itọsọna
Awọn itọsọna diẹ sii ti a yan fun koko yii