Ci bi mu melGis bi ci topp
Réjection de échantillonnage de rejection
IA làkk
Làkk AI GUIDE
Saytu misaalu xalaat
Speculative sampling uses a draft model to propose tokens and a target model to verify them.
Déglul tegtal biiIA-nettali · ~2 min
Ci xët wii2 simili jàng
Résumé
With the exact rejection-sampling correction, it preserves the target distribution; speedup depends on draft cost, acceptance, target verification and workload.
Plongeur bu xóot
Autoregressive generation is slow because each token needs a full forward pass of a huge model. Speculative sampling fixes this by pairing a cheap draft model with the expensive target model. The draft proposes a short run of tokens (say 4-8); the target then scores all of them in one parallel forward pass. A modified rejection-sampling rule accepts the longest prefix that is consistent with the target's own distribution and resamples at the first rejected position. Because acceptance is probabilistic and corrected, the final token stream is provably distributed exactly as if the target had generated alone, no quality loss. Speedups vary with draft cost, acceptance rate, target verification, batching and hardware. A parallel verification pass can confirm multiple tokens, but is not free.
Doxal ci àdduna dëgg
Liggéey ci xeetu waxtaan 70B ak xeetu projet 7B ngir dagg latency tontu ci genn-wàll ak kalite génne bu nuróo.
Medusa-style boppu ci benn model di wax luy waaja am ci ay token yu bari, ba noppi di leen xool ndax amul benn reso bu wuute.
Decodage speculatif bu sukkandiko ci garab, di proposer ay branching yu bari yuy wéyal, ba noppi di leen xool ñépp ci benn pass target.
Gaawaale ndimmbalu yeggali kode fu modelu projet bi di jëfandikoo plaque chaudière buñu mëna wax luy waaja am, te model bu mag bi dafay gaaw firnde.
Ëlëgu Xool Sampling Spekulatif
Dekodaas speculatif mingi nekk standard ci stack inference yi. Variante yu bees yi dañuy bàyyi xeetu projet bu wuute: xalaat sa bopp dafay jëfandikoo boppu génn bu teel wala boppu prediction bu gëna bari (Medusa, EAGLE), projet bu sukkandiko ci garab dafay firndeel wéyal kàndidaa yu bari ci benn yoon, ak dekodaasu kanam buy parallelize n-gram guesses. Xaarandil lëkkaloo bu gëna dëgër ak yoriinu batching ak KV-cache, dimensionnement bu xam hardware bi, ak jëfandikoo bu gëna yaatu ci produit yu sensible ci latency lu melni assistant chat ak jumtukaayi codage fu millisecond bu nekk am solo.
Teg ko ci jëf
Lu tax mu am solo
- Gaawaay ak yaatuwaay. Liggéeyukaay yi ci làkk yi mën nañu gëna gaaw te duñu yàq deggoo gi.
- Dugg ak yegg. Dafay yaatal jëfandikoo gi ci làkk yi ak ci anam yi ñuy jokkoo.
- dogal yu gëna leer. Ekip yi mën nañu gëna yàgg ci àtte ci jamono ji otomatisation di liggéey ci baamtu.
Moytul
- Lépp lu jaarul yoon mën na dugg ci rapoor yi, jàppale ci liggéey bi, wala ci njariñu gëstu bi.
- Sensibilite bu gaaw mën na jur njariñ yu wuute ci laajte yu noonu mel.
- Done yu am solo mën nañu feeñ sudee seytu jëfandikoo gi néew doole.
Jéego yi ci topp
- Mandargal formaa génne gi, melokaan bi, ak standard kalite yi laata ngay dugal ko.
- Tontu yu am solo ak balluwaay yu wóor saa yu dëggu bi di am solo.
- Fexeel am barabu xool nit ñi ngir am njariñ yu am solo.
- Toppal anami gacce yi ak di faral di tàggataat ay laaj wala def-liggéey.
Weyal di banneexu
Bulletin bu amul fayda
Jàngal xibaar AI bu bes
Ñatti jaar-jaari IA yuñ firndeel bis bu nekk ci ayu-bis bi, ñu bind leen ci Àngle bu leer. Gratis ba fàww, amul yëgle.
Benn email bis bu nekk ci ayu-bis bi. Benn klik nga dindi abonemaa bi. Dunu musa jaay wala séddoo sa adres.
Nattal sa bopp
Take the Speculative Sampling Verification quiz
Feedback ci saasi ci bépp tontu, ak sertifikaa buñ mëna séddoo ak ID buñ mëna saytu soo jàllee ci njàng mi.
Jàppale njàngum IA te doo fay. AI Understanding mbootaay la buy def te Yàlla tax — amul yëgle, amul paywall, musul am. Defal ab sarax
Laaj yi ñuy faral di laaj
What is Speculative Sampling Verification?
Speculative sampling uses a draft model to propose tokens and a target model to verify them. With the exact rejection-sampling correction, it preserves the target distribution; speedup depends on draft cost, acceptance, target verification and workload.
Lan mooy xalaat bu mag bi ci ginaaw échantillonnage speculatif?
Benn modelu projet bu yomb dafay xayma ay token yu bari ci kanam, ba noppi modelu target bu seer bi dafay saytu leen ñépp ci benn pass forward parallèle.
Lan moo waral échantillonnage speculatif du yàq kalite génne gi?
Coppite biñu bañ-sampling correction dafay garanti jetons yiñ nangu ñu séddale leen ci anam wi model biñ bëgga defaree kese.
In exact speculative sampling, what happens when a drafted token is rejected?
Exact speculative sampling uses a rejection-sampling correction and discards the later draft suffix.
Under exact speculative sampling, what is guaranteed about the output distribution?
The rejection-sampling rule preserves the target distribution; this does not guarantee the same sampled string for each random seed.
What information does the target verification pass provide in exact speculative sampling?
The target pass scores the draft proposals and supplies target probabilities used by the acceptance rule and residual correction; it has a computation cost.
Weyal di jàng
Gid yu jëm ci loolu
Tann nañu yeneen njiit ngir topic bii