Saytu misaalu xalaat
Sampling speculatif dafay gaawlu defar model làkk bu mag ci bàyyi model 'projet' bu ndaw mu xalaat ay token yu bari ci kanam, ba noppi model bu mag bi xool leen ci benn yoon.
Résumé
The clever verification step guarantees the output matches what the big model would have produced on its own.
Plongeur bu xóot
Generation autorégressive dafa yeex ndax token bu nekk dafa soxla paas bu mat sëkk ci model bu rëy. Échantillonnage speculatif dafay saafara loolu ci boole modelu projet bu yomb ak modelu target bu seer bi. Projet bi dafay digle ay token yu gàtt (wax 4-8); target bi dugal leen ñépp ci benn paas paralel ci kanam. Regle buñ soppeeku ngir jël misaal, dafay nangu prefix bi gëna gudd te méngoo ak distribution bi ci boppam, ba noppi jëlaat misaal yi ci barab biñ njëkka bañ. Ndax nangu gi dafa am probabilite ak jubbanti, stream token bu mujj bi dafay séddale ndànk ni target bi defar boppam, amul benn perte ci kalite. Speedups yi ñuy gëna faral di def ñooy 2-3x sudee projet bi gaaw na bu baax, ndax dañuy firndeel token yu bari ci woote bu seer bu nekk.
Gis-gis xarala
Bépp token buñu bind, dangay méngale probabilite bi nga bëgga q ak probabilite biñ nara defar p. Nanngu ak wërsëg min(1, q/p); suñu ko bañee, jël misaal bi ci séddaleb residual buñ normalisee max (0, q-p). Regle bu bañ bii dafay tax séddale marginaal bi nuróo ak échantillon biñ bëgga jël. Paas paralel bi target bi itam dafay joxe distribution token bi ci topp 'gratuit' ginaaw token bi mujjee nangu, kon progress bi du musa taxaw.
njeextalu pexe
Gaawaay ak yaatuwaay
Liggéeyukaay yi ci làkk yi mën nañu gëna gaaw te duñu yàq deggoo gi.
Dugg ak yegg
Dafay yaatal jëfandikoo gi ci làkk yi ak ci anam yi ñuy jokkoo.
dogal yu gëna leer
Ekip yi mën nañu gëna yàgg ci àtte ci jamono ji otomatisation di liggéey ci baamtu.
Ëlëgu Xool Sampling Spekulatif
Dekodaas speculatif mingi nekk standard ci stack inference yi. Variante yu bees yi dañuy bàyyi xeetu projet bu wuute: xalaat sa bopp dafay jëfandikoo boppu génn bu teel wala boppu prediction bu gëna bari (Medusa, EAGLE), projet bu sukkandiko ci garab dafay firndeel wéyal kàndidaa yu bari ci benn yoon, ak dekodaasu kanam buy parallelize n-gram guesses. Xaarandil lëkkaloo bu gëna dëgër ak yoriinu batching ak KV-cache, dimensionnement bu xam hardware bi, ak jëfandikoo bu gëna yaatu ci produit yu sensible ci latency lu melni assistant chat ak jumtukaayi codage fu millisecond bu nekk am solo.
Doxal ci àdduna dëgg
Liggéey ci xeetu waxtaan 70B ak xeetu projet 7B ngir dagg latency tontu ci genn-wàll ak kalite génne bu nuróo.
Medusa-style boppu ci benn model di wax luy waaja am ci ay token yu bari, ba noppi di leen xool ndax amul benn reso bu wuute.
Decodage speculatif bu sukkandiko ci garab, di proposer ay branching yu bari yuy wéyal, ba noppi di leen xool ñépp ci benn pass target.
Gaawaale ndimmbalu yeggali kode fu modelu projet bi di jëfandikoo plaque chaudière buñu mëna wax luy waaja am, te model bu mag bi dafay gaaw firnde.
Risk yi ak balustrade yi
Lépp lu jaarul yoon mën na dugg ci rapoor yi, jàppale ci liggéey bi, wala ci njariñu gëstu bi.
Sensibilite bu gaaw mën na jur njariñ yu wuute ci laajte yu noonu mel.
Done yu am solo mën nañu feeñ sudee seytu jëfandikoo gi néew doole.
Roadmap ngir samp gi
Mandargal formaa génne gi, melokaan bi, ak standard kalite yi laata ngay dugal ko.
Tontu yu am solo ak balluwaay yu wóor saa yu dëggu bi di am solo.
Fexeel am barabu xool nit ñi ngir am njariñ yu am solo.
Toppal anami gacce yi ak di faral di tàggataat ay laaj wala def-liggéey.
Weyal di banneexu
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the Speculative Sampling Verification quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Gis bi ci topp
Réjection de échantillonnage de rejection
Laaj yi ñuy faral di laaj
What is Speculative Sampling Verification?
Sampling speculatif dafay gaawlu defar model làkk bu mag ci bàyyi model 'projet' bu ndaw mu xalaat ay token yu bari ci kanam, ba noppi model bu mag bi xool leen ci benn yoon. Jéego xool bu am xel bi dafay garanti ni génne gi méngoo ak li model bu mag bi mënoon defar boppam.
Lan mooy xalaat bu mag bi ci ginaaw échantillonnage speculatif?
Benn modelu projet bu yomb dafay xayma ay token yu bari ci kanam, ba noppi modelu target bu seer bi dafay saytu leen ñépp ci benn pass forward parallèle.
Lan moo waral échantillonnage speculatif du yàq kalite génne gi?
Coppite biñu bañ-sampling correction dafay garanti jetons yiñ nangu ñu séddale leen ci anam wi model biñ bëgga defaree kese.
Lan moo waral sampling speculatif mëna dem ca kanam doonte dañu bañ benn token ci diggu toppalante bi?
Benn paas biy jëm kanam mooy defar distribution token bi ci topp ginaaw token biñ mujjee nangu, kon soo bëggee benn token dafay awaase saa yu nekk.
Lan mooy 'self-speculatif' biy moytu ab xeetu projet bu wuute?
Medusa ak EAGLE yokk yeneen bopp wala jëfandikoo genn yu teel suko defee benn model bi digle tokens yu ëlëg, dindi soxla model bu wuute.