Samfuran Ƙididdigar Ƙwararren Ƙwararru
Ƙididdigar hasashe tana amfani da ƙaramin ƙirar 'daftarin aiki' mai sauri don hasashen alamu masu zuwa da yawa waɗanda babban ƙirar ke tantancewa a cikin wucewa ɗaya.
Dubawa
It speeds up text generation 2-3x with no change to the output.
Zurfafa nutsewa
Manya-manyan ƙirar harshe suna haifar da rubutu alama ɗaya a lokaci guda, kuma kowane mataki yana buƙatar cikakken wucewa ta biliyoyin sigogi - a hankali da ɗaure ƙwaƙwalwa. Ƙididdigar ƙididdiga ta kai hari ga wannan ta hanyar haɗa babban samfurin 'manufa' tare da ƙirar 'daftarin' mai arha. Daftarin samfurin cikin hanzari yana ba da shawarar guntun, a ce, alamun ɗan takara 4-8. Babban samfurin sai ya aiwatar da su gaba ɗaya a cikin layi ɗaya na gaba kuma yana duba kowannensu. Alamun da suka dace da abin da babban samfurin zai samar an karɓa; an gyara rashin daidaiton farko sannan a watsar da sauran. Saboda tabbatar da alamu da yawa lokaci guda yana tsada kusan iri ɗaya da samar da ɗaya, rundunonin da aka karɓa kusan kyauta ne. Mahimmanci, matakin ƙima-samfur yana tabbatar da rarrabawar ƙarshe daidai yake da gudanar da babban ƙirar ita kaɗai - saurin ba tare da asara mai inganci ba.
Fahimtar Fasaha
Dabarar maɓalli shine gyare-gyaren ƙin yarda-samfur gwajin. Ga kowane daftarin alama, yuwuwar ƙirar ƙirar da aka yi niyya ana kwatanta ta da daftarin ƙirar. Idan makasudin ya ba da damar daidai ko mafi girma, ana karɓar alamar; in ba haka ba an yarda da shi tare da yuwuwar daidai da rabo, kuma a kan kin amincewa an ƙididdige alamar da aka gyara daga ragowar rabon da aka gyara. Wannan lissafin yana sa fitarwa ta zama daidai da yin samfur kai tsaye daga babban samfuri.
Dabarun Tasiri
Gudu da sikelin
Gudun aikin harshe na iya tafiya da sauri ba tare da sadaukar da daidaito ba.
Shiga ku isa
Yana faɗaɗa damar shiga cikin harsuna da salon sadarwa.
Shawarwari masu haske
Ƙungiyoyi za su iya ciyar da ƙarin lokaci akan hukunci yayin da aiki da kai ke sarrafa maimaitawa.
Makomar Hasashen Samfuran Zane-zane
Yi tsammanin daftarin samfuri don zama daidaitattun abubuwan more rayuwa a cikin sabobin ƙididdiga kamar vLLM da TensorRT-LLM. Bambance-bambancen hasashe na kai (Medusa, EAGLE) sun sauke keɓan ƙirar ƙirar gaba ɗaya ta ƙara shugabannin hasashen masu nauyi, da tsarar bishiya na tabbatar da ci gaban ɗan takara da yawa a lokaci ɗaya. Kamar yadda mahallin windows ke girma da farashin hidimar ya mamaye, mafi wayo, masu tsara ƙirar ƙira da tabbatarwa-warewar kayan aiki za su tura ƙimar karɓa da haɓaka mafi girma.
Aiwatar da Gaskiyar Duniya
Anthropic, OpenAI, da Google suna amfani da ƙwaƙƙwaran ƙididdigewa don rage jinkiri da biyan kuɗi akan mataimakan taɗi na hidimar miliyoyin masu amfani.
vLLM da NVIDIA TensorRT-LLM jirgin da aka gina a cikin ƙididdige ƙididdiga don masu ɗaukar nauyin kansu na iya hanzarta tura Llama ko Mistral.
Haɗa samfurin daftarin 7B tare da maƙasudin 70B (misali, dangin Llama-3) zuwa kusan alamun ninki biyu-dakika akan GPU ɗaya.
Kayan aikin da aka kammala lambar suna amfani da ƙaramin ƙirar ƙira don ba da shawarar tukunyar jirgi wanda mafi girman ƙirar ke tabbatarwa, yana kiyaye shawarwari masu daɗi a cikin edita.
Hatsari & Tsare-tsare
Abubuwan da aka ruɗe suna iya shigar da rahotanni cikin nutsuwa, kwararar tallafi, ko abubuwan bincike.
Hankali na gaggawa na iya ƙirƙirar sakamako mara daidaituwa a cikin buƙatun iri ɗaya.
Za a iya fallasa bayanan rubutu mai ma'ana idan ikon samun dama yana da rauni.
Taswirar Hanya
Ƙayyade tsarin fitarwa, sautin, da ma'auni masu inganci kafin fitowa.
Amsa a ƙasa tare da amintattun tushe a duk lokacin da daidaito ya shafi mahimmanci.
Ajiye wurin binciken ɗan adam don abubuwan da ake samu masu girma.
Bibiyar tsarin gazawar kuma sake horar da tsokaci ko tafiyar aiki akai-akai.
Ci gaba da Bincike
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the Speculative Decoding Draft Models quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Jagora na gaba
Gyaran Hasashen don Samfuran Code
Tambayoyin da ake yawan yi
What is Speculative Decoding Draft Models?
Ƙididdigar hasashe tana amfani da ƙaramin ƙirar 'daftarin aiki' mai sauri don hasashen alamu masu zuwa da yawa waɗanda babban ƙirar ke tantancewa a cikin wucewa ɗaya. Yana haɓaka haɓakar rubutu 2-3x ba tare da canji ga fitarwa ba.
Menene babban aikin samfurin 'daftarin' a cikin ƙididdige ƙididdiga?
Karamin daftarin samfurin yana yin hasashen alamu masu zuwa cikin arha, wanda babban samfurin da aka yi niyya sannan zai bincika a cikin layi ɗaya na layi ɗaya.
Me yasa tabbatar da ƙididdiga masu yawa a lokaci ɗaya ke adana lokaci?
Generation yana da iyakacin ƙwaƙwalwar ajiya; duba alamu da yawa a cikin fas ɗin batched yana da kusan arha kamar mataki ɗaya, don haka shirye-shiryen da aka karɓa kusan kyauta ne.
Menene zai faru da alamun da aka tsara bayan alamar farko da ƙirar da aka yi niyya ta ƙi?
Karɓar yana ci gaba har sai rashin jituwa ta farko; An gyara alamar kuma an zubar da duk abubuwan da aka tsara na gaba.
Ta yaya ƙwaƙƙwaran hasashe ke tabbatar da ingancin fitarwa ba ta ƙasƙanta ba?
Gwajin ƙin yarda da gyare-gyare yana ba da garantin rabon alamar ƙarshe daidai da samfurin kai tsaye daga ƙirar da aka yi niyya.
Wanne daga cikin waɗannan shine tsarin 'kishin kai' wanda ke guje wa wani tsari daban?
Medusa da EAGLE suna ƙara matakan hasashe masu nauyi zuwa babban ƙirar don ta iya tsara nata alamun nan gaba.