Ƙididdigar Hasashen
Ƙididdigar ƙididdiga ta sa manyan nau'ikan harshe su samar da rubutu cikin sauri ta amfani da ƙarami, samfurin 'daftarin aiki' mai sauri don kimanta alamu da yawa a gaba, sannan samun babban samfurin ya tabbatar da su gaba ɗaya.
Dubawa
It speeds up inference 2-3x with identical output quality.
Zurfafa nutsewa
A al'ada LLM yana haifar da rubutu alama ɗaya a lokaci guda: kowane alamar yana buƙatar cikakken wucewa ta ƙaton ƙirar, kuma ba za ku iya fara na gaba ba har sai na yanzu ya ƙare. Wannan yana jinkirin saboda yana da ɗaure ƙwaƙwalwar ajiya, ba mai ƙididdigewa ba - GPU yana kashe mafi yawan lokutan sa yana ɗaukar nauyi, ba yin lissafi ba. Ƙididdigar hasashe tana karya ƙulli. Karamin samfurin daftarin mai arha yana ba da shawarar guntun, a ce, alamun ɗan takara biyar. Babban samfurin 'manufa' sannan ya aiwatar da duka biyar a cikin fasinja guda ɗaya na gaba da duba su. Alamu waɗanda suka dace da abin da zai samar ana karɓa; a farkon sabani ya gyara ya watsar da sauran. Domin tabbatar da alamun da yawa suna kusan iri ɗaya da samar da ɗaya, zato da aka karɓa kusan kyauta ne.
Fahimtar Fasaha
Bangaren wayo shine ƙa'idar ƙima-samfur wanda ke ba da garantin rarraba kayan fitarwa ya yi daidai da lissafin lissafi da tafiyar da ƙirar da aka yi niyya kaɗai - don haka ingancin ba a kusanta ba, daidai ne. Adadin karɓa yana tafiyar da saurin gudu: mafi kyawun ƙaramin ƙirar yana tsinkayar babba, ƙarin alamar alama a kowane matakin tabbatarwa. Bambance-bambancen kamar Medusa suna ƙara ƙarin shugabannin tsinkaya ga ƙirar da aka yi niyya da kanta, da kuma zayyana EAGLE a cikin sararin samaniya, cire buƙatar ƙirar ƙirar daban.
Dabarun Tasiri
Kudin da kasafin kuɗi
Hukunce-hukuncen gine-gine suna haifar da aiki da tsadar aiki na shekaru.
Shawarwari masu haske
Ilimin fasaha yana taimaka wa ƙungiyoyi su zaɓi tari mai kyau, ba kawai sabon abu ba.
Kula da inganci
Zaɓuɓɓukan injiniya mafi kyau suna rage abin dogaro a cikin samarwa.
Makomar Ƙwararren Ƙwararru
Ƙididdigar ƙididdigewa yana zama tsoho a cikin hidimar tarawa kamar vLLM da TensorRT-LLM. Yi tsammanin hanyoyin zayyana kai (Medusa, EAGLE, Lookahead) za su mamaye tunda sun guji kiyaye samfuri na biyu, da hasashe na tushen bishiya wanda ke tabbatar da rassan ɗan takara da yawa kowane mataki. Yayin da samfura ke girma, ƙwaƙƙwaran ƙwaƙwalwar ajiya na ƙara tabarbarewa, yana yin hasashe har ma ya fi kima, kuma masu tsara kayan aiki za su tura saurin-sauri na gaske.
Aiwatar da Gaskiyar Duniya
Samfurin daftarin 7B yana ba da shawarar alamu don ƙirar hira ta 70B don yanke jinkirin amsawa a cikin mataimakan samarwa.
Shugabannin Medusa sun makale akan LLM don haka yana tsinkayar alamu da yawa na gaba lokaci guda ba tare da wani tsari na daban ba.
vLLM yana ba da damar ƙididdige ƙididdigewa don haɓaka abubuwan samarwa-da-biyu a kan gungu na hidima
Ƙirƙirar EAGLE a cikin ɓoyayyun sararin ƙirar ƙirar don haɓaka ƙimar karɓa da saurin gaba ɗaya
Hatsari & Tsare-tsare
Haɓaka ma'auni ɗaya na iya ɓoye manyan raunin tsarin.
Sau da yawa ana raina kayan more rayuwa da kuma kuɗin kulawa.
Tsaro da gibin lura na iya girma yayin da tsarin ke ƙara haɓaka.
Taswirar Hanya
Ƙayyade latency, inganci, da maƙasudin farashi kafin aiwatarwa.
Alamar ma'auni a ƙarƙashin ainihin kaya da yanayin bayanai.
Kula da kayan aiki don kurakurai, ɗigo, da tasirin mai amfani.
Shirya bijirowa da hanyoyin mayar da martani kafin sikeli.
Ci gaba da Bincike
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the Speculative Decoding quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Jagora na gaba
Ƙwarewar Hasashen da EAGLE
Tambayoyin da ake yawan yi
What is Speculative Decoding?
Ƙididdigar ƙididdiga ta sa manyan nau'ikan harshe su samar da rubutu cikin sauri ta amfani da ƙarami, samfurin 'daftarin aiki' mai sauri don kimanta alamu da yawa a gaba, sannan samun babban samfurin ya tabbatar da su gaba ɗaya. Yana haɓaka ƙima 2-3x tare da ingancin fitarwa iri ɗaya.
Menene ainihin ra'ayin ƙwanƙwasa hasashe?
Samfurin daftarin gaggawa yana ba da shawarar alamu da yawa, kuma babban samfurin manufa yana duba su duka a cikin layi ɗaya na layi ɗaya.
Me yasa tsarar LLM na al'ada ɗaya-a-lokaci ke jinkiri?
Kowane mataki yana ɗaukar manyan matrices masu nauyi daga ƙwaƙwalwar ajiya maimakon yin ƙididdigewa mai nauyi, don haka ba a amfani da GPU ba.
Menene zai faru a farkon alamar inda daftarin aiki da ƙirar ƙira suka saba?
Ana karɓar alamun har zuwa rashin jituwa; daga rashin daidaiton su gaba an ƙi su kuma ana kiyaye madaidaicin ƙirar ƙirar da aka yi niyya.
Ta yaya zaɓen hasashe ke shafar ingancin fitarwa?
Dokar kin amincewa-samfurin yana ba da garantin rarraba ƙarshe daidai daidai da ƙirar da aka yi niyya, don haka inganci ba ya canzawa.
Menene mafi ƙayyadaddun gudu kai tsaye daga ƙididdige ƙididdiga?
Da ƙarin tsararrun alamomin da abin da aka yi niyya ke karɓar kowane mataki na tabbatarwa, mafi girman saurin gudu.