Hasashen Yawo da Hasashen Multi-Token
Hasashen yawo da hasashen alamu da yawa suna haɓaka ƙirar ƙirar harshe ta hanyar ƙididdige alamu da yawa na gaba lokaci ɗaya da kuma tabbatar da su a cikin fasfo ɗaya, maimakon samar da alama ɗaya a lokaci guda.
Dubawa
They cut latency without changing the text the model would have written.
Zurfafa nutsewa
Decoding autoregressive na al'ada yana jinkirin saboda kowane alamar yana buƙatar cikakken izinin wucewa kuma ana samar da alamun gaba ɗaya bayan ɗaya, yana barin GPU mara amfani. Ƙididdigar ƙididdigewa tana gyara wannan tare da mai ƙira mai arha wanda ke ba da shawara ga gunkin alamun ɗan takara, wanda babban samfurin manufa sannan ya tabbatar a layi daya; duk wani kari da ya yi daidai da abin da maƙasudin zai samar ana karɓa kyauta, kuma an gyara rashin daidaituwa na farko. Hasashen yawo da salo iri-iri na Medusa na ninke mai ƙira a cikin ƙirar kanta: ƙarin shuwagabannin hasashen masu nauyi (ko rafi na alamun hasashe) bar ƙira ɗaya duka daftarin kuma tabbatarwa, guje wa keɓan ƙirar ƙirar. Saboda tabbatarwa daidai ne, rarrabawar kayan aiki iri ɗaya ne da daidaitaccen yanke hukunci, kawai kuna samun ƙarancin matakai sau 2 zuwa 3.
Fahimtar Fasaha
Makullin shi ne cewa na'urar taswira na iya samun matsayi da yawa a cikin wucewa ɗaya mai arha kamar ɗaya, tunda yana da ɗauren ƙwaƙwalwar ajiya-bandwidth, ba ƙididdigewa ba, yayin yanke hukunci. Shugabannin hasashen da yawa suna fitar da alamun ɗan takara don mukamai da yawa na gaba; bishiya ko jerin ƴan takara an tabbatar da su tare, kuma karɓa yana amfani da samfurin ƙi (ko madaidaicin haɗama) don haka alamun da aka yarda da su suna bin ainihin rarraba manufa. Tsawon da aka yarda kowane mataki yana ƙayyade saurin gudu.
Dabarun Tasiri
Kudin da kasafin kuɗi
Hukunce-hukuncen gine-gine suna haifar da aiki da tsadar aiki na shekaru.
Shawarwari masu haske
Ilimin fasaha yana taimaka wa ƙungiyoyi su zaɓi tari mai kyau, ba kawai sabon abu ba.
Kula da inganci
Zaɓuɓɓukan injiniya mafi kyau suna rage abin dogaro a cikin samarwa.
Makomar Hasashen Yawo da Hasashen Multi-Token
Hanyoyin hasashe na kai waɗanda ba su buƙatar wani samfuri daban-daban suna zama tsoho a cikin injunan ƙididdiga, kuma bincike yana tura ƙimar karɓa mafi girma tare da mafi kyawun daftarin shugabannin, ƴan takarar da aka tsara bishiya, da horar da ƙirar tushe tare don tsinkayar alama mai yawa (wanda kuma zai iya haɓaka inganci). Yi tsammanin waɗannan fasahohin za su haɗu tare da ƙididdigewa da batching don haka mataimakan hulɗa suna jin nan take ko da ƙirar ke girma.
Aiwatar da Gaskiyar Duniya
Yanke jinkirin mayar da martani na mai taimakawa taɗi ta 2 zuwa 3x ta amfani da ƙarin shugabannin tsinkaya irin na Medusa
Ƙara ƙididdige ƙididdiga na kai-tsaye zuwa uwar garken ra'ayi don haka babu wani samfurin daftarin aiki daban da ke buƙatar ɗaukar bakuncin
Ƙaddamar da ƙaƙƙarfan lambar inda dogon, alamar da za a iya tsinkaya ke gudana ana karɓar karɓa cikin manyan gungu
Rage farashin GPU akan buƙatu ta hanyar ciro ƙarin alamu daga kowane fasfo na gaba mai ɗaure ƙwaƙwalwar ajiya
Hatsari & Tsare-tsare
Haɓaka ma'auni ɗaya na iya ɓoye manyan raunin tsarin.
Sau da yawa ana raina kayan more rayuwa da kuma kuɗin kulawa.
Tsaro da gibin lura na iya girma yayin da tsarin ke ƙara haɓaka.
Taswirar Hanya
Ƙayyade latency, inganci, da maƙasudin farashi kafin aiwatarwa.
Alamar ma'auni a ƙarƙashin ainihin kaya da yanayin bayanai.
Kula da kayan aiki don kurakurai, ɗigo, da tasirin mai amfani.
Shirya bijirowa da hanyoyin mayar da martani kafin sikeli.
Ci gaba da Bincike
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the Speculative Streaming and Multi-Token Prediction quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Jagora na gaba
Horon Hasashen Multi-Token
Tambayoyin da ake yawan yi
What is Speculative Streaming and Multi-Token Prediction?
Hasashen yawo da hasashen alamu da yawa suna haɓaka ƙirar ƙirar harshe ta hanyar ƙididdige alamu da yawa na gaba lokaci ɗaya da kuma tabbatar da su a cikin fasfo ɗaya, maimakon samar da alama ɗaya a lokaci guda. Sun yanke latency ba tare da canza rubutun da samfurin zai rubuta ba.
Me yasa daidaitaccen ƙaddamarwa na autoregressive ke yawan jinkiri akan GPU?
Kowace alamar tana buƙatar izinin wucewa ta gaba kuma suna da tsari sosai, don haka GPU ɗin yana daure-bandwidth na ƙwaƙwalwar ajiya kuma ba a yi amfani da shi ba yayin yanke hukunci.
Menene 'mawallafi' ke yi a cikin ƙima?
Mai ƙira mai arha yana ba da ɓangarorin alamun ɗan takara wanda babban samfurin manufa sannan ya tabbatar a layi daya.
Ta yaya ake kiyaye ingancin fitarwa a cikin tsinkayar hasashe?
Tabbatarwa (madaidaicin madaidaicin ko ƙirƙira samfurin ƙi) yana tabbatar da karɓuwa alamun suna bin daidaitaccen rarraba abin da abin da aka yi niyya zai samar, don haka fitarwa ba ta canzawa.
Menene ya bambanta hasashen nau'ikan nau'ikan nau'ikan Medusa da ƙirar ƙima?
Hanyoyin nau'in Medusa suna ninka rubutawa cikin ƙirar tare da ƙarin shugabannin tsinkaya, guje wa buƙatar ɗaukar nauyin ƙirar ƙirar daban.
Me yasa na'urar taransifoma zata iya tantance mukaman 'yan takara da yawa kusan da arha kamar na daya yayin yanke hukunci?
Yayin zazzage bakin kwalbar yana motsa ma'auni daga ƙwaƙwalwar ajiya, don haka ƙira ƙarin matsayi a cikin fasfo ɗaya yana ƙara ɗan ƙididdige farashi.