Ukusakaza-bukhoma okuqagelayo kanye nokubikezela kwamathokheni amaningi
Ukusakaza-bukhoma okuqagelayo nokubikezela kwamathokheni amaningi kusheshisa ukukhiqizwa kwemodeli yolimi ngokuqagela amathokheni ambalwa esikhathi esizayo kanye nokuwaqinisekisa ngokudlula okukodwa, esikhundleni sokukhiqiza ithokheni eyodwa ngesikhathi.
Uhlolojikelele
They cut latency without changing the text the model would have written.
I-Deep Dive
Ukukhipha ikhodi okuzenzakalelayo okujwayelekile kuhamba kancane ngoba ithokheni ngayinye idinga ukudlula okugcwele okuya phambili futhi amathokheni akhiqizwa ngokuqinile ngokulandelana, okushiya i-GPU ingasetshenziswa kancane. Ukukhipha amakhodi okuqagela kulungisa lokhu ngohlaka olushibhile oluphakamisa ingxenye yamathokheni ekhandidethi, imodeli enkulu eqondiwe bese iwaqinisekisa ngokuhambisana; noma isiphi isiqalo esifana nalokho okuhlosiwe obekungakukhiqiza siyamukelwa mahhala, futhi ukungafani kokuqala kuyalungiswa. Ukusakaza-bukhoma okuqagelayo kanye nokubikezela kwamathokheni amaningi kwesitayela se-Medusa kusonga okusalungiswa kumodeli ngokwayo: amakhanda okuqagela angasindi okwengeziwe (noma uchungechunge lwamathokheni aqagelayo) vumela imodeli eyodwa ukuthi ibhale futhi iqinisekise, kugwenywe imodeli ehlukile yokusalungiswa. Ngenxa yokuthi ukuqinisekiswa kunembile, ukusatshalaliswa kokuphumayo kuyafana nokukhishwa kwamakhodi okujwayelekile, uthola kalula izinyathelo ezilandelanayo eziphindwe izikhathi ezingu-2 kuya kwezingu-3.
I-Technical Insight
Isihluthulelo siwukuthi i-transformer ingakwazi ukuthola izikhundla eziningi ekudluleleni okukodwa okuya phambili ngokushibhile njengeyodwa, njengoba iboshelwe ku-memory-bandwidth, ayiboshiwe ngekhompyutha, ngesikhathi sokuqopha. Amakhanda amaningi okubikezela akhipha amathokheni ekhandidethi ezikhundleni ezimbalwa ezilandelayo; isihlahla noma ukulandelana kwamakhandidethi kuqinisekiswa ndawonye, futhi ukwamukelwa kusebenzisa isampula yokunqatshelwa (noma ukufanisa okuhahayo) ukuze amathokheni amukelekile alandele ukusatshalaliswa okuqondisiwe okuqondile. Ubude obamukelwe ngesinyathelo ngasinye bunquma ukusheshisa.
I-Strategic Impact
Izindleko kanye nesabelomali
Izinqumo zezakhiwo ziqhuba ukusebenza kanye nezindleko zokusebenza iminyaka.
Izinqumo ezicacile
Imfundo yobuchwepheshe isiza amaqembu ukuthi akhethe isitaki esifanele, hhayi nje esisha.
Ukulawulwa kwekhwalithi
Izinketho ezingcono zobunjiniyela zinciphisa izehlakalo ezinokwethenjelwa ekukhiqizeni.
Ikusasa Lokusakaza Okucatshangelwayo kanye Nokubikezelwa Kwezimpawu Eziningi
Izindlela zokuziqagela ezingadingi imodeli ehlukile yokusalungiswa ziba yizinjini ezizenzakalelayo, futhi ucwaningo lucindezela amanani okwamukelwa phezulu ngamakhanda asalungiswa angcono, amakhandidethi anesakhiwo sesihlahla, nokuqeqesha imodeli eyisisekelo ngokuhlanganyela ukubikezela kwamathokheni amaningi (okungase futhi kuthuthukise ikhwalithi). Lindela lawa masu ukuthi ahlanganiswe nokulinganisa nokuhlanganisa ukuze abasizi abasebenzisanayo bazizwe ngokushesha njengoba amamodeli ekhula.
Ukuqaliswa Komhlaba Wangempela
Ukusika ukubambezeleka kokuphendula komsizi wengxoxo ngo-2 ukuya ku-3x kusetshenziswa amakhanda okubikezela engeziwe esitayela se-Medusa
Ukwengeza ukuqopha okuzicabangela ngokwakho kuseva ye-inference ukuze kungabikho imodeli ehlukile yokusalungiswa okudingeka isingathwe
Ukusheshisa ukuqedwa kwekhodi lapho amathokheni amade, abikezelwayo amukelwa ngeziqephu ezinkulu
Ukunciphisa izindleko ze-GPU ngesicelo ngasinye ngokukhipha amathokheni engeziwe kuphasi ngalinye eliboshwe ngememori
Izingozi & Guardrails
Ukuthuthukisa ibhentshimakhi eyodwa kungafihla ubuthakathaka obubanzi besistimu.
Izindleko zengqalasizinda nezokulungisa zivame ukubukelwa phansi.
Izikhala zokuphepha nokubonakala zingakhula njengoba izinhlelo ziba nzima kakhulu.
Ukuqalisa Umhlahlandlela
Chaza ukubambezeleka, ikhwalithi, nezindleko ezihlosiwe ngaphambi kokuqaliswa.
Ibhentshimakhi ngaphansi komthwalo wangempela nezimo zedatha.
Ukuqapha amathuluzi amaphutha, ukukhukhuleka, nomthelela wabasebenzisi.
Lungiselela izindlela zokuhlehlisa nezigameko ngaphambi kokukala.
Qhubeka Uhlole
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the Speculative Streaming and Multi-Token Prediction quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Umhlahlandlela olandelayo
Ukuqeqeshwa Kwezibikezelo Zezimpawu Eziningi
Imibuzo evame ukubuzwa
What is Speculative Streaming and Multi-Token Prediction?
Ukusakaza-bukhoma okuqagelayo nokubikezela kwamathokheni amaningi kusheshisa ukukhiqizwa kwemodeli yolimi ngokuqagela amathokheni ambalwa esikhathi esizayo kanye nokuwaqinisekisa ngokudlula okukodwa, esikhundleni sokukhiqiza ithokheni eyodwa ngesikhathi. Basika ukubambezeleka ngaphandle kokushintsha umbhalo imodeli ebingawubhala.
Kungani ukukhishwa kwekhodi okuzenzakalelayo okujwayelekile kuvame ukuhamba kancane ku-GPU?
Ithokheni ngayinye idinga iphasi yayo eya phambili futhi ilandelana ngokuqinile, ngakho-ke i-GPU iboshelwe kumkhawulokudonsa wememori futhi ayisetshenziswanga kahle ngesikhathi kukhishwa amakhodi.
Wenzani 'umdidiyeli' ekuqopheni amakhodi okuqagela?
Umhleli oshibhile uphakamisa ingxenye yamathokheni ekhandidethi imodeli enkulu eqondiwe bese iwaqinisekisa ngokuhambisana.
Ikhwalithi yokuphuma igcinwa kanjani ekuqoqweni okucatshangelwayo?
Ukuqinisekisa (ukufanisa okunobugovu noma amasampula okwenqatshwa) kuqinisekisa ukuthi amathokheni amukelwayo alandela ukusatshalaliswa okuqondile imodeli eqondiwe ebingakhiqiza, ngakho okukhiphayo akushintshile.
Yini ehlukanisa ukubikezela kwamathokheni amaningi kwesitayela se-Medusa ekubhalweni kwakudala kokuqagela?
Izindlela zesitayela se-Medusa zigoqa okusalungiswa kumodeli ngamakhanda engeziwe wokuqagela, kugwenywe isidingo sokusingatha imodeli ehlukile yokusalungiswa.
Kungani i-transformer ingaqinisekisa izikhundla eziningi zamakhandidethi cishe eshibhile njengeyodwa ngesikhathi sokuqopha?
Ngesikhathi kukhishwa amakhodi i-bottleneck ihambisa izisindo kusuka enkumbulweni, ngakho-ke ukushaya izikhundla ezengeziwe ngephasi efanayo kunezela izindleko ezincane zokubala.