Amamodeli Okusalungiswa Okuqanjwa Kwekhodi
Ukukhipha amakhodi okuqagela kusebenzisa imodeli 'esalungiswa' encane, esheshayo ukuqagela amathokheni ambalwa ezayo imodeli enkulu bese iwaqinisekisa ngephasi eyodwa.
Uhlolojikelele
It speeds up text generation 2-3x with no change to the output.
I-Deep Dive
Amamodeli olimi amakhulu akhiqiza umbhalo ithokheni eyodwa ngesikhathi, futhi isinyathelo ngasinye sidinga ukudlula okugcwele kwezigidigidi zamapharamitha — kuhamba kancane futhi kuboshwe inkumbulo. Ukukhipha amakhodi okuqagela kuhlasela lokhu ngokumatanisa imodeli 'eqondiwe' enkulu nemodeli 'esalungiswa' eshibhile. Imodeli esalungiswa iphakamisa ngokushesha ingxenye, ithi, amathokheni ekhandidethi angu-4-8. Imodeli enkulu ibe isiwacubungula wonke ngephasi eyodwa ehambisanayo bese ihlola ngayinye. Amathokheni afana nalokho imodeli enkulu ebingakhiqiza ayamukelwa; ukungafani kokuqala kuyalungiswa futhi okunye kulahlwe. Ngenxa yokuthi ukuqinisekisa amathokheni amaningana ngesikhathi esisodwa kubiza cishe okufanayo nokukhiqiza elilodwa, ukugijima okwamukelwe cishe kumahhala. Okubaluleke kakhulu, isinyathelo sokulahla-isampula siqinisekisa ukuthi ukusatshalaliswa kokugcina kuyafana nokusebenzisa imodeli enkulu iyodwa - isivinini ngaphandle kokulahlekelwa kwekhwalithi.
I-Technical Insight
Iqhinga elibalulekile wukuhlola okushintshiwe kwesampula-yenqaba. Kuthokheni ngayinye esakhiwe, amathuba emodeli eqondiwe aqhathaniswa namamodeli asalungiswa. Uma okuhlosiwe kunikeza amathuba alinganayo noma aphezulu, ithokheni iyamukelwa; uma kungenjalo yamukelwa ngamathuba alingana nesilinganiso, futhi lapho kwenqatshwa ithokheni elungisiwe ithathwa njengesampula ekusabalaliseni okuyinsalela okulungisiwe. Lesi zibalo senza okukhiphayo kufane ngokuphawulekayo nokusampula ngokuqondile kusukela kumodeli enkulu.
I-Strategic Impact
Isivinini nesikali
Ukugeleza komsebenzi wolimi kungahamba ngokushesha ngaphandle kokudela ukuvumelana.
Finyelela futhi ufinyelele
Yandisa ukufinyelela kuzo zonke izilimi nezitayela zokuxhumana.
Izinqumo ezicacile
Amaqembu angachitha isikhathi esiningi ekwahluleleni kuyilapho i-automation isingatha impinda.
Ikusasa Lamamodeli Okusalungiswa Okuqanjwa Kwekhodi
Lindela amamodeli asalungiswa ukuthi abe ingqalasizinda ejwayelekile kumaseva wokuqagela njenge-vLLM ne-TensorRT-LLM. Okuhlukile kokuziqagela (i-Medusa, UKHOZI) kwehlisa imodeli ehlukene yokusalungiswa ngokuphelele ngokungeza amakhanda okuqagela angasindi, futhi okusalungiswa okusekelwe esihlahleni kuqinisekisa ukuqhubeka kwekhandidethi okuningi ngesikhathi esisodwa. Njengoba amafasitela omongo akhula futhi izindleko zokuhlinzeka zibusa, abasalungiswa abafaniswa nemodeli abahlakaniphile kanye nokuqinisekiswa kwe-hardware-aware kuzonyusa amazinga okwamukelwa kanye nokusebenza phezulu.
Ukuqaliswa Komhlaba Wangempela
Anthropic, OpenAI, kanye Google basebenzisa ukubhala okucatshangelwayo ukuze kwehlise ukubambezeleka nokunikeza izindleko kubasizi bengxoxo abasebenzela izigidi zabasebenzisi.
I-vLLM kanye nomkhumbi we-NVIDIA TensorRT-LLM owakhelwe ngaphakathi ekubhaleni amakhodi okuqagela ukuze abasingathi ngokwabo bakwazi ukusheshisa ukuthunyelwa kwe-Llama noma kwe-Mistral.
Ukumatanisa imodeli yokusalungiswa engu-7B nethagethi engu-70B (isb., umndeni wakwa-Llama-3) ukuze icishe ibe amathokheni aphindwe kabili ngesekhondi ku-GPU eyodwa.
Amathuluzi okugcwalisa ikhodi asebenzisa imodeli encane yokusalungiswa ukuze enze isiphakamiso se-boilerplate eqinisekiswa yimodeli enkulu, okugcina iziphakamiso zisheshayo kusihleli.
Izingozi & Guardrails
Amaqiniso akhonjiwe angafaka ngokuthula imibiko, ukugeleza kosekelo, noma imiphumela yocwaningo.
Ukuzwela okusheshayo kungadala imiphumela engahambisani kuzo zonke izicelo ezifanayo.
Idatha yombhalo ebucayi ingase idalulwe uma izilawuli zokufinyelela zibuthakathaka.
Ukuqalisa Umhlahlandlela
Chaza ifomethi yokuphumayo, ithoni, namazinga wekhwalithi ngaphambi kokukhishwa.
Izimpendulo eziyisisekelo ngemithombo ethembekile noma nini lapho ukunemba kubalulekile.
Gcina indawo yokuhlola isibuyekezo somuntu ukuze uthole imiphumela ephezulu.
Landela amaphethini okuhluleka futhi uqeqeshe kabusha imiyalo noma ukuhamba komsebenzi njalo.
Qhubeka Uhlole
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the Speculative Decoding Draft Models quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Umhlahlandlela olandelayo
Ukuhlela Okuqagelwayo Kwamamodeli Ekhodi
Imibuzo evame ukubuzwa
What is Speculative Decoding Draft Models?
Ukukhipha amakhodi okuqagela kusebenzisa imodeli 'esalungiswa' encane, esheshayo ukuqagela amathokheni ambalwa ezayo imodeli enkulu bese iwaqinisekisa ngephasi eyodwa. Isheshisa ukukhiqizwa kombhalo 2-3x ngaphandle koshintsho kokuphumayo.
Iyiphi indima eyinhloko yemodeli 'esalungiswa' ekuqopheni amakhodi okuqagela?
Imodeli encane yokusalungiswa iqagela ngamathokheni azayo, okuthi imodeli enkulu eqondiwe bese iwahlola ngephasi eyodwa ehambisanayo.
Kungani ukuqinisekisa amathokheni abhaliwe amaningi ngesikhathi esisodwa kusindisa isikhathi?
Isizukulwane siboshelwe inkumbulo; ukuhlola amathokheni amaningana kuphasi elilodwa eliqoqiwe kucishe kushibhe njengesinyathelo esisodwa, ngakho-ke ukugijima okwamukelwe cishe kumahhala.
Kwenzekani kumathokheni abhaliwe ngemva kokuba ithokheni yokuqala imodeli eqondiwe yenqaba?
Ukwamukela kuqhubeke kuze kube ukungezwani kokuqala; lolo phawu luyalungiswa futhi wonke amathokheni alandelayo asalungiswa ayalahlwa.
Ukukhipha amakhodi okuqagela kuqinisekisa kanjani ukuthi ikhwalithi yokuphumayo ayonakaliswa?
Ukuhlolwa okuguquliwe kwesampula yokwenqaba kuqinisekisa ukufana kokugcina kokusabalalisa kwamathokheni amasampula asuka kumodeli eqondiwe.
Iyiphi kulokhu okuyindlela 'yokuzicabangela' egwema imodeli ehlukile yokusalungiswa?
I-Medusa ne-EAGLE zengeza amakhanda okuqagela angasindi engeziwe kumodeli eyinhloko ukuze ikwazi ukubhala amathokheni ayo esikhathi esizayo.