Ubuyobozi bwa tekiniki

Gutandukanya Prefill na Decode Gukora

Ubwubatsi bukora bugabanya imvugo nini yerekana urugero mubice bibiri bitandukanye - prefill na decode - ikanabikoresha kubidendezi bitandukanye bya GPU.

Umva iki gitaboAI-byavuzwe · ~2 min

Kuriyi page2 min soma
  1. Incamake
  2. Kwibira cyane
  3. Gushyira mu bikorwa Isi
  4. Kazoza Kutuzuye Kuzuza no Gukora Dode
  5. Kubishyira mubikorwa
  6. Komeza Ubushakashatsi
  7. Ibibazo bikunze kubazwa

Incamake

Ntacyo bitwaye kuko ibyo byiciro byombi bifite ubushake buke bwibikoresho, kandi kubahatira kumashini imwe isesagura ubushobozi kandi bikababaza ubukererwe.

Kwibira cyane

Iyo LLM isubije, ikora mubyiciro bibiri. Prefill isoma icyarimwe icyarimwe hanyuma yubake urufunguzo-agaciro (KV) cache; iyi nini nini, ibangikanye, kubara-guturika kuzuza ibice byimibare ya GPU. Decode noneho itanga ibimenyetso kimwekimwe, buri ntambwe isoma cache yose ya KV-yibuka-umurongo-wuzuye, uhuza-kubara. Kwiruka hamwe, prefill ndende ihagarika decode ya buriwese (umutwe-wumurongo uhagarika), kandi gukubita byombi bitera kwivanga. Gutandukana bishyira prefill kuri pisine imwe ya GPU hanyuma ukarangiza kurindi, kwimura cache cache hagati yabo hejuru yimikoranire yihuse nka NVLink cyangwa InfiniBand. Buri kidendezi cyarateguwe kandi gipimwa mu bwigenge, kunoza ibicuruzwa byiza, koroshya umurizo, no kureka ababikora bagakubita igihe-kuri-mbere-ikimenyetso-nigihe-cyo-gusohora-icyerekezo icyarimwe.

Gushyira mu bikorwa Isi

Ikiganiro cyunganira kuganira inzira ndende yerekana compte-iremereye ya prefill cluster, hanyuma imigezi isubiza uhereye kumyibutsa-optimiz ya decode cluster kugirango ukomeze kwandika neza.

NVIDIA Dynamo na vLLM bareke abakoresha bashire prefill zitandukanye hamwe na decode yitsinda ryabakozi kugirango guturika kwigihe kirekire ntibishobora guhagarika ibisekuru bikomeza.

Mooncake (ikoreshwa na Kimi ya Moonshot AI) itandukanya prefill na decode kandi ikongeramo pisine yagabanijwe ya KV-cache kugirango igabanye kwishura inshuro nyinshi mubipimo.

Serivisi yo kurangiza-kode yegurira pisine ntoya ya progaramu ya progaramu ngufi na pisine nini ya decode, kubera ko ikiguzi kinini kiva mugutanga ibimenyetso byinshi bisohoka.

Kazoza Kutuzuye Kuzuza no Gukora Dode

Witege ko gutandukana bizaba nkibisanzwe mubikorwa. Sisitemu nka DistServe, Splitwise, na Mooncake yarayamamaye, kandi vLLM na NVIDIA Dynamo ubu ubwato butandukanye. Ubushakashatsi burimo gusunika KV-cache kwimura optimizasiyo, guhuza cache no kongera gukoresha ibyifuzo, imbaraga zo kongera kuringaniza ibipimo bya prefill / decode munsi yimodoka ihindagurika, hamwe no guhuza cyane hamwe na prefix cishing hamwe na prefill chunked prefill. Mugihe imiterere ya Windows ikura miriyoni yikimenyetso, gutandukanya ibi byiciro biba ngombwa cyane kubiciro bikoresha neza, bitinze cyane.

Kubishyira mubikorwa

Impamvu ari ngombwa

  • Igiciro na bije. Ibyemezo byubwubatsi bitwara imikorere nigiciro cyimikorere kumyaka.
  • Ibyemezo bisobanutse. Ubuhanga bwa tekinike bufasha amakipe guhitamo umurongo ukwiye, ntabwo ari shyashya gusa.
  • Kugenzura ubuziranenge. Guhitamo neza bya injeniyeri bigabanya ibintu byizewe mubikorwa.

Witondere

  • Gutezimbere igipimo kimwe gishobora guhisha intege nke za sisitemu.
  • Ibikorwa Remezo no kubungabunga akenshi usanga bidahabwa agaciro.
  • Icyuho cyumutekano no kwitegereza birashobora kwiyongera uko sisitemu igenda igorana.

Intambwe ikurikira

  1. Sobanura ubukererwe, ubuziranenge, nigiciro cyibiciro mbere yo kubishyira mubikorwa.
  2. Ibipimo byerekana umutwaro ufatika hamwe namakuru yimiterere.
  3. Gukurikirana ibikoresho kubikosa, drift, ningaruka zabakoresha.
  4. Tegura inzira yo gusubiza ibyabaye mbere yo gupima.

Komeza Ubushakashatsi

Ibaruwa y'amakuru y'ubuntu

Ibuka amakuru ya buri munsi ya AI

Inkuru eshatu zizewe za AI buri gitondo cy'iminsi y'akazi, zanditse mu rurimi rworoshye. Ubuntu iteka ryose, nta kwamamaza.

Imeri imwe buri munsi y'akazi. Kwiyimura mu buryo bumwe. Ntitwigera tugurisha cyangwa ngo dusangire aderesi yawe.

Isuzume

Fata Prefgregated Prefill na Decode Gukora ikibazo

Ibisubizo ako kanya kuri buri kibazo, n'icyemezo gishobora gusangirwa gifite ID yagenzurwa umaze gutsinda isomo.

Tangira ikibazo

Fasha uburezi bwa AI bw'ubuntu. AI Understanding ni umuryango utegamiye kuri leta wa 501(c)(3) — nta kwamamaza, nta kwishyura, iteka ryose. Tanga impano

Ibibazo bikunze kubazwa

Niki Gutandukanya Prefill na Decode Gukora?

Ubwubatsi bukora bugabanya imvugo nini yerekana urugero mubice bibiri bitandukanye - prefill na decode - ikanabikoresha kubidendezi bitandukanye bya GPU. Ntacyo bitwaye kuko ibyo byiciro byombi bifite ubushake buke bwibikoresho, kandi kubahatira kumashini imwe isesagura ubushobozi kandi bikababaza ubukererwe.

Niyihe mpamvu yibanze yibikoresho byo gutandukanya prefill na decode kubidendezi bitandukanye bya GPU?

Prefill itunganya ibintu byose muburyo bubangikanye kandi yuzuza compte, mugihe decode isoma cache cache buri ntambwe kandi igarukira kububiko bwibuke-butandukanye nibyifuzo byerekana ibidengeri bitandukanye, byigenga.

Ni ubuhe buryo bw'imibare bugomba kwimurwa kuva kubakozi bambere kugeza kubakozi?

Prefill yubaka cache cache kubibazo; decode ikeneye iyo cache kugirango ikomeze kubyara, bityo cache yoherejwe hejuru yihuta ihuza pisine.

Ni ikihe kibazo gutandukana bitagabanije muburyo busangiwe-GPU?

Kuri GPU isangiwe birebire biturika birashobora guhagarika intambwe ya decode ikomeza; kubitandukanya birinda ibyo kwivanga no guhagarika ubukererwe bwumurizo.

Ni ukubera iki prefill ishobora gutegurwa bikabije ariko ikagabanya inyungu ziva muburyo butandukanye?

Kuzuza ibintu byose byihuta hamwe, bityo ibyiciro binini bigaburira tensor cores neza; decode itanga ikimenyetso kimwe icyarimwe kandi igashyirwaho na memoire, bityo igapima ukundi.

Ni ubuhe buryo buhuza busanzwe bukoreshwa mu kwimura KV cache hagati y'ibidendezi bitandukanijwe?

Umuyoboro mwinshi, umuyoboro muto-utinda nka NVLink (intra-node) na InfiniBand (inter-node) birakenewe kugirango ihererekanyabubasha rya KV ntirihinduka icyuho gishya.