InoteveraGaidhi rinotevera
Cloud Cost Optimization for ML Workloads
Tekinoroji
Nhungamiro yehunyanzvi
Mutengo wakazara wevaridzi (TCO) wekuzvimiririra wega LLM ndizvo zvese zvinodhura kumhanyisa yakavhurika-huremu modhi uri wega kana hardware yekurenda.
Teerera kugwaro iriAI-yakarondedzerwa · ~3 min
Izvo zvinosanganisira maGPU, simba, network, software, vashandi veinjiniya, kusaita basa uye kukwidziridzwa, kwete chete mutengo weGPUs. Kuenzanisa iyo yakazara ne-per-token API mitengo inoratidza kana wega-yega inochengetedza mari. Mhinduro inowanzoenderana nekuchengeta Hardware yakabatikana nguva zhinji.
Kuzvigamuchira wega kunoreva kudhawunirodha yakavhurika-huremu modhi, seLlama, Mistral, Qwen kana Gemma musiyano, uye kuishandira pane zvivakwa zvaunotonga. Iwo anogona kunge ari maseva aunawo kana maGPU aunorenda mugore. Iwo uremu anogona kunge ari emahara kurodha, asi kumhanya nawo hakusi. Mutengo wese wekuve muridzi une akati wandei: - **Hardware.** MaGPU akatengwa anodzikira nekufamba kwemakore mashoma. MaGPU akarendwa anobhadharwa neawa angave akabatikana kana asina basa. Zvigadzirwa zvemidziyo zvinodawo simba, kutonhora, rack nzvimbo uye networking. - **Software uye mashandiro.** Unoda inference server sevLLM, SGlang kana NVIDIA TensorRT-LLM. Iwe zvakare unoda kuyera kuyera, kutarisa, kutema matanda, kuchengetedza kuomesa, uye redundancy kuitira kuti imwe yakundikana node isatore sevhisi pasi. - **Vashandi.** Uyu ndiwo unowanzo mutengo mukuru wakavanzika: mainjiniya anotumira, tune, chigamba nekusimudzira stack, pamwe nemumwe munhu parunhare. - **Kukwidziridzwa.** Mamodheru matsva anouya kazhinji. Kutama kwega kwega kunoreva kuongororazve, kugadzirisa zvakare uye dzimwe nguva kutenga Hardware nyowani. Kushandiswa kunosarudza mhedzisiro. An API inobhadharisa chete ma tokeni aunoshandisa. Iyo yega-yega GPU inodhura zvakafanana ingave ichishandira chikumbiro chimwe kana zviuru. Mutengo unoshanda pa tokeni ndiwo mutengo wepamwedzi wega wega wakakamurwa nematokeni anoshandiswa. Boka rinoshandiswa zvishoma rinogona kudhura zvakanyanya pane tokeni pane API, nepo iro rakaremerwa rakaremerwa rinogona kudhura zvakanyanya. Pfungwa nhatu dzisiridzo dzakajairika. Chekutanga ndechekuti GPU yekurenda mutengo nguva maawa inokwana mutengo, iyo inofuratira vashandi uye nguva isina basa. Chechipiri ndechekuti zviremu zvakavhurika zvinoreva mahara. Chechitatu ndechekuti yekuzviitisa inongozvivanzika. Sevha yakachengetedzwa zvisina kunaka inogona kuve nengozi kupfuura mupi ane akaongororwa zvinodzora. Pane zvikonzero zvakanaka zvekuzvigashira: - kutonga pamusoro pe data rako - mitengo inofanofungidzira pakakwirira, yakatsiga vhoriyamu - kugadzirisa kuburikidza nekugadzirisa zvakanaka - rusununguko kubva kune mupi achichinja kana kuregedza modhi.
Kambani inorenda maGPU kuti ishandise 70-bhiriyoni-parameter modhi yemubatsiri wemukati. Traffic inouya chete panguva dzehofisi, saka maGPU anogara asina chaanoita husiku huzhinji uye chiratidzo chega chega chinoguma chave kudhura kupfuura API yayakatsiva.
Bhangi rine mutemo wemukati unoda data rakadzama kuti rirambe riri pazvivakwa rinodzora wega-yepakati-saizi yakavhurika modhi munzvimbo yayo yedata. Inobvuma mutengo wepamusoro mukutsinhana nekutonga ikoko.
Pombi yegwaro-yemhando yepombi inotenderera wachi inoshandira diki diki modhi paGPU imwechete ine batching inoenderera. Mutengo waro pachiratidzo unopera zvakanaka pazasi pe API mitengo.
Iyo yekutanga bhajeti chete yegore GPU yekurenda. Inozoona kuti inoda injinjini pakufona kuti ibate kubuda, kutarisa, zvigamba zvekuchengetedza uye kusimudzira modhi, uye mutengo wevashandi unodarika bhiri rehardware.
Iko kuverengera kwega-hosting inoramba ichifamba mumativi ese. Yakavhurika-huremu modhi dzave kugona zvakanyanya, uye quantization uye inoshanda sevhisi software rega diki hardware ibate basa rinoshanda. Panguva imwecheteyo, mitengo yeAPI yakadonha, uye masevhisi akatarisirwa anowedzera kuvhurika mamodheru emubhadharo wechiratidzo, izvo zvinobvisa mutoro wakawanda wekushanda. Kune akawanda masangano maitiro angangove ari mahybrid: APIs e spiky kana miganhu yemabasa, uye yekuzvibata wega kune yakatsiga, yakakwira-vhoriyamu kana inonzwira tsitsi. Bata sarudzo sechinhu chekuongorora nguva dzose, kwete sarudzo yenguva imwe chete. Zvizvarwa zve Hardware, kuburitswa kwemodhi uye mitengo zvese zvinoshanduka mukati menguva yekuderera kweiyo nguva.
Tsamba yemahara
Nhatu dzakasimbiswa dzeAI nyaya mangwanani ega ega pakati pevhiki, dzakanyorwa neChirungu chakajeka. Yemahara zvachose, hapana ads.
Imwe email zuva rega rega revhiki. Bvisa kunyoresa kamwe chete. Hatimbotengese kana kugovera kero yako.
Zviedze iwe pachako
Mhinduro yepakarepo pamhinduro yega yega, uye chitupa chinogovaniswa chine ID inobatika kana wapedza kosi.
Tsigira dzidzo yemahara yeAI. AI Understanding ndeye 501(c)(3) isingabatsiri - hapana ads, hapana paywall, zvachose. Ita chipo
Mutengo wakazara wevaridzi (TCO) wekuzvimiririra wega LLM ndizvo zvese zvinodhura kumhanyisa yakavhurika-huremu modhi uri wega kana hardware yekurenda. Izvo zvinosanganisira maGPU, simba, network, software, vashandi veinjiniya, kusaita basa uye kukwidziridzwa, kwete chete mutengo weGPUs. Kuenzanisa iyo yakazara ne-per-token API mitengo inoratidza kana wega-yega inochengetedza mari. Mhinduro inowanzoenderana nekuchengeta Hardware yakabatikana nguva zhinji.
Pa 16 bits (2 bytes) paparameter, 70 bhiriyoni × 2 bytes inenge 140 GB. Iyo inopfuura imwechete 80 GB GPU, isati yawedzerwa chero KV cache memory.
Kugovanisa mari dzese dzepamwedzi, kusanganisira vashandi uye simba rekuita, nematokeni anoshandiswa zvinopa mufananidzo waunogona kuenzanisa zvakananga ne API per-tokeni mitengo.
An API mabhiri ekushandisa chete, asi ega-yakagadzirirwa hardware uye vashandi vanodhura zvakafanana zvisinei nemutoro. Kushandiswa kwakaderera kunoparadzira iyo mitengo yakagadziriswa pamusoro pematokeni mashoma.
Mabhajeti anowanzo kuverenga Hardware chete. Kumhanyisa sevhisi yakavimbika yekufungidzira kunotora nguva yeinjiniya, uye basa iro rinogona kudarika bhiri reGPU.
Iyo KV cache inochengetedza makiyi ekutarisisa uye kukosha kune zvinoshanda zvikumbiro. Mamiriro akareba uye vashandisi venguva imwe chete vanoda yakawanda.
Ramba uchidzidza
Mamwe madhairekitori akasarudzirwa nyaya iyi
InoteveraGaidhi rinotevera
Cloud Cost Optimization for ML Workloads
Tekinoroji