Ubuyobozi bwa GPU bwo kwibuka no gucamo ibice
Uburyo imikorere ya AI igenera, ikongera, kandi igasubirana ububiko buke kuri GPU, nimpamvu icyuho gisigaye (gucamo ibice) gishobora gutera amakosa yibuke nubwo yibuka tekinike nyinshi.
Incamake
Understanding it is key to fitting big models and avoiding mysterious crashes.
Kwibira cyane
Ububiko bwa GPU burakosowe kandi bufite agaciro: ikarita ishobora kuba ifite 24, 80, cyangwa 192 GB yose hamwe, isangiwe nuburemere bwikitegererezo, ibikorwa, gradients, leta nziza, hamwe na buffer zigihe gito. Guhamagara umushoferi kugirango agabanye kwibuka kuri buri gikorwa cyaba gahoro, bityo ibice nka PyTorch ukoreshe cishing yagabanije ifata ibinini binini imbere hanyuma igatanga uduce duto, hanyuma igakomeza ibice byidegembya muri pisine kugirango ikoreshwe. Ifatwa ni ugucamo ibice: nkuko mirongo yubunini butandukanye yagabanijwe kandi ikarekurwa, umwanya wubusa ucamo ibice bitatanye. Urashobora kugira 5 GB kubuntu muri rusange nyamara ukananirwa gutanga tensor ihuza 2 GB kuko nta cyuho na kimwe kinini bihagije. Iyi niyo mpamvu imyitozo ishobora guhanuka hamwe namakosa yo hanze yibuka nubwo bigaragara ko ari icyumba cyumutwe.
Ubushishozi
PyTorch's CUDA itanga cache igabanya kwibuka mumigezi ya bice hanyuma igakoresha bloks zidegembya zihuye nubunini bwasabwe, wirinda guhamagara cudaMalloc / cudaFree. Gucamo ibice bivuka mugihe ibice bitandukanijwe bidashobora guhuzwa. Ibikoresho nka torch.cuda.empty_cache, PYTORCH_CUDA_ALLOC_CONF ihitamo kwaguka_ibice, hamwe nibisobanuro bifasha kwibuka. Uburyo bushya bwo kuguza ibitekerezo-byibukwa byibitekerezo, gushushanya urupapuro rwumubiri rudahuza muburyo bugaragara kuburyo ibyifuzo binini bigenda neza nubwo bitandukanijwe.
Ingaruka z'Ingamba
Igiciro na bije
Ibyemezo byubwubatsi bitwara imikorere nigiciro cyimikorere kumyaka.
Ibyemezo bisobanutse
Ubuhanga bwa tekinike bufasha amakipe guhitamo umurongo ukwiye, ntabwo ari shyashya gusa.
Kugenzura ubuziranenge
Guhitamo neza bya injeniyeri bigabanya ibintu byizewe mubikorwa.
Ejo hazaza ha GPU yo kwibuka no gucamo ibice
Gucunga kwibuka bigenda birushaho kuba byiza no kurupapuro, byatewe na sisitemu y'imikorere. Tekinike nka virtual-memoire-yuburyo butangwa hamwe no kwitondera paji (ikoreshwa mugucunga cache cache mugihe cyo gufata umwanzuro) igabanya imyanda no gucikamo ibice. Tegereza ibyingenzi kugirango bishoboke kwaguka, kubitandukanya, kubigaragaza neza binyuze mumashusho yubatswe, hamwe no gufatana hamwe no gupakurura no kubara bityo sisitemu ihuza GPU, CPU, hamwe na disiki ya disiki mu buryo bwikora kugirango ikoreshwe hejuru kandi ikora impanuka zidasanzwe.
Gushyira mu bikorwa Isi
Amahugurwa yiruka akorana na 'CUDA hanze yububiko' nubwo ububiko bwabitswe bwerekana umwanya wubusa, bugenwa no gushiraho PYTORCH_CUDA_ALLOC_CONF kugirango ushoboze ibice byaguka.
Ukoresheje torch.cuda.memory_summary cyangwa ifoto yo kwibuka kugirango umenye ibice hamwe nibice birimo kurya GPU ya 80 GB.
vLLM's PagedIcyifuzo cyo gucunga neza KV cache mumapaji yagenwe kugirango akorere ibyifuzo byinshi byo kuganira bidatakaje kwibuka.
Kugabanya ingano yicyiciro cyangwa gushoboza kugenzura buhoro buhoro kugirango ugabanye ibikorwa byo kwibuka hanyuma wirinde gucikamo ibice-biturutse hanze yibuka.
Ingaruka & Kurinda
Gutezimbere igipimo kimwe gishobora guhisha intege nke za sisitemu.
Ibikorwa Remezo no kubungabunga akenshi usanga bidahabwa agaciro.
Icyuho cyumutekano no kwitegereza birashobora kwiyongera uko sisitemu igenda igorana.
Igishushanyo mbonera
Sobanura ubukererwe, ubuziranenge, nigiciro cyibiciro mbere yo kubishyira mubikorwa.
Ibipimo byerekana umutwaro ufatika hamwe namakuru yimiterere.
Gukurikirana ibikoresho kubikosa, drift, ningaruka zabakoresha.
Tegura inzira yo gusubiza ibyabaye mbere yo gupima.
Komeza Ubushakashatsi
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the GPU Memory Management and Fragmentation quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Ubuyobozi bukurikira
Gahunda ya GPU na Orchestre ya Cluster
Ibibazo bikunze kubazwa
What is GPU Memory Management and Fragmentation?
Uburyo imikorere ya AI igenera, ikongera, kandi igasubirana ububiko buke kuri GPU, nimpamvu icyuho gisigaye (gucamo ibice) gishobora gutera amakosa yibuke nubwo yibuka tekinike nyinshi. Gusobanukirwa ni urufunguzo rwo guhuza moderi nini no kwirinda impanuka zidasanzwe.
Gutandukanya GPU ni iki?
Gucamo ibice bisobanura ububiko bwuzuye bwubusa burahagije, ariko bwacitsemo ibice, kuburyo nta cyuho na kimwe kinini kinini bihagije kuri tensor nini.
Ni ukubera iki ibice nka PyTorch bikoresha cache yibikoresho?
Hamagara cudaMalloc / cudaFree kuri buri op iratinda, nuko uwagabanije cishing afata blok nini hanyuma akoresha ababohowe muri pisine.
Urabona 5 GB kubuntu ariko ntushobora gutanga tensor ya 2 GB. Ni izihe mpamvu zishobora kuba?
Iki kimenyetso cya kera cyo gutandukana kibaho mugihe ububiko bwubusa bubaho ariko ntabwo arikintu kimwe gihuza kinini bihagije kubisabwa.
Ni ubuhe buryo bwa PyTorch bufasha kugabanya gucamo ibice byemerera ibice byo kwibuka gukura neza?
Gushiraho PYTORCH_CUDA_ALLOC_CONF kugirango ushoboze kwaguka_ibice bituma uwagabanije gukura ibice kandi bikagabanya kunanirwa bijyanye no gucikamo ibice.
Niki Urupapuro rwa VLLM rwitondewe rushobora kugabanya imyanda yo kwibuka mugihe cyo gufata umwanzuro?
Urupapuro rwitondewe rubika cache cache mumapaji yubunini bugaragara nka OS yibuka ya OS, guca ibice no gutanga ibyifuzo byinshi neza.