HejuruUbuyobozi bukurikira
Estimating GPU Memory Needed for LLMs
Tekiniki
Ubuyobozi bwa tekiniki
Ububiko Bwinshi bwa Memory Memory (HBM) ni ububiko bwibitse bushyizwe kuruhande rwa GPU butanga amakuru byihuse kuruta RAM isanzwe.
Umva iki gitaboAI-byavuzwe · ~2 min
It is what keeps AI accelerators fed, preventing the powerful compute cores from sitting idle while they wait for model weights and data.
HBM ikemura ikibazo cyibanze: chip ya kijyambere ya AI irashobora gukora trillioni yibikorwa kumasegonda, ariko mugihe amakuru ageze vuba bihagije. Ububiko busanzwe bwa GDDR buhuza hejuru ya bisi isa naho ifunganye, mugihe HBM itondekanya DRAM nyinshi ipfa guhagarikwa kandi ikabahuza ninsinga ibihumbi nini ziciriritse ziciriritse bita binyuze muri silicon vias (TSVs). Iyi stack yicaye kuri milimetero ya silicon interposer kuva kuri GPU, itanga inzira yagutse cyane, tekereza ibihumbi nibihumbi icyarimwe aho kuba amagana. Igisubizo ni umurongo wapimwe muri terabaýt kumasegonda. Ibisekuruza byateye imbere kuva HBM2 kugeza HBM2e, HBM3, na HBM3e, buri kimwe kizamura ubushobozi n'umuvuduko. Kuri moderi nini yindimi, uburemere bugomba guhora butambuka, ubushobozi bwa HBM hamwe numuyoboro mugari akenshi bifite agaciro kuruta kubara.
Gufata uburemere icumi cyangwa amagana ya gigabayiti yuburemere bwururimi runini rwegereye GPU kugirango zishobore gutambuka muri buri ntambwe zifatika.
Gushoboza NVIDIA H100 na H200 datacenter GPUs kugera kuri terabaýt nyinshi kumasegonda ya memoire ya memoire ya mahugurwa.
Guha ingufu amahugurwa ya AI aho GPUs nyinshi buri wese yishingikiriza kuri HBM kugirango yirinde guhagarara mubikorwa bya matrix.
Gushyigikira-hejuru-yerekana amashusho menshi na moderi ya videwo igomba kwimura tensors nini cyane kandi ikanibuka vuba.
Umuyoboro wibuke ubu ni imbogamizi iyobora kuri AI, bityo HBM iratera imbere byihuse. HBM3e yohereza mubyihuta byihuta, hamwe na HBM4 kuri horizon isezeranya intera nini, imirongo miremire, hamwe nubushobozi bwinshi kuri paki. Tegereza gufatanya gushushanya hagati yibuka na logique, birashoboka ko base base apfa no gutunganya-hafi-yibuka, wongeyeho amarushanwa akaze mubatanga nka SK hynix, Samsung, na Micron. Mugihe icyitegererezo gikura, kubona amakuru menshi hafi yo kubara, byihuse kandi ku mbaraga nkeya, biguma hagati yibikoresho bya AI gutera imbere.
Ibaruwa y'amakuru y'ubuntu
Inkuru eshatu zizewe za AI buri gitondo cy'iminsi y'akazi, zanditse mu rurimi rworoshye. Ubuntu iteka ryose, nta kwamamaza.
Imeri imwe buri munsi y'akazi. Kwiyimura mu buryo bumwe. Ntitwigera tugurisha cyangwa ngo dusangire aderesi yawe.
Isuzume
Ibisubizo ako kanya kuri buri kibazo, n'icyemezo gishobora gusangirwa gifite ID yagenzurwa umaze gutsinda isomo.
Fasha uburezi bwa AI bw'ubuntu. AI Understanding ni umuryango utegamiye kuri leta wa 501(c)(3) — nta kwamamaza, nta kwishyura, iteka ryose. Tanga impano
Ububiko Bwinshi bwa Memory Memory (HBM) ni ububiko bwibitse bushyizwe kuruhande rwa GPU butanga amakuru byihuse kuruta RAM isanzwe. Nicyo gituma umuvuduko wa AI ugaburirwa, ukabuza imbaraga za compte zikomeye kwicara ubusa mugihe bategereje uburemere bwikitegererezo hamwe namakuru.
AI chip irashobora gukora trillioni yibikorwa kumasegonda gusa iyo amakuru ageze vuba bihagije; HBM itanga umurongo mugari kuburyo cores ziticwa ninzara.
HBM itondekanya DRAM ipfa hejuru yundi kandi ikayihuza ninsinga zibihumbi zihagaritse (TSVs), ikora inzira yagutse cyane.
Aho kugira ngo byihute, HBM yerekana intera nini cyane (bits 1024 kuri stack no hejuru) bytes nyinshi zigenda icyarimwe.
Insinga ngufi kuri interposer isanganywe igabanya ingufu zikenewe kuri biti byimuwe kandi bigabanya ubukererwe hagati yo kwibuka na compte.
LLM umwanzuro uhoraho utambutsa matrices nini yuburemere, kubwibyo ubushobozi bwo kwibuka hamwe numuyoboro mugari akenshi biba ibintu bigabanya aho kubara ibicuruzwa.
Komeza wige
Abandi bayobozi batoranijwe kuriyi ngingo
HejuruUbuyobozi bukurikira
Estimating GPU Memory Needed for LLMs
Tekiniki