InoteveraGaidhi rinotevera
GPTQ uye AWQ Post-Training Quantization
Tekinoroji
Nhungamiro yehunyanzvi
DeepSpeed uye Megatron Kudzidzisa Stacks
DeepSpeed (Microsoft) neMegatron-LM (NVIDIA) ndiwo masisitimu esoftware anoita kuti mamodheru ekudzidzisa ane mabhiriyoni emaparamita muzviuru zveGPU zvigoneke.
Teerera kugwaro iriAI-yakarondedzerwa · ~2 min
Pa peji ino2 min verenga
Pfupiso
Pasina ivo, mamodheru emazuva ano haakwanise kukwana mundangariro kana kupedza kudzidziswa nenguva yakafanira.
Kudzika Kwakadzika
Kudzidzisa modhi hombe pane imwe GPU hazvigoneke nekuti uremu, magiradhi, uye optimizer nyika hazvikwane. Aya masheki anopatsanura basa kune akawanda maGPU. Megatron-LM yakapayona tensor parallelism, kuchekerera matrix ega ega mukati mechikamu chimwe nechimwe paGPUs, pamwe nepombi parallelism, iyo inoisa akasiyana maGPU akasiyana. DeepSpeed's siginecha mupiro ndeye ZeRO (Zero Redundancy Optimizer), iyo shards optimizer nyika, gradients, uye maparamita mhiri kweGPUs panzvimbo yekuzvidzokorora, kucheka per-GPU ndangariro zvinoshamisa. Iwo maviri anowanzo kusanganiswa (Megatron-DeepSpeed ) kudzidzisa mhando seBLOOM-176B uye Megatron-Turing NLG. Ivo zvakare vanowedzera yakasanganiswa-chaiyo, activation yekutarisa, uye kurodha kuCPU kana NVMe saka mahombe mamodheru anodzidzisa pane mashoma hardware.
Kuitwa muNyika yeChokwadi
Kudzidzisa yakavhurika mitauro yakawanda BLOOM-176B modhi uchishandisa yakasanganiswa Megatron-DeepSpeed stack mumazana emaGPU.
Microsoft neNVIDIA vachidzidzisa 530-bhiriyoni-parameter Megatron-Turing NLG modhi ine 3D parallelism.
ZeRO-Offload ichibvumira vaongorori kuti vagadzirise akawanda-mabhiriyoni-parameta modhi pane imwechete workstation GPU nekudurura optimizer nyika kuCPU RAM.
Kushandisa activation yekutarisa mune aya mastacks kuti ikwane kureba mamiriro windows nekudzokorodza ma activation pachinzvimbo chekuachengeta ese.
Ramangwana reDeepSpeed uye Megatron Kudzidzisa Stacks
Tarisira kubatanidzwa kwakasimba nePyTorch's yekuzvarwa FSDP (Fully Sharded Data Parallel), iyo yakatora akawanda ZeRO mazano, kudzima mutsetse pakati petsvakiridzo stacks uye musimboti masisitimu. Compiler-driven approaches uye otomatiki parallelism kuronga vanovavarira kubvisa manyorero tuning. Sezvo masumbu ekudzidziswa achikura akananga kumazana ezviuru zveanokwidziridza, kushivirira kukanganisa, elastic scaling, uye kupindirana kutaurirana nemakomputa inova iyo inotungamira miganhu yeinjiniya, pamwe nerutsigiro rwehutsva hutsva seNVIDIA Blackwell uye tsika yekudzidzira machipisi.
Kuzvishandisa
Nei zvichikosha
- Mutengo uye bhajeti. Zvisarudzo zvezvivakwa zvinotyaira kuita uye mutengo wekushandisa kwemakore.
- Sarudzo dzakajeka. Dzidzo yehunyanzvi inobatsira zvikwata kusarudza murwi wakakodzera, kwete iwo mutsva chete.
- Kudzora kwemhando yepamusoro. Sarudzo dzeinjiniya dziri nani dzinoderedza zviitiko zvekuvimbika mukugadzira.
Chenjera
- Kugadzirisa imwe bhenji kunogona kuvanza yakafara system kushaya simba.
- Infrastructure uye mari yekugadzirisa inowanzotarisirwa pasi.
- Chengetedzo uye kucherechedzwa mapundu anogona kukura sezvo masisitimu anowedzera kuoma.
Matanho anotevera
- Tsanangura latency, mhando, uye mutengo zvinangwa usati waitwa.
- Benchmark pasi pechokwadi mutoro uye data mamiriro.
- Chishandiso chekutarisa zvikanganiso, kudonha, uye mushandisi maitiro.
- Gadzirira nzira dzekudzosera kumashure uye dzezviitiko usati wawedzera.
Ramba Uchiongorora
Tsamba yemahara
Tora iyo yemazuva ese AI pfupiso
Nhatu dzakasimbiswa dzeAI nyaya mangwanani ega ega pakati pevhiki, dzakanyorwa neChirungu chakajeka. Yemahara zvachose, hapana ads.
Imwe email zuva rega rega revhiki. Bvisa kunyoresa kamwe chete. Hatimbotengese kana kugovera kero yako.
Zviedze iwe pachako
Take the DeepSpeed and Megatron Training Stacks quiz
Mhinduro yepakarepo pamhinduro yega yega, uye chitupa chinogovaniswa chine ID inobatika kana wapedza kosi.
Tsigira dzidzo yemahara yeAI. AI Understanding ndeye 501(c)(3) isingabatsiri - hapana ads, hapana paywall, zvachose. Ita chipo
Mibvunzo inowanzo bvunzwa
Chii chinonzi DeepSpeed uye Megatron Kudzidzisa Stacks?
DeepSpeed (Microsoft) neMegatron-LM (NVIDIA) ndiwo masisitimu esoftware anoita kuti mamodheru ekudzidzisa ane mabhiriyoni emaparamita muzviuru zveGPU zvigoneke. Pasina ivo, mamodheru emazuva ano haakwanise kukwana mundangariro kana kupedza kudzidziswa nenguva yakafanira.
Chii chinangwa chekutanga cheDeepSpeed's ZeRO optimizer?
ZeRO (Zero Redundancy Optimizer) inobvisa ndangariro kudzokororwa nekugovanisa optimizer nyika, gradients, uye maparamita mhiri kweGPUs pane kuzvidzokorora pane yega yega mudziyo.
Tensor parallelism, sekupayona muMegatron-LM, inotsemura modhi sei?
Tensor parallelism inopatsanura masvomhu mukati mechikamu chimwe chete (senge matrix hombe kuwanda) kuyambuka akawanda maGPU, iri intra-layer kupatsanura.
Ndeipi ZeRO nhanho inopa yakanyanya ndangariro kuchengetedza nekugovanisa modhi paramita pachayo?
ZeRO Stage 3 shards paramita mukuwedzera kune gradients uye optimizer nyika, kuzviunganidza pazvinodikanwa, zvichipa yakanyanya kuderedzwa ndangariro.
Chii chinonzi 'activation cheki' chinotengeserana kure kuchengetedza ndangariro panguva yekudzidziswa?
Activation yekutarisa inochengetedza mashoma epakati activation uye anoadzokorora panguva yekudzokera shure, kutengesa yakawedzera computation yekuderedzwa ndangariro.
Sei kusanganisa matekiniki aya kuchiwanzonzi '3D parallelism'?
3D parallelism inorongedza matatu orthogonal nzira: data parallelism, tensor (intra-layer) parallelism, uye pipeline (inter-layer) parallelism.
Ramba uchidzidza
Mazano anoenderana
Mamwe madhairekitori akasarudzirwa nyaya iyi