Kugenzura Kugenzura no Guhugura
Ubuhanga bwo kuzigama imiterere yicyitegererezo mubice (shards) kugirango moderi nini irashobora gukizwa no kwongera kwipakurura utiriwe unanirwa kububiko cyangwa imipaka ya disiki, bityo kwiruka gukora impanuka birashobora gufata neza neza aho bigeze.
Incamake
Essential for any training job that runs for days or weeks across many GPUs.
Kwibira cyane
Igenzura ryamahugurwa nigishushanyo cyibintu byose bikenewe kugirango usubukure: uburemere bwikitegererezo, optimizer ivuga, gahunda yo kwiga-igipimo, umwanya wabatwara amakuru, nimbuto zitanga umubare. Kuri moderi nini iyi snapshot irashobora kuba amagana ya gigabytes, nini cyane kuri dosiye imwe cyangwa ububiko bwimashini imwe. Kugenzura ibice bigabanya ifoto yerekana amadosiye menshi ninzego nyinshi, buri GPU rero yandika ibice byayo gusa. Amahugurwa asubukuwe noneho yongere yongere asubiremo kandi agarure leta yuzuye neza. Bitabaye ibyo, ibyumweru byinshi biruka bigwa kumasaha 200 byagomba gutangira guhera. Ibikorwa nka PyTorch Yagabanijwe Kugenzura, Umuvuduko Wihuse, hamwe na Hugging Face Hub imiterere ya safetensors ikarishye ikora iyi gahunda.
Ubushishozi
Sharding ikora kuko ikwirakwizwa ryamahugurwa asanzwe agabanya uburemere hamwe na optimizas leta murwego (ukoresheje data, tensor, cyangwa ZeRO parallelism). Buri ntera ikurikirana ibice byayo gusa, akenshi kumiterere nka safetensors yemerera ubunebwe, ikarita yibikoresho. Indangantego ya dosiye ikarita yerekana amazina yama dosiye. Kugirango usubukure neza, sisitemu nayo ikomeza leta za RNG, optimizer intambwe yo kubara, hamwe na dataloader nyayo, bityo rerun ikabyara urutonde rumwe rwibyiciro.
Ingaruka z'Ingamba
Igiciro na bije
Ibyemezo byubwubatsi bitwara imikorere nigiciro cyimikorere kumyaka.
Ibyemezo bisobanutse
Ubuhanga bwa tekinike bufasha amakipe guhitamo umurongo ukwiye, ntabwo ari shyashya gusa.
Kugenzura ubuziranenge
Guhitamo neza bya injeniyeri bigabanya ibintu byizewe mubikorwa.
Ejo hazaza ha Kugenzura Kugenzura no Guhugura
Kugenzura ni uguhinduka kuva igihe cyo guhagarara-kwisi-ibintu ku kintu kidahuje kandi hafi yubusa. Tegereza byinshi murwibutso hamwe no kugenzura byanditseho ibice inyuma mugihe imyitozo ikomeje, wongeyeho gusiba-kode kandi ikoporora kuri bariyeri irokoka kunanirwa bikunze kugaragara ku gihumbi-GPU. Igicu cyibubiko hamwe nibyihuta byibanze bya NVMe bizakira ibice, kandi imiterere isanzwe nka safetensors izakomeza kunoza umutekano, byihuse, kwipakurura igice kubwamahugurwa asubukurwa no kohereza.
Gushyira mu bikorwa Isi
Imipaka-moderi ikora ibihumbi n'ibihumbi GPU ikoresha-ibika igenzura ryagabanijwe kuri buri ntambwe magana make kuburyo node imwe yananiwe igura iminota gusa, ntabwo ari iminsi.
Guhobera Isura ikwirakwiza moderi nini ifunguye nkibikoresho byinshi bya safetensors wongeyeho index.json kugirango abayikoresha bashobore gukuramo no kuyipakurura igice.
Umushakashatsi asubukuye umurongo-mwiza uhagaritse kugarura imbaraga za optimizer, intambwe yo kubara, hamwe na dataloader kugirango bikomeze nta nkomyi.
Umwanya-wamahugurwa kubiciro byigiciro cyambere GPUs, aho kugenzura kenshi kugenzura kureka akazi kakarokoka no kwimurwa.
Ingaruka & Kurinda
Gutezimbere igipimo kimwe gishobora guhisha intege nke za sisitemu.
Ibikorwa Remezo no kubungabunga akenshi usanga bidahabwa agaciro.
Icyuho cyumutekano no kwitegereza birashobora kwiyongera uko sisitemu igenda igorana.
Igishushanyo mbonera
Sobanura ubukererwe, ubuziranenge, nigiciro cyibiciro mbere yo kubishyira mubikorwa.
Ibipimo byerekana umutwaro ufatika hamwe namakuru yimiterere.
Gukurikirana ibikoresho kubikosa, drift, ningaruka zabakoresha.
Tegura inzira yo gusubiza ibyabaye mbere yo gupima.
Komeza Ubushakashatsi
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the Checkpoint Sharding and Resumable Training quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Ubuyobozi bukurikira
Slurm ya AI Amahugurwa
Ibibazo bikunze kubazwa
What is Checkpoint Sharding and Resumable Training?
Ubuhanga bwo kuzigama imiterere yicyitegererezo mubice (shards) kugirango moderi nini irashobora gukizwa no kwongera kwipakurura utiriwe unanirwa kububiko cyangwa imipaka ya disiki, bityo kwiruka gukora impanuka birashobora gufata neza neza aho bigeze. Ibyingenzi kumurimo uwo ariwo wose wamahugurwa ukora iminsi cyangwa ibyumweru muri GPU nyinshi.
Kuki igenzura rinini-ryigabanyijemo ibice?
Ibirindiro binini (amagana ya GB) ntibisanzwe nka dosiye imwe; Sharding ireka urwego rwinshi rwandika ibice byarwo kandi rushobora gupakira kimwe.
Usibye uburemere bw'icyitegererezo, ni iki kindi kigomba gusubirwamo gisanzwe gishobora kuzigama?
Kugirango usubukure neza, igenzura ryibika optimizer leta, random-numero-itanga leta, intambwe yo kubara, hamwe nuwashinzwe gutanga amakuru yagiye.
Ni izihe nyungu nyamukuru zamahugurwa asubukurwa kumirimo miremire?
Hamwe na bariyeri isubukurwa, ihagarikwa ryongeye gukora reta yanyuma yakijijwe kandi irakomeza, aho guta iminsi ya compte.
Nigute buri rwego rwa GPU rusanzwe rutanga umusanzu kuri bariyeri igabanijwe?
Kuberako amahugurwa yatanzwe asaranganya icyitegererezo murwego, buri ntera yandika ibice byayo gusa, bigatuma kuzigama kubangikanye no kwibuka neza.
Ni uruhe ruhare dosiye yerekana ifite muri bariyeri zagabanijwe?
Ironderero (urugero, index.json) inyandiko zerekana shard ifata buri kintu cyose kugirango abayitwara bashobore kuzana no guteranya ibice bikwiye.