Amahugurwa avanze neza
Amahugurwa avanze neza yihutisha imyitozo ya neteur kandi agabanya imikoreshereze yibikorwa mukora imibare myinshi muri 16-bit kureremba aho kuba 32-bit.
Incamake
It lets the same GPU train bigger models faster with almost no loss in accuracy.
Kwibira cyane
Imyitozo gakondo ibika uburemere kandi ikoresha imibare muri 32-bit ireremba (FP32). Ivangavanze rivanze rikoresha imiterere-16-bito (FP16 cyangwa bfloat16) kugwiza matrix iremereye, mugihe ubitse 32-bit 'master kopi' yuburemere kugirango ivugururwe rihamye. Kuberako imibare 16-bit ni kimwe cya kabiri cyubunini, ikwiranye cyane na GPU yibuka na Tensor Cores itunganya hafi 2-8x byihuse. Ifatwa ni intera ntoya ya FP16: uduce duto duto dushobora gutemba kugeza kuri zeru. Igipimo gisanzwe ni igipimo cyo gutakaza, kigwiza igihombo kubintu byinshi mbere yo gusubira inyuma kuburyo gradients ntoya iguma ihagarariwe, hanyuma ikayigabanya inyuma mbere yo kuvugurura ibiro. Apex ya NVIDIA kandi yubatswe muri AMP (Automatic Mixed Precision) muri PyTorch na TensorFlow itangiza ibi.
Ubushishozi
FP16 ifite ibice 5 gusa byerekana, itanga urwego ruto rutera intambwe igenda itera. Bfloat16 ibika ibice 8 byerekana (bihuye nurwego rwa FP32) ariko bito bya mantissa bike, kubwibyo ntibikenera gukenera igihombo - impamvu nyamukuru Google TPU na GPU zigezweho zirabishyigikiye. Tensor Cores yihutisha umurimo mukugwiza 16-bit operands ariko ikusanya amafaranga igice muri FP32, ikabika neza aho amakosa yo guteranya yakwiyongera.
Ingaruka z'Ingamba
Igiciro na bije
Ibyemezo byubwubatsi bitwara imikorere nigiciro cyimikorere kumyaka.
Ibyemezo bisobanutse
Ubuhanga bwa tekinike bufasha amakipe guhitamo umurongo ukwiye, ntabwo ari shyashya gusa.
Kugenzura ubuziranenge
Guhitamo neza bya injeniyeri bigabanya ibintu byizewe mubikorwa.
Ejo hazaza h'amahugurwa avanze neza
Icyitonderwa gikomeza kugabanuka. Amahugurwa ya FP8, ashyigikiwe na NVIDIA Hopper na Blackwell GPUs, arimo kuba urugero rwiza rwimipaka, kandi ubushakashatsi muburyo bwa FP4 na microscaling (MXFP) butera imbere. Tegereza ibice kugirango uhitemo-guhitamo kuri buri cyiciro neza, ibyuma kugirango ukore kavukire kugirango uhindure imiterere irambuye, hamwe namahugurwa-yo kumenya umubare kugirango uhindure umurongo uri hagati yimyitozo ngororangingo idahwitse hamwe no gufata umwanzuro, bigabanya ikiguzi cyamahugurwa ya trillion-parameter.
Gushyira mu bikorwa Isi
Itara rya PyTorch.cuda.amp.autocast kuzinga imyitozo kugirango ugabanye hafi kwibuka no kwinjiza kabiri kuri GPU imwe
Guhugura imiterere nini yindimi nka GPT yuburyo bwa transformateur muri bfloat16 kuri TPU kugirango wirinde igihombo-gipima
Guhuza ingano nini kumuguzi RTX GPU uhindura imyitozo ya ResNet ishusho kuva FP32 ikajya kuri FP16
FP8 ivanze neza kuri NVIDIA H100 GPUs kugirango igabanye igiciro cyo kwipimisha imipaka-imipaka
Ingaruka & Kurinda
Gutezimbere igipimo kimwe gishobora guhisha intege nke za sisitemu.
Ibikorwa Remezo no kubungabunga akenshi usanga bidahabwa agaciro.
Icyuho cyumutekano no kwitegereza birashobora kwiyongera uko sisitemu igenda igorana.
Igishushanyo mbonera
Sobanura ubukererwe, ubuziranenge, nigiciro cyibiciro mbere yo kubishyira mubikorwa.
Ibipimo byerekana umutwaro ufatika hamwe namakuru yimiterere.
Gukurikirana ibikoresho kubikosa, drift, ningaruka zabakoresha.
Tegura inzira yo gusubiza ibyabaye mbere yo gupima.
Komeza Ubushakashatsi
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the Mixed Precision Training quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Ubuyobozi bukurikira
Pseudo-Kwandika no Kwitoza
Ibibazo bikunze kubazwa
What is Mixed Precision Training?
Amahugurwa avanze neza yihutisha imyitozo ya neteur kandi agabanya imikoreshereze yibikorwa mukora imibare myinshi muri 16-bit kureremba aho kuba 32-bit. Kureka GPU imwe igahugura moderi nini byihuse nta gihombo mubyukuri.
Ni ukubera iki imyitozo ivanze neza ituma 32-bit 'master copy' yuburemere?
Kuvugurura ibiro akenshi ni bito cyane; kubiteranya muri 16-bit byatakaza ubusobanuro, kubwibyo byuzuye-byuzuye kopi ikomeza ivugurura neza.
Ni ikihe kibazo cyo gupima igihombo gikemura mumahugurwa ya FP16?
FP16 ifite imipaka igarukira, kuburyo gradients nto zishobora kuzenguruka kuri zeru; kugwiza igihombo mbere yinyuma ituma bahagararirwa.
Ni izihe nyungu bfloat16 ifite kuri FP16?
Bfloat16 ibika ibintu 8 byerekana nka FP32, urwego rwayo rero ni runini kandi buhoro buhoro ni gake.
Nigute Tensor Cores ibika neza mugihe ukoresha 16-biti?
Tensor Cores igwiza 16-bit ikora ariko ikusanya muri 32-bit, ikabuza amakosa yo guteranya guterana.
Ni izihe nyungu zibanze zo gukoresha 16-bit aho gukoresha 32-biti mugihe cy'amahugurwa?
Igice cya kabiri cyimibare ihuye namakuru menshi mububiko bwa GPU kandi ureke ibyuma byabigenewe byihariye matrix imibare inshuro nyinshi byihuse.