Tababarka saxda ah ee isku dhafan
Tababarka saxda ah ee isku dhafan waxa uu dadajiyaa tababbarka shabakada neerfaha waxana uu gooyaa isticmaalka xusuusta iyada oo ku samaynaysa inta badan xisaabta 16-bit barta sabaynaysa halkii ay ka ahaan lahayd 32-bit.
Dulmar
It lets the same GPU train bigger models faster with almost no loss in accuracy.
quusid qoto dheer
Tababar dhaqameedku wuxuu kaydiyaa miisaanka wuxuuna ku shaqeeyaa xisaabta 32-bit barta sabbaynaysa (FP32). Saxnimada isku dhafan waxay isticmaashaa qaabab 16-bit ah oo sax ah oo hooseeya (FP16 ama bfloat16) isku dhufashada matrixka culus, iyadoo la ilaalinayo 32-bit 'nuqul sare' ee miisaanka si loo cusbooneysiiyo xasilloon. Sababtoo ah tirooyinka 16-bit waa kala badh cabbirka, oo aad ugu habboon xusuusta GPU-ga iyo Kooreyada Tensor-ku waxay u habeeyaan qiyaas ahaan 2-8x si degdeg ah. Qabashadu waa kala duwanaanshaha cidhiidhiga ah ee FP16: jaranjarooyinka yaryar waxay hoos ugu qulquli karaan eber. Hagaajinta caadiga ah waa miisaanka luminta, kaas oo ku dhufanaya khasaaraha wax weyn ka hor faafinta si markaas gradients yaryar ay u noqdaan kuwo la matali karo, ka dibna dib u kala qaybiya ka hor cusboonaysiinta miisaanka. NVIDIA's Apex iyo AMP-ku-dhismay (Automatic Mixed Precision) ee PyTorch iyo TensorFlow ayaa tan otomaatig u ah.
Aragtida Farsamada
FP16 waxa ay leedahay 5 xabbo oo jibbaaran oo keliya, taas oo siinaysa kala duwanaansho yar oo firfircoon oo keenta qulqulka qulqulka. Bfloat16 waxay haysaa 8 xabbo oo jibbaaran (oo ku habboon kala duwanaanta FP32) laakiin mantissa qaniinyada yaryar, markaa dhif ayay u baahan tahay miisaan lumis - sabab muhiim ah Google TPUs iyo GPU-yada casriga ah ayaa jecel. Kooreyada Tensor-ku waxay dardargeliyaan shaqada iyagoo ku dhufanaya 16-bit operands laakiin uruurinaya qayb ahaan FP32, iyadoo la ilaalinayo saxnaanta halka khaladaadka soo koobid haddii kale ay isku biiraan.
Saamaynta Istiraatijiyadeed
Qiimaha iyo miisaaniyada
Go'aamada qaab-dhismeedku waxay horseedaan waxqabadka iyo kharashka hawlgalka sannadaha.
Go'aamo cad
Waxbarashada farsamada waxay ka caawisaa kooxaha inay doortaan xidhmo sax ah, ma aha oo kaliya kan ugu cusub.
Xakamaynta tayada
Doorashooyinka injineernimada ee wanaagsan waxay yareeyaan shilalka la isku halleyn karo ee wax soo saarka.
Mustaqbalka Tababarka Saxnaanta Isku Dhafan
Saxnimada ayaa hoos u sii socota. Tababarka FP8, oo lagu taageeray NVIDIA Hopper iyo Blackwell GPUs, ayaa noqonaysa halbeeg loogu talagalay moodooyinka xuduudaha, iyo cilmi baarista FP4 iyo qaababka microscaling (MXFP) ayaa sii riixaya. Filo qaab-dhismeedka si ay si toos ah u doortaan saxnaanta lakab kasta, qalab si ay u maamulaan qaabab weligood cidhiidhi ah, iyo tababbarka xog-ururinta si ay u kala fogeeyaan khadka u dhexeeya tababbarka saxda ah ee hooseeya iyo fikradda, hoos u dhigista qiimaha tababarka moodooyinka trillion-parameter.
Dhaqangelinta Adduunka-dhabta ah
PyTorch's torch.cuda.amp.autocast oo duubaysa wareegga tababarka si uu qiyaas ahaan kala badh u dhimo xusuusta iyo labanlaab u soo saarista hal GPU
Tababarka moodooyinka luqadaha waaweyn sida transformers-ka qaabka GPT ee bfloat16 ee TPU-yada si looga fogaado hagaajinta miisaan lumis
Ku rakibida cabbir weyn oo macaamiisha RTX GPU ah adiga oo ka beddelaya tababarka sawirka ResNet FP32 una beddelaya FP16
FP8 isku dhafan oo sax ah oo ku saabsan NVIDIA H100 GPUs si loo yareeyo kharashka ka hor tabobarrada moodooyinka xadka
Khatarta & Dariiqyada Ilaalada
Hagaajinta hal bartilmaameed waxay qarin kartaa daciifnimada nidaamka ballaaran.
Kaabayaasha dhaqaalaha iyo dayactirka inta badan waa la dhayalsadaa.
Nabadgelyada iyo daldaloolada u fiirsashada ayaa kori kara marka nidaamyadu noqdaan kuwo aad u adag.
Qorshe Hawleedka Dhaqangelinta
Qeex daahida, tayada, iyo bartilmaameedyada qiimaha ka hor inta aan la hirgelin.
Benchmark marka la eego culeyska dhabta ah iyo xaaladaha xogta.
La socodka qalabka khaladaadka, leexashada, iyo saamaynta isticmaalaha.
U diyaari dib-u-noqoshada iyo dariiqyada jawaab-celinta dhacdada ka hor inta aanad miisaan.
Sii wad Sahaminta
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the Mixed Precision Training quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Hagaha xiga
Calaamadaynta beenta ah iyo is-tababarka
Su'aalaha soo noqnoqda
What is Mixed Precision Training?
Tababarka saxda ah ee isku dhafan waxa uu dadajiyaa tababbarka shabakada neerfaha waxana uu gooyaa isticmaalka xusuusta iyada oo ku samaynaysa inta badan xisaabta 16-bit barta sabaynaysa halkii ay ka ahaan lahayd 32-bit. Waxay u ogolaataa isla GPU-ga inay si dhakhso leh u tababaraan moodooyinka waaweyn iyada oo aan wax khasaare ahi ka jirin saxnaanta.
Muxuu tababarka saxda ah ee isku dhafan u hayaa 32-bit 'koobiga sare' ee miisaanka?
Cusboonaysiinta miisaanka ayaa badanaa aad u yar; ku ururintooda 16-bit waxay luminaysaa saxnaanta, sidaa darteed nuqulka sayid-ku-buuxa ee saxda ah wuxuu ilaaliyaa cusboonaysiinta saxda ah.
Dhibaato noocee ah ayay miisaanka lumisku ku xalliyaa tababarka FP16?
FP16 waxa ay leedahay kala duwanaansho firfircoon oo xaddidan, sidaa awgeed jifyada yaryar waxay u wareegi karaan eber; ku dhufashada khasaaraha ka hor inta aysan dib-u-dhac ku hayn iyaga la matali karo.
Waa maxay faa'iidada ay bfloat16 ka leedahay FP16?
Bfloat16 waxay haysaa 8 xabbo oo jibbaaran sida FP32, markaa baaxaddeeda firfircooni waa weyn tahay oo qulqulka qulqulka hooseeya waa naadir.
Sidee buu Tensor Cores u ilaaliyaa saxnaanta marka la isticmaalayo 16-bit gelinta?
Kooreyada Tensor-ku waxay ku dhuftaan 16-bit operands laakiin waxay ku ururaan 32-bit, iyaga oo ka hortagaya khaladaadka soo koobida inay isku daraan.
Waa maxay faa'iidada aasaasiga ah ee isticmaalka 16-bit halkii aad ka isticmaali lahayd 32-bit qiimaha inta lagu jiro tababarka?
Nambarada cabbirka nuska ah waxay ku habboon yihiin xog badan oo ku jirta xusuusta GPU-da oo ha u oggolaadaan in qalabka khaaska ah uu farsameeyo xisaabta xisaabaadka dhowr jeer si degdeg ah.