Xiga xigaHagaha xiga
Qiyaasta saxda ah ee AI iyo farmashiyaha
Farsamo
HAGAHA Farsamada
Tababarka saxda ah ee isku dhafan
Mixed precision uses lower-precision arithmetic for selected operations while retaining precision where needed.
Dhageyso hagahanAI-waa laga sheekeeyay · ~2 min
Boggaan2 daqiiqo akhri
Dulmar
It can reduce memory use and improve throughput on supported hardware, but gains and numerical effects depend on the model, workload, hardware and training recipe.
quusid qoto dheer
Traditional training stores weights and runs math in 32-bit floating point (FP32). Mixed precision uses lower-precision FP16 or bfloat16 for eligible operations while retaining FP32 where needed. Some FP16 recipes maintain a separate FP32 master copy to accumulate small weight updates; autocast workflows may instead keep model parameters in FP32 while casting selected operation inputs. Using 16-bit tensors can reduce memory used by eligible operations. Supported accelerators may execute some low-precision operations faster, but end-to-end throughput depends on hardware, workload and which operations autocast selects. FP16 has a narrower range, so tiny gradients can underflow. Loss scaling multiplies the loss before backpropagation and reverses that scale before the optimizer update; tools such as torch.amp.GradScaler can manage this process. Bfloat16 has a wider exponent range than FP16 and often does not need the same loss-scaling strategy, though model and hardware behavior still need testing.
Dhaqangelinta Adduunka-dhabta ah
PyTorch's torch.cuda.amp.autocast oo duubaysa wareegga tababarka si uu qiyaas ahaan kala badh u dhimo xusuusta iyo labanlaab u soo saarista hal GPU
Tababarka moodooyinka luqadaha waaweyn sida transformers-ka qaabka GPT ee bfloat16 ee TPU-yada si looga fogaado hagaajinta miisaan lumis
Ku rakibida cabbir weyn oo macaamiisha RTX GPU ah adiga oo ka beddelaya tababarka sawirka ResNet FP32 una beddelaya FP16
FP8 isku dhafan oo sax ah oo ku saabsan NVIDIA H100 GPUs si loo yareeyo kharashka ka hor tabobarrada moodooyinka xadka
Mustaqbalka Tababarka Saxnaanta Isku Dhafan
Saxnimada ayaa hoos u sii socota. Tababarka FP8, oo lagu taageeray NVIDIA Hopper iyo Blackwell GPUs, ayaa noqonaysa halbeeg loogu talagalay moodooyinka xuduudaha, iyo cilmi baarista FP4 iyo qaababka microscaling (MXFP) ayaa sii riixaya. Filo qaab-dhismeedka si ay si toos ah u doortaan saxnaanta lakab kasta, qalab si ay u maamulaan qaabab weligood cidhiidhi ah, iyo tababbarka xog-ururinta si ay u kala fogeeyaan khadka u dhexeeya tababbarka saxda ah ee hooseeya iyo fikradda, hoos u dhigista qiimaha tababarka moodooyinka trillion-parameter.
Ku dhaqangelinta
Maxay muhiim u tahay
- Qiimaha iyo miisaaniyada. Go'aamada qaab-dhismeedku waxay horseedaan waxqabadka iyo kharashka hawlgalka sannadaha.
- Go'aamo cad. Waxbarashada farsamada waxay ka caawisaa kooxaha inay doortaan xidhmo sax ah, ma aha oo kaliya kan ugu cusub.
- Xakamaynta tayada. Doorashooyinka injineernimada ee wanaagsan waxay yareeyaan shilalka la isku halleyn karo ee wax soo saarka.
Iska ilaali
- Hagaajinta hal bartilmaameed waxay qarin kartaa daciifnimada nidaamka ballaaran.
- Kaabayaasha dhaqaalaha iyo dayactirka inta badan waa la dhayalsadaa.
- Nabadgelyada iyo daldaloolada u fiirsashada ayaa kori kara marka nidaamyadu noqdaan kuwo aad u adag.
Tallaabooyinka xiga
- Qeex daahida, tayada, iyo bartilmaameedyada qiimaha ka hor inta aan la hirgelin.
- Benchmark marka la eego culeyska dhabta ah iyo xaaladaha xogta.
- La socodka qalabka khaladaadka, leexashada, iyo saamaynta isticmaalaha.
- U diyaari dib-u-noqoshada iyo dariiqyada jawaab-celinta dhacdada ka hor inta aanad miisaan.
Sii wad Sahaminta
Warqad bilaash ah
Hel warbixinta maalinlaha ah ee AI
Saddex sheeko oo la xaqiijiyay oo AI ah maalin kasta oo shaqo ah, oo lagu qoray Ingiriis fudud. Bilaash had iyo jeer, aan xayaysiis lahayn.
Hal iimayl maalin kasta oo shaqo ah. Ka bax hal gujin. Marna ma iibinno ama ma wadaagno ciwaankaaga.
Is tijaabi
Take the Mixed Precision Training quiz
Fikrad degdeg ah oo ku saabsan jawaab kasta, iyo shahaado la wadaajin karo oo leh aqoonsi la xaqiijin karo markaad ka gudubto koorso.
Taageer waxbarashada AI bilaashka ah. AI Understanding waa 501(c)(3) aan faa'iido doon ahayn — xayaysiis la'aan, albaab lacag ah ma jiro, had iyo jeer. Samee deeq
Su'aalaha soo noqnoqda
Waa maxay tababbarka saxda ah ee isku dhafan?
Mixed precision uses lower-precision arithmetic for selected operations while retaining precision where needed. It can reduce memory use and improve throughput on supported hardware, but gains and numerical effects depend on the model, workload, hardware and training recipe.
In an FP16 training recipe that keeps an FP32 master weight, why retain it?
Some FP16 recipes keep an FP32 master weight so small optimizer updates are not lost to low-precision rounding; not every mixed-precision recipe uses a separate copy.
Dhibaato noocee ah ayay miisaanka lumisku ku xalliyaa tababarka FP16?
FP16 waxa ay leedahay kala duwanaansho firfircoon oo xaddidan, sidaa awgeed jifyada yaryar waxay u wareegi karaan eber; ku dhufashada khasaaraha ka hor inta aysan dib-u-dhac ku hayn iyaga la matali karo.
Waa maxay faa'iidada ay bfloat16 ka leedahay FP16?
Bfloat16 waxay haysaa 8 xabbo oo jibbaaran sida FP32, markaa baaxaddeeda firfircooni waa weyn tahay oo qulqulka qulqulka hooseeya waa naadir.
For many supported Tensor Core matrix operations, how are partial sums commonly accumulated?
Many common Tensor Core operations accumulate in FP32 to reduce rounding error; this does not make the computation exact.
What is a common benefit of using 16-bit values for supported training operations?
Lower precision can reduce memory and speed supported operations, but total results depend on the model, hardware, and kernels used.
Sii wad waxbarashada
Tilmaamaha la xidhiidha
Tilmaamayaal badan ayaa loo doortay mawduucan