FP8 ak formaa yu woyof
FP8 formaa nimero floating 8-bit la buy may model IA yi ñu denc ay poid ak def math ci jëfandikoo ñeent ci memory nimero 32-bit yiñ miin.
Résumé
It is a key trick for making giant models cheaper and faster to train and serve.
Plongeur bu xóot
Reseau neuronal yi dañu defaree ay miliyaar ciy lim. Bu njëkkoon, nimero yooyu dañu daan jëfandikoo 32 bit (FP32) wala 16 bit (FP16/BF16) bu nekk. FP8 daf leen wàññi ba 8 bit, dagg memory ak bandwidth ci genn-wàll ak 16-bit. Amna ñaari xeetu FP8 yuñ gëna xam: E4M3 (4 bit exponent, 3 bit mantissa) dafay joxe lu gëna leer waaye lu gëna ndaw, E5M2 (5 exponent, 2 mantissa) dafay joxe lu gëna yaatu waaye jéego yu gëna ñaawe. Kompromis bi mooy njub: bit yu néew mooy njuumte yu wër. Ngir baña gaawa jeex, kaadar yi dañuy jëfandikoo facteur scaling tensor bu nekk wala bloc bu nekk yuy scale valeur yi ci rang biñ mëna jëfandikoo ci FP8. GPU Hopper ak Blackwell yu NVIDIA yokk nañu ci motëri matrix FP8, muy lu baax ci tàggat ak ci jël doggal. Format yu bees yu melni MXFP8, MXFP4, ak NVFP4 dañuy gëna wàcci ak ay blok yu ñuy séddoo.
Gis-gis xarala
Jafe-jafe FP8 mooy rang dynamique. Ak ay bit exponent yu néew, aktivasioŋ yu mag wala yu ndaw dañuy fees wala ñuy wàcci ba zero. Fix bi mooy scaling: yokk benn tensor ak benn facteur suko defee valeur yi wàcci ci palanteer bi FP8 mëna representé, defal FP8 yokk-accumuler, ba noppi xaajalewaat, di faral di accumuler sommes partielles ci gëna dëggu (FP16/FP32). E4M3 dañu koy gëna jëfandikoo ngir diisaay ak aktivaasioŋ, E5M2 ngir degrade yu rang bi gëna am solo ci njub.
njeextalu pexe
Njëgg ak budget
Dogal yi architecture di jël dañuy indi njariñ ak njëgu liggéey bi ay at ci ginaaw.
dogal yu gëna leer
Njàngalem xarala yi dafay jàppale ekip yi ñu tànn li gën, te baña yam ci li gëna bees daal.
Xool kalite
Tanneef yu gëna baax ci wàllu ingeñër dina wàññi jafe-jafe yi ci wàllu wóor ci liggéey bi.
Ëlëgu FP8 ak formaa yu woyof yi
Precision mingi wàcci. Ginaaw FP8 dafa am formaa yu ndaw yu am 4 bit (MXFP4, NVFP4) yu am eskaal bu ndaw bu ñuy séddoo ci blok bu ndaw bu nekk, te leegi aparey Blackwell dafay gaaw FP4 ci saasi. Xaarandil rëset yu wuute ci njub, fu ay diisaay yu wuute di jëfandikoo yaatuwaayu bit yu wuute, boole ci tàggat yaram bu gëna am xam-xam ci kantite, suko defee 4-bit nekk default ngir inference. Endgame bi mooy tëye model yu ndaw yi ci chips yu gëna néew, yu yomb te duñu ñàkk benn kalite buñ mëna natt.
Doxal ci àdduna dëgg
Taggat xeetu làkk yu mag ci GPU NVIDIA Hopper/Blackwell di jëfandikoo FP8 ngir yokk lu tollu ci ñaari yoon limu mëna def ci BF16
Liggéeyukaay chatbot inference ci FP8 suko defee benn model mëna méngoo ak GPU yu néew te tontu laajte yu bari ci segond bu nekk
Jëfandikoo E5M2 ngir jokkoo gradient ci diiru tàggat buñ séddale ngir dagg bandwidth reso bi ci diggante node yi
Doxal MXFP4/NVFP4-modèle yu bari ngir méngoo ak modelu scale frontière ci benn GPU bu am mémoire bu bari ngir am inference bu yomb
Risk yi ak balustrade yi
Optimize benn benchmark mën na nëbb ñakk kattan yu gëna yaatu ci sistem bi.
Njëg li ñuy fay ci infrastructure yi ak ci toppatoo dañuy faral di suufeel.
Bu sistem yi di gëna xawa jafee xam, jafe-jafe yi am ci wàllu kaaraange ak seetlu mën nañu gëna bari.
Roadmap ngir samp gi
Mandargal latency, kalite, ak njëg yi laata ngay jëfandikoo.
Benchmark ci biir sargal ak done yu dëggu.
Jumtukaay bi di saytu njuumte yi, derive bi ak njeextalu jëfandikukat bi.
Waajal rollback ak yooni tontu ci jafe-jafe yi laata ngay eskale.
Weyal di banneexu
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the FP8 and Low-Precision Formats quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Gis bi ci topp
Format serialisasioŋ model
Laaj yi ñuy faral di laaj
What is FP8 and Low-Precision Formats?
FP8 formaa nimero floating 8-bit la buy may model IA yi ñu denc ay poid ak def math ci jëfandikoo ñeent ci memory nimero 32-bit yiñ miin. Pexe bu am solo la ngir defar model yu mag yu gëna yomb te gaaw ci tàggat ak serwiis.
Ñaata bit la nimero FP8 di jëfandikoo suñu ko méngale ak nimero FP32 buñ miin?
FP8 dafay jëfandikoo 8 bit waaye FP32 dafay jëfandikoo 32 bit, kon FP8 dafay jël benn ci ñatti xaaju dencukaay bi ak bandwidth bi.
Lan la etiketu 'E4M3' ak 'E5M2' di fësal ci formaa FP8?
E4M3 mooy 4 bit exponent ak 3 bit mantissa; E5M2 mooy 5 exponent ak 2 bit mantissa. Bit exponent yu bari dañuy yaatal rang bi; bits mantissa yu bari yokk nañu ci njub.
Lan moo waral gasoduk FP8 di jëfandikoo fakteer yuy yamale ci tensor yi?
Ak bit exponent yu néew nii, valeur yu mag yi dañuy fees ba noppi yu ndaw yi dañuy wàcci ba zero. Eskalaasioŋ dafay indiwaat tensor yi ci palanteer biñ mëna jëfandikoo bu FP8 balaa math bi.
Ban kompromis mooy gëna ñaawe jëfandikoo FP8?
Bit yu néew mooy représentation bu gëna dëgër ak njuumte yu gëna bari ci rounding. Njariñ li mooy memory bi ak bandwidth bi dafay gëna néew, ak math bu gëna gaaw ci hardware biñ jàppale.
Ban GPU NVIDIA moo njëkka yokk ndimmbalu hardware ngir FP8 matrix math?
GPU yu lalu ci Hopper yu melni H100 dugal nañu ndimbalu FP8 Tensor Core, ginaaw ga Blackwell yokk ko ba FP4.