NVLink ak GPU dañuy boole
NVLink ak interconnects yi ko méngoo ñooy lëkkalekaay yu gaaw yiy may GPU yu bari ñu waxtaan seen biir ci saa si te gaaw.
Résumé
They are essential because training and serving the largest AI models requires hundreds or thousands of GPUs to act like one giant accelerator.
Plongeur bu xóot
Benn GPU mënul tëye model yi gëna mag, moo tax dañu leen xaaj puce yu bari yu wara weccoo ay done saa yu nekk, lu ci melni poid, gradient, ak aktivasioŋ. Bus PCIe biñ miin dafa yeex lool, moo tax NVIDIA sos NVLink, muy lëkkalekaay GPU-to-GPU buy joxe bandwidth bu gëna rëy ak latency bu gëna néew. Chips NVSwitch dañuy yokk lii ci benn tissu suko defee GPU bu nekk ci serwër bi mëna yegg ci beneen bi ci gaawaay bu mat sëkk, mu soppi juróom ñatti GPU ci benn memory bu mag ak pool ordinatër. Ci wàllu rack, sistem yu melni NVL72 bu NVIDIA dañuy boole fukki-fukki GPU ci benn domen NVLink buñ boole. Lu weesu benn rack, xaralay reso yu melni InfiniBand ak Ethernet (dañuy faral di ànd ak RDMA) dañuy boole ay junni node ci benn cluster. Kalite interconnect yooyu dafay tënk ni model yi di yaatoo ak ni ñu gaawee tàggatoo.
Gis-gis xarala
NVLink dafay joxe yoon yuñ jagleel point-à-point ci diggante GPU yi ak bandwidth bu ëpp PCIe bi ak latency bu gëna ndaw, loolu mooy tax GPU yi di jàng seen memory melni dañu ci dëkk bi. NVSwitch dafay melni crossbar bu gaaw, suko defee GPU yépp ci benn node di jokkoo te duñu bloke ci bandwidth bu mat. Liggéeyukaay yu mbooloo yu melni all-reduce, yuy boole gradient yi ci GPU yi ci diiru tàggat, dañu gëna gaaw ci kaw tissu bii, moo tax bandwidth interconnect dafay am njeexital yu am solo ci ni tàggat yaram di doxee ci puce yu bari.
njeextalu pexe
Njëgg ak budget
Dogal yi architecture di jël dañuy indi njariñ ak njëgu liggéey bi ay at ci ginaaw.
dogal yu gëna leer
Njàngalem xarala yi dafay jàppale ekip yi ñu tànn li gën, te baña yam ci li gëna bees daal.
Xool kalite
Tanneef yu gëna baax ci wàllu ingeñër dina wàññi jafe-jafe yi ci wàllu wóor ci liggéey bi.
Ëlëgu lëkkaloog NVLink ak GPU
Ginaaw model yi dañuy gëna bari ay serwër yu benn, lëkkaloo gi dafay nekk sistem bi. NVLink mingi wéy di am bandwidth ci jamono bu nekk, ba noppi domen NVLink yu mag yi (lu melni NVL72) dañuy gëna yokk limu GPU yiy doxalee ni benn. Xaarandi domen yu gëna yaatu yuñ boole, lëkkaloo bu gëna seere ci ordinatër ak reso, lëkkalekaay optik ngir wàññi doole ci distance, ak jeegoowu liggéeyukaay yi ngir ubbi lëkkaloo (lu melni UALink) ngir xëcc tissu yiñ moom. Scaling IA dafay gëna aju ci toxal ay done ci diggante puce yi ni ci puce yi ci seen bopp.
Doxal ci àdduna dëgg
Keppal juróom ñatti GPUs ci biir benn serwër (lu melni sistem NVIDIA DGX) jaaraleko ci NVSwitch ngir ñu bokk mémoire ak tàggat benn model bu mag ñoom ñaar.
Def lépp luy wàññi senkronizaasioŋ gradient ci GPU yi ci diiru tàggat buñ séddale, gaaw ci yaatuwaayu band NVLink.
Lëkkaloo ay fukki-fukki GPU ci sistem NVL72 bu yam ci rack ci benn domen NVLink buñ boole muy model yu am ay trillioŋu paramet.
Teg ay junni serveur GPU ci benn cluster di jëfandikoo InfiniBand wala RDMA-ci kaw-Ethernet ngir tàggat modelu fondasioŋ bu yaatu.
Risk yi ak balustrade yi
Optimize benn benchmark mën na nëbb ñakk kattan yu gëna yaatu ci sistem bi.
Njëg li ñuy fay ci infrastructure yi ak ci toppatoo dañuy faral di suufeel.
Bu sistem yi di gëna xawa jafee xam, jafe-jafe yi am ci wàllu kaaraange ak seetlu mën nañu gëna bari.
Roadmap ngir samp gi
Mandargal latency, kalite, ak njëg yi laata ngay jëfandikoo.
Benchmark ci biir sargal ak done yu dëggu.
Jumtukaay bi di saytu njuumte yi, derive bi ak njeextalu jëfandikukat bi.
Waajal rollback ak yooni tontu ci jafe-jafe yi laata ngay eskale.
Weyal di banneexu
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the NVLink and GPU Interconnects quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Gis bi ci topp
GPU ak TPU ngir IA
Laaj yi ñuy faral di laaj
What is NVLink and GPU Interconnects?
NVLink ak interconnects yi ko méngoo ñooy lëkkalekaay yu gaaw yiy may GPU yu bari ñu waxtaan seen biir ci saa si te gaaw. Dañu am solo ndax tàggat ak liggéey ci xeetu IA yi gëna mag dafay laaj téemeeri wala junni GPU ngir doxal ni benn gaawaay bu mag.
Lan moo tax ñu soxla lëkkaloo yu gaaw yu melni NVLink ci model IA yu mag yi?
Modèle yu mag yi dañu ëpp benn GPU, moo tax dañu tasaaroo ci chips yu bari yuy wéy di séddoo poid, gradient, ak aktivasioŋ, te loolu dafay laaj lëkkalekaay yu gaaw.
Ban njariñ la NVLink joxe ci bus PCIe biñ miin ngir jokkoo GPU-to-GPU?
NVLink lëkkalekaay la bu jub diggante GPU ak GPU, am bandwidth bu gëna mag ak latency bu gëna ndaw PCIe, loolu mooy tax ñu mëna weccoo done ci digganté chips yi.
Lan mooy wareefu NVSwitch ci serwëru GPU yu bari?
NVSwitch dafay yokk NVLink ci tissu bu dul bloke, may GPU yépp ci benn node ñu jokkoo seen biir ci gaawaay bu mat sëkk.
Ban liggéey bu mbooloo, bu bari ci tàggat-yaram buñ séddale, mooy gëna am njariñ ci lëkkaloo yu gaaw yi?
Wàññi lépp ak séddoo gradient ci GPUs yépp jéego bu nekk ci tàggat yaram, kon gaawaayam mingi aju bu bari ci bandwidth biy boole.
Lu weesu benn serwër, ban xarala mooy boole ay junni node GPU ci benn cluster?
Ci wàllu cluster, reso yu melni InfiniBand wala Ethernet ak RDMA dañuy boole node yu bari, di mottali NVLink ci biir serwër bu nekk.