Ikibazo gihuza ibibazo · Biragoye Urwego

Icyitegererezo na Pipeline Iringaniza Ikibazo

Irasobanura gutandukanya tensor nuyoboro, kwimura ibikorwa, gahunda ya microbatch, imiyoboro myinshi, hamwe nu bucuruzi bwitumanaho.

Inzira zijyanye no kuyoboraIcyitegererezo Numuyoboro
Ikibazo 1 Bya 8

Ni ikihe kibazo cyibanze icyitegererezo nicyerekezo gikemura?

Reba buri kibazo8 ibibazo

Ibisubizo nibisobanuro bigaragara mugihe ufata ikibazo.

  1. Ni ikihe kibazo cyibanze icyitegererezo nicyerekezo gikemura?
    • Icyitegererezo kuba kinini cyane kuburyo kidahuye na memoire imwe ya GPU
    • Buhoro buhoro amakuru yikuramo kuri disiki
    • Gradients iturika mugihe cy'amahugurwa
    • Ingero nke cyane zamahugurwa arahari
  2. Nigute tensor (intra-layer) parallelism itandukanya akazi?
    • Igenera ibice byose kuri GPU zitandukanye
    • Igabanya imibare imbere murwego, nka matrix igwira, hakurya ya GPU
    • Igwiza buri cyiciro kuri buri GPU
    • Ikoresha buri cyiciro kabiri kugirango kirengere
  3. Niki 'bubble' muburyo bubangikanye?
    • Kwibuka kumeneka gukura mugihe
    • Kopi yikopi ya optimizer leta
    • Igihe cyubusa iyo epfo GPUs itegereje ibyiciro byo hejuru kugirango bitange ibikorwa
    • Igicucu mu ndangagaciro
  4. Nigute imiyoboro ibangikanye igabanya ububobere?
    • Ukoresheje igipimo gito cyo kwiga
    • Mugukuraho ibice bimwe kurugero
    • Kubika ibikorwa kuri CPU
    • Mugabanye buri cyiciro mo micro-batches zinyura mubyiciro muburyo butangaje
  5. Ni ukubera iki uburinganire bwa tensor busanzwe bubikwa mumutwe umwe?
    • Kuberako inshuro nyinshi-kugabanya ibisubizo byigice byunguka biva mumirongo yihuse nka NVLink
    • Kuberako bidasaba itumanaho iryo ariryo ryose
    • Kuberako imitwe idashobora gusangira amakuru na gato
    • Kuberako ishyigikira GPU imwe gusa
  6. Ni ubuhe buryo buzwi cyane kubijyanye na tensor parallelism mumahugurwa ya transformateur?
    • scikit-wige
    • NVIDIA Megatron-LM
    • Gufungura
    • Panda
  7. Ni iyihe data igomba kwimuka hagati y'ibice mu gutambutsa pipeline mu gihe cyo gukora forward pass?
    • Dataset yose y'imyitozo nyuma ya buri rwego
    • Umuvuduko w'igishoboka w'umuhindura
    • Tensors z'ibikorwa zakozwe n'igice kimwe zikaba zakirwa n'igice gikurikiraho
    • Nta data, kuko ibice ari models zihariye
  8. Kuki tensor parallelism akenshi ikeneye itumanaho ryihuse mu itsinda ryayo?
    • Iha GPU buri dataset itandukanye kandi ntiyongera guhuza
    • Itumanaho ribaho rimwe gusa iyo model yinjijwe
    • Ntabwo ikoresha imikorere yo guhuza ibikoresho
    • Ibikoresho bihinduranya ibisubizo bice cyangwa activations mu rwego mu gihe cyo kubara