Ikibazo gihuza ibibazo · Hagati Urwego

Gukora Ibarura ry'ubucuruzi Ikibazo

Ikizamini cyo gukora igenzura ryerekana ububiko bwibicuruzwa, guhitamo gutoranya, no gusubiramo ukuri, harimo n'impamvu compte yinyongera idakemura ibiciro byose byo kwibuka.

Inzira zijyanye no kuyoboraIgikorwa cyo Kwishura Ibicuruzwa
Ikibazo 1 Bya 8

Ni ubuhe buryo bwo gusubiramo ibicuruzwa bigurisha kure kugirango ubike kwibuka?

Reba buri kibazo8 ibibazo

Ibisubizo nibisobanuro bigaragara mugihe ufata ikibazo.

  1. Ni ubuhe buryo bwo gusubiramo ibicuruzwa bigurisha kure kugirango ubike kwibuka?
    • Icyitegererezo
    • Kubara ibirenze (FLOPs)
    • Umuyoboro mugari
    • Umubare wuzuye
  2. Ni ukubera iki ibikorwa-byimbere-bikorwa bisanzwe bibikwa muri byose?
    • Kugaragaza kubakoresha
    • Guhagarika icyitegererezo
    • Barasabwa kubimenyetso
    • Kuberako backpropagation ibakeneye kubara gradients
  3. Why should activation checkpointing overhead be measured on the actual model?
    • Checkpoint placement, operators, and hardware determine the cost of replay
    • It always adds exactly forty percent to wall-clock time
    • It guarantees a tenfold speed improvement
    • It removes all forward computation
  4. For an idealized N-layer chain with comparable activations, what storage scaling can square-root checkpoint placement achieve?
    • O(N squared) for all model parameters
    • O(1) for all training memory regardless of N
    • O(sqrt(N)) for activation storage
    • O(N squared) for optimizer state
  5. Ni ikihe gitekerezo kiri inyuma yo gutoranya (kutuzuye)?
    • Ongera ibikorwa byose kabiri
    • Ntuzigere ubara ikintu na kimwe
    • Ongera usubiremo gusa
    • Ongera wibuke gusa-uburemere ariko kubara-bihendutse ops, cache ihenze
  6. Ni ubuhe buryo bwuzuzanya bukunze guhuzwa no kubara kugirango ubike kwibuka cyane?
    • Kongera ingano yicyiciro
    • Kurekura ibikorwa kuri CPU cyangwa NVMe
    • Kuraho ibice byose bisanzwe
    • Guhindura uburyo bwo gufata umwanzuro
  7. A checkpointed block contains dropout. Why does consistent RNG state matter during replay?
    • Dropout becomes a model-weight quantizer
    • The optimizer no longer needs gradients
    • Backward recomputation should reproduce the intended forward randomness
    • The batch size must become one
  8. Checkpointing reduces stored activations, but optimizer states still exceed GPU memory. Why is the job still out of memory?
    • Activation recomputation does not remove optimizer-state storage
    • Checkpointing increases every optimizer tensor to FP64
    • Backward no longer consumes any memory
    • The model cannot use checkpointing with an optimizer