Guide-yakabatana mibvunzo · Pakati Nhanho

Activation Recomputation Tradeoffs Quiz

Muedzo activation yekutarisa ndangariro tradeoffs, yakasarudzika kudzokorodza, uye replay kurongeka, kusanganisira nei imwe komputa isingagadzirise mutengo wendangariro.

Nzira dzekutungamira dzinoenderanaActivation Recomputation Tradeoffs
Mubvunzo 1 ye 8

Chii chinoita activation recomputation kutengeserana kure kuchengetedza ndangariro?

Tarisa mubvunzo wese8 mibvunzo

Mhinduro uye tsananguro zvinoonekwa paunenge uchitora bvunzo.

  1. Chii chinoita activation recomputation kutengeserana kure kuchengetedza ndangariro?
    • Muenzaniso wakarurama
    • Yakawedzera compute (FLOPs)
    • Network bandwidth
    • Kurongeka kwenhamba
  2. Sei ma-forward-pass activations achiwanzo chengetwa zvachose?
    • Kuti uzviratidze kumushandisi
    • Kumanikidza modhi
    • Ivo vanodiwa kuti tokenization
    • Nekuti backpropagation inoda ivo kuti vaverenge ma gradients
  3. Why should activation checkpointing overhead be measured on the actual model?
    • Checkpoint placement, operators, and hardware determine the cost of replay
    • It always adds exactly forty percent to wall-clock time
    • It guarantees a tenfold speed improvement
    • It removes all forward computation
  4. For an idealized N-layer chain with comparable activations, what storage scaling can square-root checkpoint placement achieve?
    • O(N squared) for all model parameters
    • O(1) for all training memory regardless of N
    • O(sqrt(N)) for activation storage
    • O(N squared) for optimizer state
  5. Ndeipi pfungwa iri shure kwekusarudza (kwete kuzara) recomputation?
    • Dzokorora kushanda kwese kaviri
    • Usambodzokorora chero chinhu
    • Dzokorora chete embedding layer
    • Recompute chete ndangariro-inorema asi compute-yakachipa ops, cache inodhura iwo
  6. Ndeipi nzira yekuwedzera inowanzosanganiswa nekudzokorora kuchengetedza ndangariro dzakatowanda?
    • Kuwedzera batch size
    • Kurodha ma activation kuCPU kana NVMe
    • Kubvisa ese normalization layers
    • Kuchinjira kune inference mode
  7. A checkpointed block contains dropout. Why does consistent RNG state matter during replay?
    • Dropout becomes a model-weight quantizer
    • The optimizer no longer needs gradients
    • Backward recomputation should reproduce the intended forward randomness
    • The batch size must become one
  8. Checkpointing reduces stored activations, but optimizer states still exceed GPU memory. Why is the job still out of memory?
    • Activation recomputation does not remove optimizer-state storage
    • Checkpointing increases every optimizer tensor to FP64
    • Backward no longer consumes any memory
    • The model cannot use checkpointing with an optimizer