quiz bu lëkkale ak tegtal · Diggante Niveau

Quiz ngir saytu gradient

Dafay leeral rematerialisasioŋ biy tàmbali, sakkanal mémoire, njëgu ordinatër bu aju ci kalendriye, ak boole ci sharding ak jaxasoo ci njub.

Yooni njiit yu jëm ciSaytu degrade
Laaj 1 de 8

Lan mooy njëkk jënd ak jaay gradient checkpointing ngir sakkanal mémoire?

Xoolaatal laaj bu nekk8 laaj

Sooy def quiz bi, tontu yi ak leeral yi dañuy feeñ.

  1. Lan mooy njëkk jënd ak jaay gradient checkpointing ngir sakkanal mémoire?
    • Xayma bu gëna bari (jëfandikoowaat) ngir wàññi jëfandikoo mémoire
    • modelu njub ngir gaawaay
    • Espace disk ngir yaatuwaayu band
    • Njàngale ci dayo lots
  2. Lan moo waral ñuy denc aktivaasioŋ yi ci diiru jàll bi?
    • Ngir wane leen jëfandikukat bi
    • Ndax tasaare ci ginaaw daf leen soxla ñu xayma degrade yi
    • Ngir gaawlu yobbu done yi
    • Duñu leen soxla
  3. Ci benn chaîne bu yomb bu am N couche yu tollu noonu, lan la checkpoint yi ñu tolloo ci sqrt(N) couche yu wuute mëna def ci kalendriye téere bi?
    • It stores no activations and performs no recomputation
    • It guarantees constant memory for every model graph
    • Activation storage can scale on the order of sqrt(N) checkpoints
    • It makes optimizer-state memory scale as sqrt(N)
  4. Lan mooy gëna fësal poñ yi ñuy saytu ci gradient bi ci modelu pratik bi?
    • Only the model’s file extension
    • A fixed 20–30% slowdown for every model
    • The number of labels in the dataset
    • Which operations are recomputed, checkpoint placement, graph structure, and hardware
  5. Ci PyTorch, ban njariñ lañuy faral di jëfandikoo ngir saytu gradient ci benn modle?
    • torch.utils.point de contrôle
    • torch.optim.SGD
    • torch.nn.Bàyyi
    • torch.cuda.cache_bu amul dara
  6. What happens to operations inside a checkpointed region during backward?
    • Their gradients are skipped entirely
    • Their forward computations are rerun to regenerate needed activations
    • Their weights are permanently discarded
    • They are executed only during inference
  7. Which training techniques can be combined with activation checkpointing to reduce different memory costs?
    • Only increasing the number of labels
    • Turning off backpropagation
    • FSDP for sharded model state and mixed precision for lower-precision tensors
    • Replacing the model with a tokenizer
  8. What is selective activation checkpointing intended to choose?
    • Which training labels are removed
    • Which model outputs are published
    • Which GPU becomes the tokenizer
    • Which operations to save and which to recompute based on their memory and runtime costs