그라데이션 체크포인트 퀴즈

퀴즈를 풀면 정답과 설명이 나타납니다.

1. 메모리를 절약하기 위해 그라디언트 체크포인트는 주로 무엇을 거래합니까?
   • 더 적은 메모리 사용을 위한 추가 계산(재계산)
   • 속도에 대한 모델 정확도
   • 네트워크 대역폭을 위한 디스크 공간
   • 배치 크기의 정밀도
2. 정방향 전달 중에 활성화가 일반적으로 저장되는 이유는 무엇입니까?
   • 사용자에게 표시하려면
   • 역전파에서는 그래디언트를 계산하는 데 필요하기 때문입니다.
   • 데이터 로딩 속도를 높이려면
   • 실제로는 필요하지 않습니다.
3. In a simplified chain of N similarly sized layers, what can checkpoint boundaries spaced about sqrt(N) layers apart achieve in the textbook schedule?
   • It stores no activations and performs no recomputation
   • It guarantees constant memory for every model graph
   • Activation storage can scale on the order of sqrt(N) checkpoints
   • It makes optimizer-state memory scale as sqrt(N)
4. What most determines gradient-checkpointing runtime overhead in a practical model?
   • Only the model’s file extension
   • A fixed 20–30% slowdown for every model
   • The number of labels in the dataset
   • Which operations are recomputed, checkpoint placement, graph structure, and hardware
5. PyTorch에서 모듈에 그래디언트 체크포인트를 적용하는 데 일반적으로 사용되는 유틸리티는 무엇입니까?
   • 토치.utils.체크포인트
   • 토치.최적.SGD
   • torch.nn.드롭아웃
   • torch.cuda.empty_cache
6. What happens to operations inside a checkpointed region during backward?
   • Their gradients are skipped entirely
   • Their forward computations are rerun to regenerate needed activations
   • Their weights are permanently discarded
   • They are executed only during inference
7. Which training techniques can be combined with activation checkpointing to reduce different memory costs?
   • Only increasing the number of labels
   • Turning off backpropagation
   • FSDP for sharded model state and mixed precision for lower-precision tensors
   • Replacing the model with a tokenizer
8. What is selective activation checkpointing intended to choose?
   • Which training labels are removed
   • Which model outputs are published
   • Which GPU becomes the tokenizer
   • Which operations to save and which to recompute based on their memory and runtime costs