Útmutatóhoz kapcsolódó kvíz · Kemény Szint

Estimating LLM Training FLOPs Quiz

Estimate dense LLM training operations with 6ND, convert throughput to runtime, distinguish utilization metrics, and check planning assumptions.

Kapcsolódó útmutató utakEstimating Training Flops 6nd
kérdés 1 -ból 8

In the dense-model approximation C ≈ 6ND, which quantities do N and D represent?

Tekintse meg az összes kérdést8 kérdéseket

A válaszok és magyarázatok megjelennek a kvíz kitöltésekor.

  1. In the dense-model approximation C ≈ 6ND, which quantities do N and D represent?
    • Parameter count and total processed training tokens
    • Number of devices and unique documents
    • Network layers and batch size
    • Numerical precision and model file size
  2. A ten-billion-token corpus is processed twice. Which D belongs in the 6ND estimate?
    • Ten billion, because only unique tokens count
    • Twenty billion, because both passes perform training work
    • Five billion, because repeated data halves the cost
    • Two, because D means the number of passes
  3. Which distinction between FLOPs and FLOP/s is needed when estimating a training schedule?
    • Both are memory capacities
    • FLOPs is speed and FLOP/s is total work
    • FLOPs counts operations; FLOP/s is an operation rate
    • Both directly specify the monetary cost
  4. For one billion parameters and twenty billion processed tokens, what does 6ND estimate?
    • 1.2 × 10¹⁰ FLOPs
    • 6 × 10²⁰ FLOPs
    • 2 × 10¹⁹ FLOPs
    • 1.2 × 10²⁰ FLOPs
  5. Eight accelerators each have a relevant peak of 100 TFLOP/s and assumed model FLOPs utilization of 50%. What effective model throughput is used?
    • 400 TFLOP/s
    • 800 TFLOP/s
    • 4 TFLOP/s
    • 40,000 TFLOP/s
  6. The worked run takes about 83.3 elapsed hours on eight accelerators. Approximately how many accelerator-hours does it consume?
    • About 10.4
    • About 667
    • About 83.3
    • About 300,000
  7. Why can hardware FLOPs utilization exceed model FLOPs utilization for a run that recomputes activations?
    • Recomputation creates new unique training tokens
    • Model utilization counts storage instead of operations
    • Hardware accounting can include recomputation that is excluded from useful model-work accounting
    • The two measures always use different device counts
  8. Before treating a 6ND runtime estimate as a practical schedule, which check does the guide recommend?
    • Assume the model fits if peak FLOP/s is high
    • Reuse any published utilization percentage unchanged
    • Count only model weights and ignore training buffers
    • Measure a representative pilot and account for memory and activities outside its timing scope