Guide-linked quiz · Hard Level

Estimating LLM Training FLOPs Quiz

Estimate dense LLM training operations with 6ND, convert throughput to runtime, distinguish utilization metrics, and check planning assumptions.

Question 1 of 8

In the dense-model approximation C ≈ 6ND, which quantities do N and D represent?

Preview every question8 questions

Answers and explanations appear as you take the quiz.

  1. In the dense-model approximation C ≈ 6ND, which quantities do N and D represent?
    • Parameter count and total processed training tokens
    • Number of devices and unique documents
    • Network layers and batch size
    • Numerical precision and model file size
  2. A ten-billion-token corpus is processed twice. Which D belongs in the 6ND estimate?
    • Ten billion, because only unique tokens count
    • Twenty billion, because both passes perform training work
    • Five billion, because repeated data halves the cost
    • Two, because D means the number of passes
  3. Which distinction between FLOPs and FLOP/s is needed when estimating a training schedule?
    • Both are memory capacities
    • FLOPs is speed and FLOP/s is total work
    • FLOPs counts operations; FLOP/s is an operation rate
    • Both directly specify the monetary cost
  4. For one billion parameters and twenty billion processed tokens, what does 6ND estimate?
    • 1.2 × 10¹⁰ FLOPs
    • 6 × 10²⁰ FLOPs
    • 2 × 10¹⁹ FLOPs
    • 1.2 × 10²⁰ FLOPs
  5. Eight accelerators each have a relevant peak of 100 TFLOP/s and assumed model FLOPs utilization of 50%. What effective model throughput is used?
    • 400 TFLOP/s
    • 800 TFLOP/s
    • 4 TFLOP/s
    • 40,000 TFLOP/s
  6. The worked run takes about 83.3 elapsed hours on eight accelerators. Approximately how many accelerator-hours does it consume?
    • About 10.4
    • About 667
    • About 83.3
    • About 300,000
  7. Why can hardware FLOPs utilization exceed model FLOPs utilization for a run that recomputes activations?
    • Recomputation creates new unique training tokens
    • Model utilization counts storage instead of operations
    • Hardware accounting can include recomputation that is excluded from useful model-work accounting
    • The two measures always use different device counts
  8. Before treating a 6ND runtime estimate as a practical schedule, which check does the guide recommend?
    • Assume the model fits if peak FLOP/s is high
    • Reuse any published utilization percentage unchanged
    • Count only model weights and ignore training buffers
    • Measure a representative pilot and account for memory and activities outside its timing scope

Keep testing yourself

More quizzes picked for your level.

View all quizzes