High Bandwidth Memory Quiz

Answers and explanations appear as you take the quiz.

1. What primary problem does High Bandwidth Memory address for AI accelerators?
   • Compute cores sitting idle while waiting for data to arrive
   • GPUs overheating from too much light
   • Storing files permanently when power is off
   • Connecting the GPU to the internet
2. How is HBM physically built differently from ordinary GDDR memory?
   • It is stored on a hard drive
   • It stacks multiple DRAM dies vertically and connects them with through-silicon vias
   • It uses fiber-optic cables inside the chip
   • It has no transistors
3. What does HBM rely on mainly to achieve its high bandwidth?
   • Much higher clock speeds than any other memory
   • Cooling with liquid nitrogen
   • Extreme parallelism through a very wide interface
   • Compressing all data to one bit
4. Why are HBM stacks placed on a silicon interposer right next to the GPU?
   • To make the package look better
   • Because HBM cannot work more than a meter away
   • To allow easy removal by the user
   • To keep the wires short, reducing power per bit and latency
5. For large language models specifically, why can HBM matter as much as raw compute?
   • Because model weights must be streamed constantly, demanding high capacity and bandwidth
   • Because models do no math at all
   • Because compute is free but memory is taxed
   • Because LLMs only use the CPU
6. Which HBM generations are part of the progression from HBM2 to currently sampling HBM4 products?
   • HBM2, HBM2E, HBM4E, then HBM3
   • HBM2, HBM2E, HBM3, HBM3E, and HBM4
   • GDDR6, GDDR7, DDR5, and HBM4
   • HBM3E directly followed by HBM1
7. Why can high HBM bandwidth help an autoregressive language model during token generation?
   • It eliminates the need to store model weights
   • It performs tokenization instead of the GPU
   • It can move model weights and other data to the compute units more quickly
   • It makes every model compute-bound
8. What is the difference between HBM capacity and HBM bandwidth?
   • They are identical measures of GPU clock speed
   • Capacity measures transfer speed and bandwidth measures data size
   • Neither is related to memory
   • Capacity is how much data can be stored; bandwidth is how quickly data can be transferred