引導式測驗 · 中等 等級

使用 WebDataset 串流大型訓練資料集測驗

使用 WebDataset 分片進行流程處理,同時考慮可迭代載入、緩衝洗牌和分散式取樣限制。

問題 1 的 8

How does WebDataset package samples for streaming?

預覽每個問題8 問題

當您參加測驗時,就會出現答案和解釋。

  1. How does WebDataset package samples for streaming?
    • A tar archive containing sample files grouped by a shared basename
    • A relational database table with one row per GPU
    • A single image without metadata
    • A model checkpoint bundle only
  2. How are files for one sample commonly associated?
    • They have identical bytes
    • They must be stored in separate databases
    • They share a basename and use different file extensions
    • They all use the .tar extension individually
  3. How can streaming WebDataset shards benefit a training pipeline?
    • It guarantees global random access
    • Training can read sequentially from disk or network without fetching the entire dataset first
    • It removes the need to validate data
    • It ensures every epoch has identical sample order
  4. Which tradeoff can arise with iterable streaming datasets?
    • They cannot be used with PyTorch
    • They require unpacking everything first
    • They only store labels
    • Random access and exact epoch accounting can be less natural
  5. How is shuffling often approximated in a stream?
    • By using a buffer rather than a global permutation of all examples
    • By sorting every sample alphabetically
    • By shuffling model parameters
    • By removing the shard manifest
  6. Why configure worker and node splitting carefully?
    • To guarantee model accuracy
    • To remove data corruption
    • To increase the number of labels
    • To reduce duplicate or missing shard consumption across workers
  7. What should be recorded for reproducible streaming runs?
    • 僅型號名稱
    • Only the GPU brand
    • Shard order, seed, worker count, and resampling policy
    • Nothing if the dataset is in tar files
  8. Is WebDataset a query database with arbitrary row lookup?
    • Yes, every item has a database index
    • No; it is a sharded streaming format and pipeline with different access tradeoffs
    • Yes, it guarantees SQL transactions
    • No, because tar archives cannot contain samples