引導式測驗 · 中等 等級
使用 WebDataset 串流大型訓練資料集測驗
使用 WebDataset 分片進行流程處理,同時考慮可迭代載入、緩衝洗牌和分散式取樣限制。
問題 1 的 8
How does WebDataset package samples for streaming?
預覽每個問題8 問題
當您參加測驗時,就會出現答案和解釋。
- How does WebDataset package samples for streaming?
- A tar archive containing sample files grouped by a shared basename
- A relational database table with one row per GPU
- A single image without metadata
- A model checkpoint bundle only
- How are files for one sample commonly associated?
- They have identical bytes
- They must be stored in separate databases
- They share a basename and use different file extensions
- They all use the .tar extension individually
- How can streaming WebDataset shards benefit a training pipeline?
- It guarantees global random access
- Training can read sequentially from disk or network without fetching the entire dataset first
- It removes the need to validate data
- It ensures every epoch has identical sample order
- Which tradeoff can arise with iterable streaming datasets?
- They cannot be used with PyTorch
- They require unpacking everything first
- They only store labels
- Random access and exact epoch accounting can be less natural
- How is shuffling often approximated in a stream?
- By using a buffer rather than a global permutation of all examples
- By sorting every sample alphabetically
- By shuffling model parameters
- By removing the shard manifest
- Why configure worker and node splitting carefully?
- To guarantee model accuracy
- To remove data corruption
- To increase the number of labels
- To reduce duplicate or missing shard consumption across workers
- What should be recorded for reproducible streaming runs?
- 僅型號名稱
- Only the GPU brand
- Shard order, seed, worker count, and resampling policy
- Nothing if the dataset is in tar files
- Is WebDataset a query database with arbitrary row lookup?
- Yes, every item has a database index
- No; it is a sharded streaming format and pipeline with different access tradeoffs
- Yes, it guarantees SQL transactions
- No, because tar archives cannot contain samples