গাইড লিঙ্ক ক্যুইজ · মাঝারি স্তর

স্কেল কুইজে ফল্ট টলারেন্স এবং GPU ব্যর্থতা

টেকসই চেকপয়েন্ট, স্বাস্থ্য সংকেত, পুনঃসূচনা নীতি এবং যাচাইকৃত পুনরুদ্ধার আচরণ সহ স্থিতিস্থাপক বিতরণকৃত প্রশিক্ষণ ডিজাইন করুন।

প্রশ্ন 1 এর 8

Why can a single worker failure interrupt a distributed training job?

প্রতিটি প্রশ্নের পূর্বরূপ দেখুন8 প্রশ্ন

আপনি কুইজ নেওয়ার সাথে সাথে উত্তর এবং ব্যাখ্যাগুলি উপস্থিত হয়।

  1. Why can a single worker failure interrupt a distributed training job?
    • Every rank can continue an update after any peer disappears
    • A process exit never affects other workers
    • Workers coordinate computation and synchronization across devices
    • Distributed updates require no communication between participants
  2. Which state may be needed to resume training faithfully?
    • Model weights without optimizer or scheduler state
    • Model, optimizer, scheduler, progress, and relevant random state
    • Epoch count without data progress
    • Train-test split metadata without model state
  3. Where should recovery checkpoints be stored for node-loss recovery?
    • Ephemeral node disk
    • GPU memory
    • A host page cache
    • Durable storage that survives the failed node
  4. Why should restart policies distinguish infrastructure faults from deterministic errors?
    • Blind retries can repeat a code or data failure indefinitely
    • A transient timeout can never recover
    • Every process crash corrupts all training data
    • Restart repairs malformed input records
  5. What can change when elastic training adjusts the number of workers?
    • The number of data-loader workers only
    • The GPU model changes when workers are removed
    • Effective batch, data sharding, and training schedule
    • The parameter count changes with rank assignment
  6. Why test recovery by terminating a worker in staging?
    • A staging run proves that hardware failures cannot recur
    • A controlled worker termination verifies checkpoint consistency and restart behavior
    • A restart test removes the need to keep durable checkpoints
    • A recovery test confirms convergence without restoring state
  7. What can GPU health monitoring reveal?
    • Class-specific prediction errors
    • Data split leakage
    • Optimizer convergence speed
    • Thermal, memory, power, driver, or interconnect issues
  8. What should be checked after a job resumes from a checkpoint?
    • Training progress and evaluation behavior remain consistent
    • A single rank restarted while others retained stale state
    • The checkpoint timestamp changed after loading
    • The worker count remains identical after a membership change