Guide-linked quiz · Medium Level

Serverless GPU Inference and Cold Starts Quiz

Identify cold-start stages in serverless GPU inference and choose mitigations while balancing latency, warm capacity, caching and cost.

Question 1 of 8

Which request is most directly experiencing a cold start on a serverless GPU endpoint?

Preview every question8 questions

Answers and explanations appear as you take the quiz.

  1. Which request is most directly experiencing a cold start on a serverless GPU endpoint?
    • A request routed to a worker that is already initialized
    • A request waiting behind other work on a warm but saturated worker
    • The first request that must wait for a new worker and runtime to initialize
    • A request delayed only by model computation after startup
  2. Which step specifically transfers model parameters into GPU memory during startup?
    • Pulling the container image
    • Loading model weights into accelerator memory
    • Allocating the GPU worker
    • Starting the runtime process
  3. Why can a warm-path benchmark understate latency for an endpoint that scales to zero?
    • A warm benchmark includes container and model startup on every call
    • Warm measurements omit startup and initialization paid by the first request after scale-up
    • A warm benchmark includes worker allocation after the container is ready
    • A warm benchmark reloads the model weights on every request
  4. Which mitigation may reduce cold starts while increasing idle expense?
    • Keeping a minimum amount of GPU capacity ready
    • Deleting model artifacts after every request
    • Increasing the model image size to force preloading
    • Retrying every request without a limit
  5. Why separate cold-start stage timings?
    • Identify whether queueing, worker setup, model loading or initialization dominates delay
    • Determine whether generated text has correct semantics
    • Measure steady-state GPU throughput only
    • Select the model confidence threshold
  6. How can aggressive retries worsen a cold-start incident?
    • They add more in-flight work while workers may still be starting
    • They guarantee the platform keeps a GPU warm without billing
    • They automatically deduplicate all repeated requests
    • They skip model initialization on retry
  7. Which measurements distinguish cold and warm behavior?
    • Only training accuracy
    • Latency distributions for each path and cold-start frequency
    • Only average GPU memory
    • Only container build time
  8. What does serverless imply about inference cost?
    • It can reduce infrastructure management, but usage and warm capacity may still cost money
    • It is automatically free when idle in every provider
    • GPU capacity has no billing limits
    • Retries remove compute charges

Keep testing yourself

More quizzes picked for your level.

View all quizzes