quiz bu lëkkale ak tegtal · Diggante Niveau
GPU bu amul serwër ak quiz tàmbali sedd
Xamme etape yu tàmbali sedd yi ci inference GPU bu amul serwër te tànnee ay wàññikaay boole ci yemale latency, kàttan gu tàng, cache ak njëg.
Laaj 1 de 8
Which request is most directly experiencing a cold start on a serverless GPU endpoint?
Xoolaatal laaj bu nekk8 laaj
Sooy def quiz bi, tontu yi ak leeral yi dañuy feeñ.
- Which request is most directly experiencing a cold start on a serverless GPU endpoint?
- A request routed to a worker that is already initialized
- A request waiting behind other work on a warm but saturated worker
- The first request that must wait for a new worker and runtime to initialize
- A request delayed only by model computation after startup
- Which step specifically transfers model parameters into GPU memory during startup?
- Pulling the container image
- Loading model weights into accelerator memory
- Allocating the GPU worker
- Starting the runtime process
- Why can a warm-path benchmark understate latency for an endpoint that scales to zero?
- A warm benchmark includes container and model startup on every call
- Warm measurements omit startup and initialization paid by the first request after scale-up
- A warm benchmark includes worker allocation after the container is ready
- A warm benchmark reloads the model weights on every request
- Which mitigation may reduce cold starts while increasing idle expense?
- Keeping a minimum amount of GPU capacity ready
- Deleting model artifacts after every request
- Increasing the model image size to force preloading
- Retrying every request without a limit
- Why separate cold-start stage timings?
- Identify whether queueing, worker setup, model loading or initialization dominates delay
- Determine whether generated text has correct semantics
- Measure steady-state GPU throughput only
- Select the model confidence threshold
- How can aggressive retries worsen a cold-start incident?
- They add more in-flight work while workers may still be starting
- They guarantee the platform keeps a GPU warm without billing
- They automatically deduplicate all repeated requests
- They skip model initialization on retry
- Which measurements distinguish cold and warm behavior?
- Only training accuracy
- Latency distributions for each path and cold-start frequency
- Only average GPU memory
- Only container build time
- What does serverless imply about inference cost?
- It can reduce infrastructure management, but usage and warm capacity may still cost money
- It is automatically free when idle in every provider
- GPU capacity has no billing limits
- Retries remove compute charges