引導式測驗 · 硬 等級

多模型服務與模型多工處理測驗

平衡共享模型利用率與路由、按需載入、快取、驅逐、GPU 記憶體限制和租戶隔離。

問題 1 的 8

What does model multiplexing commonly allow a shared serving worker to do?

預覽每個問題8 問題

當您參加測驗時,就會出現答案和解釋。

  1. What does model multiplexing commonly allow a shared serving worker to do?
    • Keep every tenant model permanently loaded before any request
    • Choose a model only after ignoring the request identity
    • Load or select a requested model or adapter when needed
    • Train all tenant models as part of each inference request
  2. Why can a shared model cache increase latency for an infrequently used model?
    • The cache changes the model output labels
    • An evicted model may need to be loaded again on its next request
    • Eviction guarantees a faster reload from device memory
    • An evicted model remains resident but is marked inactive
  3. What should an API do with a client-provided model identifier?
    • Treat the identifier as a filesystem path supplied by the client
    • Use the identifier to bypass authorization checks
    • Assume any artifact named by a client is compatible
    • Resolve the identifier through a validated registry or allowlist
  4. Which resource can limit how many models stay resident on one GPU?
    • GPU memory capacity limits resident weights and runtime state
    • The number of names in the API schema sets device memory
    • The training-set label format determines residency
    • More model identifiers always mean more available GPU memory
  5. Why might a simple least-recently-used eviction policy perform poorly?
    • It always retains the largest model regardless of request history
    • It selects entries using GPU memory as their cache key
    • It ignores model size and the cost or likelihood of reloading
    • It prevents cold starts whenever a model is evicted
  6. Which metrics help diagnose model multiplexing behavior?
    • Only the number of model names configured
    • Cache hits, evictions, load duration, and cold-start rate
    • Only training loss for each model
    • Only the API route count
  7. What should be considered when comparing shared multiplexing with dedicated replicas?
    • Compare only model-file counts across architectures
    • Compare only inference framework names
    • Compare only training accuracy
    • Compare demand patterns, resource use, latency, and isolation needs
  8. Why account for in-flight requests before unloading a model?
    • Active requests may still depend on the model weights or runtime state
    • Removing a model entry automatically redirects active work
    • Unloading updates every client schema before a response
    • In-flight requests no longer use the model once eviction starts