Kuiz berkaitan panduan · Keras Tahap

Servis Berbilang Model dan Kuiz Multiplexing Model

Seimbangkan penggunaan model kongsi dengan penghalaan, pemuatan atas permintaan, caching, pengusiran, had memori GPU dan pengasingan penyewa.

Laluan panduan yang berkaitanServis Berbilang Model Dan Multiplexing
soalan 1 daripada 8

What does model multiplexing commonly allow a shared serving worker to do?

Pratonton setiap soalan8 soalan

Jawapan dan penjelasan muncul semasa anda mengambil kuiz.

  1. What does model multiplexing commonly allow a shared serving worker to do?
    • Keep every tenant model permanently loaded before any request
    • Choose a model only after ignoring the request identity
    • Load or select a requested model or adapter when needed
    • Train all tenant models as part of each inference request
  2. Why can a shared model cache increase latency for an infrequently used model?
    • The cache changes the model output labels
    • An evicted model may need to be loaded again on its next request
    • Eviction guarantees a faster reload from device memory
    • An evicted model remains resident but is marked inactive
  3. What should an API do with a client-provided model identifier?
    • Treat the identifier as a filesystem path supplied by the client
    • Use the identifier to bypass authorization checks
    • Assume any artifact named by a client is compatible
    • Resolve the identifier through a validated registry or allowlist
  4. Which resource can limit how many models stay resident on one GPU?
    • GPU memory capacity limits resident weights and runtime state
    • The number of names in the API schema sets device memory
    • The training-set label format determines residency
    • More model identifiers always mean more available GPU memory
  5. Why might a simple least-recently-used eviction policy perform poorly?
    • It always retains the largest model regardless of request history
    • It selects entries using GPU memory as their cache key
    • It ignores model size and the cost or likelihood of reloading
    • It prevents cold starts whenever a model is evicted
  6. Which metrics help diagnose model multiplexing behavior?
    • Only the number of model names configured
    • Cache hits, evictions, load duration, and cold-start rate
    • Only training loss for each model
    • Only the API route count
  7. What should be considered when comparing shared multiplexing with dedicated replicas?
    • Compare only model-file counts across architectures
    • Compare only inference framework names
    • Compare only training accuracy
    • Compare demand patterns, resource use, latency, and isolation needs
  8. Why account for in-flight requests before unloading a model?
    • Active requests may still depend on the model weights or runtime state
    • Removing a model entry automatically redirects active work
    • Unloading updates every client schema before a response
    • In-flight requests no longer use the model once eviction starts