quiz bu lëkkale ak tegtal · Dëgër Niveau
Modèle yu bari yuy liggéey ak model yu bari
Balance jëfandikoo model buñ bokk ak yoon, sargal ci laaj, cache, dàq, àppu mémoire GPU ak tëj luwekat bi.
Laaj 1 de 8
What does model multiplexing commonly allow a shared serving worker to do?
Xoolaatal laaj bu nekk8 laaj
Sooy def quiz bi, tontu yi ak leeral yi dañuy feeñ.
- What does model multiplexing commonly allow a shared serving worker to do?
- Keep every tenant model permanently loaded before any request
- Choose a model only after ignoring the request identity
- Load or select a requested model or adapter when needed
- Train all tenant models as part of each inference request
- Why can a shared model cache increase latency for an infrequently used model?
- The cache changes the model output labels
- An evicted model may need to be loaded again on its next request
- Eviction guarantees a faster reload from device memory
- An evicted model remains resident but is marked inactive
- What should an API do with a client-provided model identifier?
- Treat the identifier as a filesystem path supplied by the client
- Use the identifier to bypass authorization checks
- Assume any artifact named by a client is compatible
- Resolve the identifier through a validated registry or allowlist
- Which resource can limit how many models stay resident on one GPU?
- GPU memory capacity limits resident weights and runtime state
- The number of names in the API schema sets device memory
- The training-set label format determines residency
- More model identifiers always mean more available GPU memory
- Why might a simple least-recently-used eviction policy perform poorly?
- It always retains the largest model regardless of request history
- It selects entries using GPU memory as their cache key
- It ignores model size and the cost or likelihood of reloading
- It prevents cold starts whenever a model is evicted
- Which metrics help diagnose model multiplexing behavior?
- Only the number of model names configured
- Cache hits, evictions, load duration, and cold-start rate
- Only training loss for each model
- Only the API route count
- What should be considered when comparing shared multiplexing with dedicated replicas?
- Compare only model-file counts across architectures
- Compare only inference framework names
- Compare only training accuracy
- Compare demand patterns, resource use, latency, and isolation needs
- Why account for in-flight requests before unloading a model?
- Active requests may still depend on the model weights or runtime state
- Removing a model entry automatically redirects active work
- Unloading updates every client schema before a response
- In-flight requests no longer use the model once eviction starts