Kedis ku xidhan hagaha · Dhexdhexaad Heerka

Isku-dheellitirka LLM iyo Kediska Isku-dheellitirka Xamuulka

Waxay hubisaa nuqulka iyo qaab-socodka moodeelka, xidhiidhka cache, telemetry nool, qiimaha calaamadda beddelka ah iyo daahitaanka-ka war-qabka ee macnaha LLM.

Wadooyinka haga ee la xidhiidhaLLM Inference Routing iyo Isku-dheelitirka Culeyska
Su'aal 1 ee 8

Waa maxay sababta wareega-wareega cad inta badan u yahay istaraatijiyad-dheellitirka culeyska liidata ee u-jeeddada LLM?

Horudhac su'aal kasta8 su'aalo

Jawaabaha iyo sharraxaadaha waxay soo baxayaan markaad qaadanayso kediska.

  1. Waa maxay sababta wareega-wareega cad inta badan u yahay istaraatijiyad-dheellitirka culeyska liidata ee u-jeeddada LLM?
    • Codsiyada aad ayay u kala duwan yihiin qiimahooda waxayna nuqul ka mid ahi hayaan khasnado dhegdheg leh oo KV ah
    • Waa sharci darro dalalka badidooda
    • Round-robin waxa uu u shaqeeyaa kaydinta xogta
    • Waxay u baahan tahay GPU si uu u shaqeeyo
  2. Waa maxay 'horgale-cache affinity' isku deyaya in la gaaro?
    • U dirida codsi kasta GPU-ga aan tooska ahayn
    • Ku-wareejinta codsiga nuqulka hore u kaydiyay horgalaha degdega ah
    • Sirta degdega ah ka hor inta aanad dirin
    • Cadaadiska miisaanka modelka
  3. Qaabka dhibka ku-saleysan dariiqinta, maxaa caadi ahaan ku dhaca weydiinta fudud?
    • Waa la diiday
    • Waxaa loo diraa qaabka ugu weyn ee la heli karo
    • Waxa loo diraa nooc yar oo ka jaban
    • Waxa ay saf ugu jirtaa 24 saac
  4. Calaamadee noole ah ayaa ugu faa'iido badan miisaanka-dhaleyaasha culeyska-ka warqaba ee LLM?
    • Dhererka magaca serverka
    • Midabka rafcaanka
    • Aagga wakhtiga isticmaalaha kaliya
    • Qoto dheer ee safka iyo joogitaanka kaydka KV ee koobi kasta
  5. Muxuu qalab sida LiteLLM ku bixiyaa habaynta adeeg bixiyayaasha badan?
    • Wakiil keli ah oo dheelli-tiraya kuna guuldarraysta dhammaan bixiyeyaasha moodeelka badan
    • Naqshad cusub oo transformer ah
    • Nidaamka qaboojinta GPU
    • Beddelka calaamadeeyaha
  6. Maxay tahay sababta mid ka mid ah codsiga wax soo saarka dheer uu uga miisaan badnaan karo kuwa gaagaaban ee dheellitirka?
    • Wax soo saarka dheer ma isticmaalo kheyraad
    • Calaamad kasta oo wax soo saar ahi waxay ku kacaysaa baas hore oo dhammaystiran, markaa wadarta guud ee xamuulka calaamaduhu waa ka badan yahay tirinta codsiga
    • Codsiyada gaaban had iyo jeer waa ka gaabiyaan
    • Router-ku gabi ahaanba ma arki karo dhererka wax soo saarka
  7. A serving system reports goodput rather than raw request count. What does goodput usually emphasize?
    • Tirada guud ee cabbiraadaha moodeelka
    • Only the number of HTTP connections opened
    • Requests served while meeting the specified latency or service-level targets
    • The fraction of requests that use the largest model
  8. A gateway first chooses a small or large model, then chooses a replica of that model. What are these two decisions?
    • Tokenization followed by decoding
    • Training followed by fine-tuning
    • Prefill followed by prompt construction
    • Model routing followed by replica load balancing