Ikibazo gihuza ibibazo · Biragoye Urwego

Ikibazo gikomeje

Koresha itera-urwego LLM itunganya, imipaka yubushobozi, kandi wuzuze ibicuruzwa, harimo n'impamvu kwemerera ibyifuzo bidasubirwaho gutegereza zeru.

Inzira zijyanye no kuyoboraGukomeza
Ikibazo 1 Bya 8

Nubuhe ntege nyamukuru zo gutondeka static (zihamye) zo gukorera LLM?

Reba buri kibazo8 ibibazo

Ibisubizo nibisobanuro bigaragara mugihe ufata ikibazo.

  1. Nubuhe ntege nyamukuru zo gutondeka static (zihamye) zo gukorera LLM?
    • Irashobora gukora kuri CPU gusa
    • Birasaba kuranga amakuru yose mbere
    • Ibyifuzo bigufi birangira hakiri kare ariko ibibanza byabo bidafite akamaro mugihe icyiciro gitegereje igihe kirekire
    • Bituma icyitegererezo kidasobanuka neza
  2. Ni ubuhe bwoko bwa granularitike guhora wongeyeho kandi ukuraho ibyifuzo?
    • Rimwe kumunsi
    • Kuri intambwe ya decoding (ikimenyetso kimwe icyarimwe)
    • Kubisabwa byuzuye
    • Igihe cyamahugurwa
  3. A continuous-batching request finishes and frees capacity. What can the scheduler do next?
    • Restart every remaining request from its first token
    • Guarantee every queued request runs immediately
    • Discard the finished answer without returning it
    • Admit eligible queued work if its resource budgets allow
  4. Ni ubuhe buryo bukoreshwa muburyo busanzwe hamwe no gukomeza guterana kugirango ushiremo / ukureho ibihe bihendutse?
    • Urupapuro rwibitekerezo rushingiye kuri KV cache
    • FP32 ibisobanuro bibiri
    • Guhinduranya disiki
    • Intoki zamakuru
  5. Ni izihe mpapuro z'ubushakashatsi zisanzwe zishimirwa no gukwirakwiza itera-urwego (rukomeza)?
    • Impapuro zumwimerere
    • Urupapuro rwa Orca
    • Urupapuro rwububiko rwa 1980
    • Urupapuro
  6. What utilization problem can continuous batching reduce compared with a fixed request batch?
    • Every model accuracy error
    • The need to store any KV cache
    • All queueing at any traffic rate
    • Waiting for the longest request before admitting new eligible work
  7. A continuous-batching server has reached its memory budget. Must a newly arrived request enter the next iteration?
    • No; admission can wait until suitable capacity is available
    • Yes; the name guarantees unlimited request admission
    • Yes; the server removes all active outputs automatically
    • Only if the request has no punctuation
  8. A long prompt’s prefill delays tokens for already decoding requests. Which serving objective exposes that tradeoff?
    • Measure only the weight-file name
    • Assume more admitted prompts always improve every latency
    • Measure time between decode tokens alongside throughput
    • Count only the number of model layers