Ikibazo gihuza ibibazo · Biragoye Urwego
Ikibazo gikomeje
Koresha itera-urwego LLM itunganya, imipaka yubushobozi, kandi wuzuze ibicuruzwa, harimo n'impamvu kwemerera ibyifuzo bidasubirwaho gutegereza zeru.
Ikibazo 1 Bya 8
Nubuhe ntege nyamukuru zo gutondeka static (zihamye) zo gukorera LLM?
Reba buri kibazo8 ibibazo
Ibisubizo nibisobanuro bigaragara mugihe ufata ikibazo.
- Nubuhe ntege nyamukuru zo gutondeka static (zihamye) zo gukorera LLM?
- Irashobora gukora kuri CPU gusa
- Birasaba kuranga amakuru yose mbere
- Ibyifuzo bigufi birangira hakiri kare ariko ibibanza byabo bidafite akamaro mugihe icyiciro gitegereje igihe kirekire
- Bituma icyitegererezo kidasobanuka neza
- Ni ubuhe bwoko bwa granularitike guhora wongeyeho kandi ukuraho ibyifuzo?
- Rimwe kumunsi
- Kuri intambwe ya decoding (ikimenyetso kimwe icyarimwe)
- Kubisabwa byuzuye
- Igihe cyamahugurwa
- A continuous-batching request finishes and frees capacity. What can the scheduler do next?
- Restart every remaining request from its first token
- Guarantee every queued request runs immediately
- Discard the finished answer without returning it
- Admit eligible queued work if its resource budgets allow
- Ni ubuhe buryo bukoreshwa muburyo busanzwe hamwe no gukomeza guterana kugirango ushiremo / ukureho ibihe bihendutse?
- Urupapuro rwibitekerezo rushingiye kuri KV cache
- FP32 ibisobanuro bibiri
- Guhinduranya disiki
- Intoki zamakuru
- Ni izihe mpapuro z'ubushakashatsi zisanzwe zishimirwa no gukwirakwiza itera-urwego (rukomeza)?
- Impapuro zumwimerere
- Urupapuro rwa Orca
- Urupapuro rwububiko rwa 1980
- Urupapuro
- What utilization problem can continuous batching reduce compared with a fixed request batch?
- Every model accuracy error
- The need to store any KV cache
- All queueing at any traffic rate
- Waiting for the longest request before admitting new eligible work
- A continuous-batching server has reached its memory budget. Must a newly arrived request enter the next iteration?
- No; admission can wait until suitable capacity is available
- Yes; the name guarantees unlimited request admission
- Yes; the server removes all active outputs automatically
- Only if the request has no punctuation
- A long prompt’s prefill delays tokens for already decoding requests. Which serving objective exposes that tradeoff?
- Measure only the weight-file name
- Assume more admitted prompts always improve every latency
- Measure time between decode tokens alongside throughput
- Count only the number of model layers