Maswali yanayounganishwa na mwongozo · Ngumu Kiwango

Maswali ya Kuunganisha yanayoendelea

Tekeleza ulinganishaji wa kiwango cha marudio cha LLM, vikomo vya uwezo, na usawazishaji wa kujaza mapema, ikijumuisha kwa nini kukubali maombi mfululizo hakuhakikishii sifuri kusubiri.

Njia za mwongozo zinazohusianaKuunganisha Kuendelea
Swali 1 ya 8

Je! ni udhaifu gani mkuu wa batching tuli (iliyowekwa) kwa huduma ya LLM?

Hakiki kila swali8 maswali

Majibu na maelezo huonekana unapojibu maswali.

  1. Je! ni udhaifu gani mkuu wa batching tuli (iliyowekwa) kwa huduma ya LLM?
    • Inaweza tu kukimbia kwenye CPU
    • Inahitaji kuweka lebo data zote kwanza
    • Maombi mafupi hukamilika mapema lakini nafasi zao hazina kazi huku kundi likisubiri lile refu zaidi
    • Inafanya mfano sio sahihi
  2. Je, ni kwa uzito gani kuendelea kubandika huongeza na kuondoa maombi?
    • Mara moja kwa siku
    • Kwa hatua ya kusimbua (ishara moja kwa wakati mmoja)
    • Kwa ombi kamili pekee
    • Kwa enzi ya mafunzo
  3. A continuous-batching request finishes and frees capacity. What can the scheduler do next?
    • Restart every remaining request from its first token
    • Guarantee every queued request runs immediately
    • Discard the finished answer without returning it
    • Admit eligible queued work if its resource budgets allow
  4. Je, ni mbinu gani huoanishwa kwa njia ya kawaida na upangaji unaoendelea ili kufanya uwekaji/uondoaji wa mlolongo kuwa nafuu?
    • Akiba ya KV yenye msingi wa PagedAttention
    • FP32 usahihi mara mbili
    • Ubadilishanaji wa diski
    • Kuweka lebo data kwa mikono
  5. Ni karatasi gani ya utafiti ambayo kwa kawaida hupewa sifa ya kueneza upangaji wa kiwango cha kurudia (kuendelea)?
    • Karatasi ya asili ya Transfoma
    • Karatasi ya Orca
    • Karatasi ya hifadhidata ya miaka ya 1980
    • Karatasi ya ImageNet
  6. What utilization problem can continuous batching reduce compared with a fixed request batch?
    • Every model accuracy error
    • The need to store any KV cache
    • All queueing at any traffic rate
    • Waiting for the longest request before admitting new eligible work
  7. A continuous-batching server has reached its memory budget. Must a newly arrived request enter the next iteration?
    • No; admission can wait until suitable capacity is available
    • Yes; the name guarantees unlimited request admission
    • Yes; the server removes all active outputs automatically
    • Only if the request has no punctuation
  8. A long prompt’s prefill delays tokens for already decoding requests. Which serving objective exposes that tradeoff?
    • Measure only the weight-file name
    • Assume more admitted prompts always improve every latency
    • Measure time between decode tokens alongside throughput
    • Count only the number of model layers