Ikibazo gihuza ibibazo · Hagati Urwego

LLM Yerekana Inzira no Kuringaniza Ikibazo

Kugenzura kopi na moderi yerekana inzira, cache affinity, telemetrie nzima, ibiciro bya token igiciro hamwe nubukererwe-bwenge bwinjiza kubintu bya LLM.

Ikibazo 1 Bya 8

Ni ukubera iki uruzinduko rusanzwe akenshi rufite ingamba mbi zo kuringaniza imitwaro ya LLM?

Reba buri kibazo8 ibibazo

Ibisubizo nibisobanuro bigaragara mugihe ufata ikibazo.

  1. Ni ukubera iki uruzinduko rusanzwe akenshi rufite ingamba mbi zo kuringaniza imitwaro ya LLM?
    • Ibisabwa biratandukanye cyane kubiciro kandi kopi zifata kashe ya KV
    • Birabujijwe mu bihugu byinshi
    • Round-robin ikora gusa kububiko
    • Birasaba GPU gukora
  2. Niki 'prefix-cache affinity' inzira igerageza kugeraho?
    • Kohereza buri cyifuzo kuri GPU idasanzwe
    • Guhindura icyifuzo kuri kopi yamaze guhisha prefix yacyo
    • Shishoza ikibazo mbere yo kohereza
    • Kugabanya uburemere bw'icyitegererezo
  3. Mubibazo bishingiye ku cyitegererezo cyerekana, mubisanzwe bigenda bite kubibazo byoroshye?
    • Ntabwo byanze
    • Yoherejwe kuri moderi nini iboneka
    • Yoherejwe kuri moderi ntoya, ihendutse
    • Itonze umurongo amasaha 24
  4. Nibihe bimenyetso bizima bifite akamaro kanini kuri LLM-imenyekanisha umutwaro?
    • Seriveri Izina ryuburebure
    • Ibara rya rack
    • Igihe cyumukoresha gusa
    • Uburebure bwumurongo hamwe na KV-cache umwanya wa buri kopi
  5. Niki igikoresho nka LiteLLM gitanga muburyo butandukanye?
    • Porokireri imwe iringaniza ikananirwa hejuru yabatanga moderi benshi
    • Ubwubatsi bushya bwububiko
    • Sisitemu yo gukonjesha GPU
    • Umusimbuzi wa tokenizer
  6. Ni ukubera iki icyifuzo kimwe kirekire gisohoka gisumba byinshi bigufi kuringaniza?
    • Ibisubizo birebire ntibikoresha ibikoresho
    • Buri gisohoka cyerekana ikiguzi cyuzuye imbere, so token yikintu cyose kirenze icyifuzo cyo kubara
    • Gusaba bigufi burigihe buhoro
    • Router ntishobora kubona ibisohoka uburebure na busa
  7. A serving system reports goodput rather than raw request count. What does goodput usually emphasize?
    • Umubare rusange wibitegererezo
    • Only the number of HTTP connections opened
    • Requests served while meeting the specified latency or service-level targets
    • The fraction of requests that use the largest model
  8. A gateway first chooses a small or large model, then chooses a replica of that model. What are these two decisions?
    • Tokenization followed by decoding
    • Training followed by fine-tuning
    • Prefill followed by prompt construction
    • Model routing followed by replica load balancing