quiz bu lëkkale ak tegtal · Dëgër Niveau

Total njëgu moomeel ngir quiz LLMs Self-Hosted

Natt liy dawal njëgu dëggu LLMs self-hosting: GPU math memory, jëfandikoo, taggat ak dogg-even ci njëg API token bu nekk.

Laaj 1 de 8

Lu tollu ci ñaata mémoire la model bu am 70 milyaar ciy parametre soxla ci 16-bit ci njub?

Xoolaatal laaj bu nekk8 laaj

Sooy def quiz bi, tontu yi ak leeral yi dañuy feeñ.

  1. Lu tollu ci ñaata mémoire la model bu am 70 milyaar ciy parametre soxla ci 16-bit ci njub?
    • Lu tollu ci 140 GB
    • Lu tollu ci 35 GB
    • Lu tollu ci 70 GB
    • Lu tollu ci 560 GB
  2. naka lay def ba tegtal bi di fësal njëg bu baax bi ci token bu nekk ci benn deployment buñ yamale boppam?
    • Njëg luwaasu GPU ci waxtu xaaj ko ak limu parametru model bi
    • Njëg limu joxekatu API bi dindi ci wàññi giñ tëral
    • Tolluwaayu njëgu weer wu nekk xaaj ko ak jeton yiñ joxe
    • Njëg li ngay jënd aparey xaaj ko ak limu jëfandikukat yi
  3. Lan moo waral cluster self-hosted buñu jëfandikoo bu baax mën na gëna njëg token bu nekk API?
    • Royuwaayi self-hosted yi defar nañu tontu yu gëna gudd
    • GPUs yi ñooy benn njëg, muy idle wala busy, kon token yu néew ñoo bokk njëg bu takku bi
    • Lisaas yu ubbeeku yi dañuy fay token bu nekk
    • Nguur yi ñooy subsidise API yi
  4. Ban njëg la guide bi ràññee ni mooy ligne bu gëna mag bi nëbbu ci self-hosting?
    • Model biy yebbi
    • Lisaas tokeniseer
    • Kuuraŋ ngir portable
    • Ndaw yiy dugal, tune, patch, yeesal ak des ci woote ngir stack
  5. Lu weesu poid model yi, lan mooy jëfandikoo mémoire GPU te dafay màgg ak guddaayi contexte bi ak limu laaj yiy boole?
    • Baatu tokenisatër bi
    • Balanceur de charge
    • cache KV
    • Dosie lisence model bi
  6. Lan moo waral ñu gëm ni self-hosting mooy gëna nëbbu ci saasi?
    • Serveur bu self-hosted bu wóorul bu baax mën na gëna am risk yeneen fournisseur yu am contrôle yuñ saytu
    • Modèle yu ubbeeku yi dañuy yónnee done yi seen defarkat yi
    • API yi duñu musa denc benn done
    • Yoon yiy tënk kàddu yu nëbbu dañu tere self-hosting
  7. Lan mooy sabab batching bu wéy ci serveur yu melni vLLM?
    • Ngir wàññi diisaayu model bi ba 4 bit
    • Tekki ay laaj ci yeneen làkk
    • Ngir yebbi ci saasi xeetu model yu bees
    • Ngir GPU yi wéy di liggéey ci boole ay laaj yu bari
  8. Lan la tegtal bi di wax ni mooy xeetu liggéey bi mbootaay yu bari di def?
    • Toxal bépp liggéey ci self-hosting
    • Hybrid: APIs ngir liggéey yu bari wala frontiere, self-hosting ngir yu dëgër, yu bari ay volume wala yu am solo
    • Jëfandikoo API yi kese ba fàww
    • Tann benn yoon te doo musa xoolaat dogal bi