quiz bu lëkkale ak tegtal · Dëgër Niveau
Total njëgu moomeel ngir quiz LLMs Self-Hosted
Natt liy dawal njëgu dëggu LLMs self-hosting: GPU math memory, jëfandikoo, taggat ak dogg-even ci njëg API token bu nekk.
Laaj 1 de 8
Lu tollu ci ñaata mémoire la model bu am 70 milyaar ciy parametre soxla ci 16-bit ci njub?
Xoolaatal laaj bu nekk8 laaj
Sooy def quiz bi, tontu yi ak leeral yi dañuy feeñ.
- Lu tollu ci ñaata mémoire la model bu am 70 milyaar ciy parametre soxla ci 16-bit ci njub?
- Lu tollu ci 140 GB
- Lu tollu ci 35 GB
- Lu tollu ci 70 GB
- Lu tollu ci 560 GB
- naka lay def ba tegtal bi di fësal njëg bu baax bi ci token bu nekk ci benn deployment buñ yamale boppam?
- Njëg luwaasu GPU ci waxtu xaaj ko ak limu parametru model bi
- Njëg limu joxekatu API bi dindi ci wàññi giñ tëral
- Tolluwaayu njëgu weer wu nekk xaaj ko ak jeton yiñ joxe
- Njëg li ngay jënd aparey xaaj ko ak limu jëfandikukat yi
- Lan moo waral cluster self-hosted buñu jëfandikoo bu baax mën na gëna njëg token bu nekk API?
- Royuwaayi self-hosted yi defar nañu tontu yu gëna gudd
- GPUs yi ñooy benn njëg, muy idle wala busy, kon token yu néew ñoo bokk njëg bu takku bi
- Lisaas yu ubbeeku yi dañuy fay token bu nekk
- Nguur yi ñooy subsidise API yi
- Ban njëg la guide bi ràññee ni mooy ligne bu gëna mag bi nëbbu ci self-hosting?
- Model biy yebbi
- Lisaas tokeniseer
- Kuuraŋ ngir portable
- Ndaw yiy dugal, tune, patch, yeesal ak des ci woote ngir stack
- Lu weesu poid model yi, lan mooy jëfandikoo mémoire GPU te dafay màgg ak guddaayi contexte bi ak limu laaj yiy boole?
- Baatu tokenisatër bi
- Balanceur de charge
- cache KV
- Dosie lisence model bi
- Lan moo waral ñu gëm ni self-hosting mooy gëna nëbbu ci saasi?
- Serveur bu self-hosted bu wóorul bu baax mën na gëna am risk yeneen fournisseur yu am contrôle yuñ saytu
- Modèle yu ubbeeku yi dañuy yónnee done yi seen defarkat yi
- API yi duñu musa denc benn done
- Yoon yiy tënk kàddu yu nëbbu dañu tere self-hosting
- Lan mooy sabab batching bu wéy ci serveur yu melni vLLM?
- Ngir wàññi diisaayu model bi ba 4 bit
- Tekki ay laaj ci yeneen làkk
- Ngir yebbi ci saasi xeetu model yu bees
- Ngir GPU yi wéy di liggéey ci boole ay laaj yu bari
- Lan la tegtal bi di wax ni mooy xeetu liggéey bi mbootaay yu bari di def?
- Toxal bépp liggéey ci self-hosting
- Hybrid: APIs ngir liggéey yu bari wala frontiere, self-hosting ngir yu dëgër, yu bari ay volume wala yu am solo
- Jëfandikoo API yi kese ba fàww
- Tann benn yoon te doo musa xoolaat dogal bi