Kuiz berkaitan panduan · Keras Tahap

Jumlah Kos Pemilikan untuk Kuiz LLM Dihoskan Sendiri

Uji perkara yang mendorong kos sebenar LLM pengehosan sendiri: matematik memori GPU, penggunaan, kakitangan dan pulang modal berbanding harga API per-token.

Laluan panduan yang berkaitanJumlah Kos Pemilikan LLM Dihoskan Sendiri
soalan 1 daripada 8

Kira-kira berapa banyak memori yang diperlukan berat model 70‑billion‑parameter pada ketepatan 16‑bit?

Pratonton setiap soalan8 soalan

Jawapan dan penjelasan muncul semasa anda mengambil kuiz.

  1. Kira-kira berapa banyak memori yang diperlukan berat model 70‑billion‑parameter pada ketepatan 16‑bit?
    • About 140 GB
    • About 35 GB
    • About 70 GB
    • About 560 GB
  2. Bagaimana panduan mendefinisikan kos efektif per token bagi penyebaran yang dihoskan sendiri?
    • GPU hourly rental price divided by the model's parameter count
    • The API provider's list price minus a fixed discount
    • Total monthly cost divided by tokens actually served
    • Hardware purchase price divided by the number of users
  3. Mengapa kluster self-hosted yang jarang digunakan boleh lebih mahal per token berbanding API?
    • Self-hosted models produce longer answers
    • GPUs cost the same whether they are idle or busy, so fewer tokens share the fixed cost
    • Open-weight licenses charge a fee per token
    • APIs are subsidized by governments
  4. Kos apa yang panduan kenal pasti sering menjadi baris tersembunyi terbesar dalam self-hosting?
    • Model download fees
    • Tokenizer licenses
    • Electricity for laptops
    • Staff who deploy, tune, patch, upgrade and stay on call for the stack
  5. Selain berat model, apa lagi yang menggunakan memori GPU dan berkembang dengan panjang konteks serta bilangan permintaan serentak?
    • Perbendaharaan kata tokenizer
    • The load balancer
    • The KV cache
    • The model license file
  6. Why is it a misconception that self-hosting is automatically more private?
    • A poorly secured self-hosted server can be riskier than a provider with audited controls
    • Open-weight models send data back to their creators
    • APIs never store any data
    • Privacy laws ban self-hosting
  7. What is the purpose of continuous batching in servers such as vLLM?
    • To compress model weights to 4 bits
    • To translate prompts into other languages
    • To download new model versions automatically
    • To keep GPUs busy by processing many requests together
  8. What does the guide describe as the likely pattern for many organizations?
    • Moving every workload to self-hosting
    • Hybrid: APIs for spiky or frontier workloads, self-hosting for steady, high-volume or sensitive ones
    • Using only APIs forever
    • Choosing once and never revisiting the decision