Quiz połączony z przewodnikiem · Ciężko Poziom

Całkowity koszt posiadania quizu LLM na własnym serwerze

Sprawdź, co wpływa na prawdziwy koszt samodzielnego hostingu LLM: obliczenia dotyczące pamięci GPU, wykorzystanie, personel i próg rentowności w porównaniu z cenami API za token.

Powiązane ścieżki prowadząceCałkowity koszt posiadania LLM z własnym hostingiem
Pytanie 1 z 8

Roughly how much memory do the weights of a 70-billion-parameter model need at 16-bit precision?

Podgląd każdego pytania8 pytania

Odpowiedzi i wyjaśnienia pojawiają się w miarę rozwiązywania quizu.

  1. Roughly how much memory do the weights of a 70-billion-parameter model need at 16-bit precision?
    • About 140 GB
    • About 35 GB
    • About 70 GB
    • About 560 GB
  2. How does the guide define the effective cost per token of a self-hosted deployment?
    • GPU hourly rental price divided by the model's parameter count
    • The API provider's list price minus a fixed discount
    • Total monthly cost divided by tokens actually served
    • Hardware purchase price divided by the number of users
  3. Why can a lightly used self-hosted cluster cost more per token than an API?
    • Self-hosted models produce longer answers
    • GPUs cost the same whether they are idle or busy, so fewer tokens share the fixed cost
    • Open-weight licenses charge a fee per token
    • APIs are subsidized by governments
  4. Which cost does the guide identify as often the largest hidden line in self-hosting?
    • Model download fees
    • Tokenizer licenses
    • Electricity for laptops
    • Staff who deploy, tune, patch, upgrade and stay on call for the stack
  5. Besides the model weights, what uses GPU memory and grows with context length and the number of concurrent requests?
    • Słownictwo tokenizera
    • The load balancer
    • The KV cache
    • The model license file
  6. Why is it a misconception that self-hosting is automatically more private?
    • A poorly secured self-hosted server can be riskier than a provider with audited controls
    • Open-weight models send data back to their creators
    • APIs never store any data
    • Privacy laws ban self-hosting
  7. What is the purpose of continuous batching in servers such as vLLM?
    • To compress model weights to 4 bits
    • To translate prompts into other languages
    • To download new model versions automatically
    • To keep GPUs busy by processing many requests together
  8. What does the guide describe as the likely pattern for many organizations?
    • Moving every workload to self-hosting
    • Hybrid: APIs for spiky or frontier workloads, self-hosting for steady, high-volume or sensitive ones
    • Using only APIs forever
    • Choosing once and never revisiting the decision