Cuestionario vinculado a una guía · duro Nivel

Cuestionario sobre el costo total de propiedad de los LLM autohospedados

Pruebe qué impulsa el costo real de los LLM con alojamiento propio: cálculos de memoria de GPU, utilización, dotación de personal y equilibrio frente al precio de API por token.

Rutas de guía relacionadasCosto total de propiedad LLM autohospedados
Pregunta 1 de 8

Roughly how much memory do the weights of a 70-billion-parameter model need at 16-bit precision?

Vista previa de cada pregunta8 preguntas

Las respuestas y explicaciones aparecen a medida que realiza el cuestionario.

  1. Roughly how much memory do the weights of a 70-billion-parameter model need at 16-bit precision?
    • About 140 GB
    • About 35 GB
    • About 70 GB
    • About 560 GB
  2. How does the guide define the effective cost per token of a self-hosted deployment?
    • GPU hourly rental price divided by the model's parameter count
    • The API provider's list price minus a fixed discount
    • Total monthly cost divided by tokens actually served
    • Hardware purchase price divided by the number of users
  3. Why can a lightly used self-hosted cluster cost more per token than an API?
    • Self-hosted models produce longer answers
    • GPUs cost the same whether they are idle or busy, so fewer tokens share the fixed cost
    • Open-weight licenses charge a fee per token
    • APIs are subsidized by governments
  4. Which cost does the guide identify as often the largest hidden line in self-hosting?
    • Model download fees
    • Tokenizer licenses
    • Electricity for laptops
    • Staff who deploy, tune, patch, upgrade and stay on call for the stack
  5. Besides the model weights, what uses GPU memory and grows with context length and the number of concurrent requests?
    • El vocabulario tokenizador
    • The load balancer
    • The KV cache
    • The model license file
  6. Why is it a misconception that self-hosting is automatically more private?
    • A poorly secured self-hosted server can be riskier than a provider with audited controls
    • Open-weight models send data back to their creators
    • APIs never store any data
    • Privacy laws ban self-hosting
  7. What is the purpose of continuous batching in servers such as vLLM?
    • To compress model weights to 4 bits
    • To translate prompts into other languages
    • To download new model versions automatically
    • To keep GPUs busy by processing many requests together
  8. What does the guide describe as the likely pattern for many organizations?
    • Moving every workload to self-hosting
    • Hybrid: APIs for spiky or frontier workloads, self-hosting for steady, high-volume or sensitive ones
    • Using only APIs forever
    • Choosing once and never revisiting the decision