গাইড লিঙ্ক ক্যুইজ · কঠিন স্তর
স্ব-হোস্টেড এলএলএম কুইজের জন্য মালিকানার মোট খরচ
স্ব-হোস্টিং এলএলএম-এর প্রকৃত খরচ কী চালায় তা পরীক্ষা করুন: প্রতি-টোকেন API মূল্যের বিপরীতে GPU মেমরির গণিত, ব্যবহার, স্টাফিং এবং ব্রেক-ইভেন।
প্রশ্ন 1 এর 8
Roughly how much memory do the weights of a 70-billion-parameter model need at 16-bit precision?
প্রতিটি প্রশ্নের পূর্বরূপ দেখুন8 প্রশ্ন
আপনি কুইজ নেওয়ার সাথে সাথে উত্তর এবং ব্যাখ্যাগুলি উপস্থিত হয়।
- Roughly how much memory do the weights of a 70-billion-parameter model need at 16-bit precision?
- About 140 GB
- About 35 GB
- About 70 GB
- About 560 GB
- How does the guide define the effective cost per token of a self-hosted deployment?
- GPU hourly rental price divided by the model's parameter count
- The API provider's list price minus a fixed discount
- Total monthly cost divided by tokens actually served
- Hardware purchase price divided by the number of users
- Why can a lightly used self-hosted cluster cost more per token than an API?
- Self-hosted models produce longer answers
- GPUs cost the same whether they are idle or busy, so fewer tokens share the fixed cost
- Open-weight licenses charge a fee per token
- APIs are subsidized by governments
- Which cost does the guide identify as often the largest hidden line in self-hosting?
- Model download fees
- Tokenizer licenses
- Electricity for laptops
- Staff who deploy, tune, patch, upgrade and stay on call for the stack
- Besides the model weights, what uses GPU memory and grows with context length and the number of concurrent requests?
- টোকেনাইজার শব্দভাণ্ডার
- The load balancer
- The KV cache
- The model license file
- Why is it a misconception that self-hosting is automatically more private?
- A poorly secured self-hosted server can be riskier than a provider with audited controls
- Open-weight models send data back to their creators
- APIs never store any data
- Privacy laws ban self-hosting
- What is the purpose of continuous batching in servers such as vLLM?
- To compress model weights to 4 bits
- To translate prompts into other languages
- To download new model versions automatically
- To keep GPUs busy by processing many requests together
- What does the guide describe as the likely pattern for many organizations?
- Moving every workload to self-hosting
- Hybrid: APIs for spiky or frontier workloads, self-hosting for steady, high-volume or sensitive ones
- Using only APIs forever
- Choosing once and never revisiting the decision