Ikibazo gihuza ibibazo · Hagati Urwego
Amategeko yo gupima Chinchilla
Gerageza Chinchilla amahugurwa-yingengo yimari, 70B / 1.4T urugero, urugero rwo gupima, kubara hafi, no gutandukanya uburyo bwo kohereza.
Ikibazo 1 Bya 8
What does Chinchilla's finding that parameters and tokens should scale in similar proportions mean?
Reba buri kibazo8 ibibazo
Ibisubizo nibisobanuro bigaragara mugihe ufata ikibazo.
- What does Chinchilla's finding that parameters and tokens should scale in similar proportions mean?
- Both should grow with comparable scaling exponents under the studied training-compute objective
- The numerical token count must equal the parameter count
- Training data does not affect the fitted loss
- Every deployment must choose the largest possible model
- What token-per-parameter ratio does the 70B-parameter, 1.4T-token Chinchilla experiment illustrate?
- Exactly one token per parameter
- About two thousand tokens per parameter
- About twenty training tokens per parameter
- A ratio determined only by the number of GPUs
- Ni ubuhe bwoko Chinchilla yarushije nubwo ari buto cyane?
- GPT-2
- Gopher
- BERT
- Mamba
- Chinchilla yashakaga kuvuga iki kubijyanye na moderi nka GPT-3 icyo gihe?
- Bararengereye
- Bafite compte nkeya cyane
- Bakoresheje tokenizer itari yo
- Bahawe imyitozo ugereranije nubunini bwabo
- Hafi ya compte yagereranijwe gute mu isesengura rya Chinchilla?
- Ugereranije numubare wa GPU gusa
- Yigenga yubunini bwamakuru
- Ugereranije nibihe ibihe byamahugurwa ibimenyetso
- Ugereranije nubunini bwamagambo
- Why might a deployment objective favor a smaller model trained on extra data rather than the training-compute optimum alone?
- Repeated inference costs can matter alongside the initial training budget
- Inference has no resource cost for any model
- The Chinchilla study proved every scaling relationship false
- More data always reduces both training and serving work to zero
- Under the approximation that training compute is proportional to N times D, how must D change if N doubles while that compute budget stays fixed?
- D must also double
- D must quadruple
- D must remain unchanged
- D must approximately halve
- Which three types of term appear in Chinchilla’s parametric loss fit?
- Only GPU count, electricity price, and elapsed seconds
- An irreducible base term, a model-size contribution, and a data-size contribution
- Only vocabulary size, file size, and model name
- Only a linear token count with no model-size term