Тест, свързан с ръководство · Среден Ниво

Chinchilla Compute-Оптимален тест за обучение

Изучете подробно модела на хартията Chinchilla и токените, трите метода за оценка, IsoFLOP профилите и настройката за обучение MassiveText.

Свързани ръководни пътекиОптимално обучение за изчисления на чинчили
Въпрос 1 на 8

How many language models did the Chinchilla study train for its scaling analysis?

Визуализирайте всеки въпрос8 въпроси

Отговорите и обясненията се появяват, докато отговаряте на теста.

  1. How many language models did the Chinchilla study train for its scaling analysis?
    • Over 400
    • Fewer than 10
    • Exactly 20
    • More than 100,000
  2. What parameter-count range did the paper’s model sweep cover?
    • 1 million to 10 million
    • About 70 million to over 16 billion parameters
    • 70 billion to over 1 trillion
    • Only exactly 70 billion
  3. What training-token range did the paper’s sweep cover?
    • 5 million to 50 million
    • Only 300 billion tokens
    • About 5 billion to 500 billion tokens
    • 1 trillion to 10 trillion tokens
  4. What did the paper’s first estimation approach vary while holding model sizes fixed?
    • The tokenizer for every step
    • The number of evaluation benchmarks only
    • The dataset language while keeping training curves unused
    • Training duration, measured by the number of training sequences or tokens
  5. In an IsoFLOP profile, what is held fixed while model sizes are compared?
    • The total training compute budget in FLOPs
    • The token count for every model regardless of size
    • The number of evaluation raters
    • The vocabulary size only
  6. What shape did the paper look for in loss versus model size on an IsoFLOP profile?
    • A line that always decreases with no minimum
    • A valley with a minimum-loss model size for that compute budget
    • A constant horizontal score for every size
    • A random pattern unrelated to compute
  7. How did the paper fit the parameters of its third, parametric loss approach?
    • By hand-labeling every token
    • Using K-means without a loss function
    • With Huber loss and L-BFGS, selecting a good fit from a grid of initializations
    • By comparing only model names
  8. Which training dataset did the Chinchilla model use?
    • ImageNet only
    • The Common Voice speech corpus
    • A dataset created only from generated model answers
    • MassiveText, with a slightly different subset distribution from Gopher