Cuestionario vinculado a una guía · Medio Nivel

Prueba de entrenamiento óptimo en computación de Chinchilla

Estudie en detalle el modelo del artículo de Chinchilla y los barridos de tokens, tres métodos de estimación, los perfiles IsoFLOP y la configuración de entrenamiento de MassiveText.

Rutas de guía relacionadasChinchilla Compute Entrenamiento Óptimo
Pregunta 1 de 8

How many language models did the Chinchilla study train for its scaling analysis?

Vista previa de cada pregunta8 preguntas

Las respuestas y explicaciones aparecen a medida que realiza el cuestionario.

  1. How many language models did the Chinchilla study train for its scaling analysis?
    • Over 400
    • Fewer than 10
    • Exactly 20
    • More than 100,000
  2. What parameter-count range did the paper’s model sweep cover?
    • 1 million to 10 million
    • About 70 million to over 16 billion parameters
    • 70 billion to over 1 trillion
    • Only exactly 70 billion
  3. What training-token range did the paper’s sweep cover?
    • 5 million to 50 million
    • Only 300 billion tokens
    • About 5 billion to 500 billion tokens
    • 1 trillion to 10 trillion tokens
  4. What did the paper’s first estimation approach vary while holding model sizes fixed?
    • The tokenizer for every step
    • The number of evaluation benchmarks only
    • The dataset language while keeping training curves unused
    • Training duration, measured by the number of training sequences or tokens
  5. In an IsoFLOP profile, what is held fixed while model sizes are compared?
    • The total training compute budget in FLOPs
    • The token count for every model regardless of size
    • The number of evaluation raters
    • The vocabulary size only
  6. What shape did the paper look for in loss versus model size on an IsoFLOP profile?
    • A line that always decreases with no minimum
    • A valley with a minimum-loss model size for that compute budget
    • A constant horizontal score for every size
    • A random pattern unrelated to compute
  7. How did the paper fit the parameters of its third, parametric loss approach?
    • By hand-labeling every token
    • Using K-means without a loss function
    • With Huber loss and L-BFGS, selecting a good fit from a grid of initializations
    • By comparing only model names
  8. Which training dataset did the Chinchilla model use?
    • ImageNet only
    • The Common Voice speech corpus
    • A dataset created only from generated model answers
    • MassiveText, with a slightly different subset distribution from Gopher