Cuestionario vinculado a una guía · Medio Nivel

Prueba de modelos de lenguaje pequeño

Compare recursos de modelos compactos, con datos seleccionados, destilación, poda, almacenamiento de peso cuantificado, cachés KV y la báscula de entrenamiento Phi-3 original.

Rutas de guía relacionadasModelos de lenguaje pequeño
Pregunta 1 de 8

What resource goal motivates a compact language-model implementation?

Vista previa de cada pregunta8 preguntas

Las respuestas y explicaciones aparecen a medida que realiza el cuestionario.

  1. What resource goal motivates a compact language-model implementation?
    • Guaranteeing it beats every larger model on every task
    • Reducing resource demands for a chosen task and device setup
    • Eliminating the need for all training data
    • Proving every phone runs every checkpoint identically
  2. What training factor do the cited Phi studies investigate alongside model size?
    • Curated, filtered or synthetic training-data choices
    • Only random unfiltered data with no objective
    • Removal of all model parameters
    • A requirement to train only at night
  3. What does teacher/student knowledge distillation train the student to learn from?
    • Only a new image-file extension
    • A guarantee of every possible teacher capability
    • The teacher’s outputs or distributions under a stated objective
    • Only a hardware-temperature label
  4. What does weight quantization change in the storage setup described here?
    • Necessarily the number of model parameters
    • The language of every output sentence
    • Only the order of filenames
    • The numerical representation and bits used for weights
  5. Which compression operation removes or suppresses selected network connections?
    • Poda
    • Only changing a prompt’s font
    • Only enlarging the vocabulary
    • Only retaining every weight at its original value
  6. What work does a key-value attention cache avoid repeating during autoregressive generation?
    • Every possible future training operation
    • All text-tokenization work for every application
    • Recomputing prior keys and values for subsequent token processing
    • The need to retain any state at all
  7. Ignoring all overhead, how much raw storage is needed for 3.8 billion scalar weights packed at 4 bits each?
    • 3.8 billion bytes
    • 15.2 billion bytes
    • 0.475 billion bytes
    • 1.9 billion bytes
  8. How many training tokens does the original Phi-3 report state for phi-3-mini?
    • 3.8 billion tokens
    • 3.3 trillion tokens
    • Only 3.3 million tokens
    • No training-token count was stated