Maswali yanayounganishwa na mwongozo · Kati Kiwango

Maswali ya Miundo Ndogo ya Lugha

Linganisha rasilimali za muundo wa kompakt, na data iliyoratibiwa, kunereka, kupogoa, uhifadhi wa uzito uliopimwa, akiba ya KV, na mizani asilia ya mafunzo ya Phi-3.

Njia za mwongozo zinazohusianaMiundo ya Lugha Ndogo
Swali 1 ya 8

What resource goal motivates a compact language-model implementation?

Hakiki kila swali8 maswali

Majibu na maelezo huonekana unapojibu maswali.

  1. What resource goal motivates a compact language-model implementation?
    • Guaranteeing it beats every larger model on every task
    • Reducing resource demands for a chosen task and device setup
    • Eliminating the need for all training data
    • Proving every phone runs every checkpoint identically
  2. What training factor do the cited Phi studies investigate alongside model size?
    • Curated, filtered or synthetic training-data choices
    • Only random unfiltered data with no objective
    • Removal of all model parameters
    • A requirement to train only at night
  3. What does teacher/student knowledge distillation train the student to learn from?
    • Only a new image-file extension
    • A guarantee of every possible teacher capability
    • The teacher’s outputs or distributions under a stated objective
    • Only a hardware-temperature label
  4. What does weight quantization change in the storage setup described here?
    • Necessarily the number of model parameters
    • The language of every output sentence
    • Only the order of filenames
    • The numerical representation and bits used for weights
  5. Which compression operation removes or suppresses selected network connections?
    • Kupogoa
    • Only changing a prompt’s font
    • Only enlarging the vocabulary
    • Only retaining every weight at its original value
  6. What work does a key-value attention cache avoid repeating during autoregressive generation?
    • Every possible future training operation
    • All text-tokenization work for every application
    • Recomputing prior keys and values for subsequent token processing
    • The need to retain any state at all
  7. Ignoring all overhead, how much raw storage is needed for 3.8 billion scalar weights packed at 4 bits each?
    • 3.8 billion bytes
    • 15.2 billion bytes
    • 0.475 billion bytes
    • 1.9 billion bytes
  8. How many training tokens does the original Phi-3 report state for phi-3-mini?
    • 3.8 billion tokens
    • 3.3 trillion tokens
    • Only 3.3 million tokens
    • No training-token count was stated