Cuestionario vinculado a una guía · Medio Nivel

Prueba de puntuación CLIP y métricas de preferencia humana

Prueba cómo la puntuación CLIP mide la alineación rápida, cómo se entrenan los modelos de preferencia como PickScore e ImageReward y dónde falla cada métrica.

Rutas de guía relacionadasPuntuación de clip y métricas de preferencia humana
Pregunta 1 de 8

Why is FID alone insufficient for evaluating text-to-image models?

Vista previa de cada pregunta8 preguntas

Las respuestas y explicaciones aparecen a medida que realiza el cuestionario.

  1. Why is FID alone insufficient for evaluating text-to-image models?
    • It requires human annotators for every image
    • It never looks at the prompt, so it cannot measure text alignment
    • Sólo funciona en imágenes en blanco y negro.
    • It measures CLIP similarity instead of realism
  2. How is a basic CLIP score computed?
    • By counting pixels that match the prompt's colors
    • By training a discriminator on the generated images
    • By averaging Inception features across a dataset
    • By taking the cosine similarity between CLIP's image and text embeddings
  3. Which dataset was PickScore trained on?
    • Pick-a-Pic, a set of choices made by users of a web app
    • COCO captions only
    • ImageNet class labels
    • A collection of FID statistics
  4. Which weakness is typical of CLIP-based scoring?
    • It cannot process color images
    • It always penalizes photorealism
    • It often misses counts, spatial relations and which attribute belongs to which object
    • It requires prompts shorter than five words
  5. Why can't CLIP scores from different backbones be compared directly?
    • All backbones produce identical scores, so there is nothing to compare
    • Different encoders such as ViT-B/32 and ViT-L/14 have different embedding spaces and score ranges
    • CLIP score is only defined for one backbone
    • Backbones only affect FID, not CLIP score
  6. What risk arises when a model is optimized directly against a preference score?
    • The score becomes impossible to compute
    • The model loses the ability to read prompts
    • It may learn to please the scorer, for example with oversaturated images, rather than people
    • FID automatically improves
  7. What typically happens to the part of a prompt beyond CLIP's 77-token limit?
    • It is scored separately and averaged in
    • The encoder expands its context window automatically
    • It is summarized into fewer tokens first
    • It is truncated, so it never affects the score
  8. Which kind of evaluation checks whether each object and attribute in a prompt actually appears?
    • Question-answering or detector-based benchmarks such as TIFA and GenEval
    • FID
    • Raw CLIP cosine similarity
    • Raising the guidance scale