Kvíz spojený s průvodcem · Střední úroveň

Kvíz skóre a lidských preferencí CLIP

Testuje, jak skóre CLIP měří rychlé zarovnání, jak jsou trénovány preferenční modely, jako je PickScore a ImageReward, a kde která metrika selhává.

Otázka 1 z 8

Why is FID alone insufficient for evaluating text-to-image models?

Náhled každé otázky8 otázky

Odpovědi a vysvětlení se zobrazují, jak budete v kvízu.

  1. Why is FID alone insufficient for evaluating text-to-image models?
    • It requires human annotators for every image
    • It never looks at the prompt, so it cannot measure text alignment
    • Funguje pouze na černobílých snímcích
    • It measures CLIP similarity instead of realism
  2. How is a basic CLIP score computed?
    • By counting pixels that match the prompt's colors
    • By training a discriminator on the generated images
    • By averaging Inception features across a dataset
    • By taking the cosine similarity between CLIP's image and text embeddings
  3. Which dataset was PickScore trained on?
    • Pick-a-Pic, a set of choices made by users of a web app
    • COCO captions only
    • ImageNet class labels
    • A collection of FID statistics
  4. Which weakness is typical of CLIP-based scoring?
    • It cannot process color images
    • It always penalizes photorealism
    • It often misses counts, spatial relations and which attribute belongs to which object
    • It requires prompts shorter than five words
  5. Why can't CLIP scores from different backbones be compared directly?
    • All backbones produce identical scores, so there is nothing to compare
    • Different encoders such as ViT-B/32 and ViT-L/14 have different embedding spaces and score ranges
    • CLIP score is only defined for one backbone
    • Backbones only affect FID, not CLIP score
  6. What risk arises when a model is optimized directly against a preference score?
    • The score becomes impossible to compute
    • The model loses the ability to read prompts
    • It may learn to please the scorer, for example with oversaturated images, rather than people
    • FID automatically improves
  7. What typically happens to the part of a prompt beyond CLIP's 77-token limit?
    • It is scored separately and averaged in
    • The encoder expands its context window automatically
    • It is summarized into fewer tokens first
    • It is truncated, so it never affects the score
  8. Which kind of evaluation checks whether each object and attribute in a prompt actually appears?
    • Question-answering or detector-based benchmarks such as TIFA and GenEval
    • FID
    • Raw CLIP cosine similarity
    • Raising the guidance scale