CLIP Score and Human Preference Metrics Quiz

Answers and explanations appear as you take the quiz.

1. Why is FID alone insufficient for evaluating text-to-image models?
   • It requires human annotators for every image
   • It never looks at the prompt, so it cannot measure text alignment
   • It only works on black-and-white images
   • It measures CLIP similarity instead of realism
2. How is a basic CLIP score computed?
   • By counting pixels that match the prompt's colors
   • By training a discriminator on the generated images
   • By averaging Inception features across a dataset
   • By taking the cosine similarity between CLIP's image and text embeddings
3. Which dataset was PickScore trained on?
   • Pick-a-Pic, a set of choices made by users of a web app
   • COCO captions only
   • ImageNet class labels
   • A collection of FID statistics
4. Which weakness is typical of CLIP-based scoring?
   • It cannot process color images
   • It always penalizes photorealism
   • It often misses counts, spatial relations and which attribute belongs to which object
   • It requires prompts shorter than five words
5. Why can't CLIP scores from different backbones be compared directly?
   • All backbones produce identical scores, so there is nothing to compare
   • Different encoders such as ViT-B/32 and ViT-L/14 have different embedding spaces and score ranges
   • CLIP score is only defined for one backbone
   • Backbones only affect FID, not CLIP score
6. What risk arises when a model is optimized directly against a preference score?
   • The score becomes impossible to compute
   • The model loses the ability to read prompts
   • It may learn to please the scorer, for example with oversaturated images, rather than people
   • FID automatically improves
7. What typically happens to the part of a prompt beyond CLIP's 77-token limit?
   • It is scored separately and averaged in
   • The encoder expands its context window automatically
   • It is summarized into fewer tokens first
   • It is truncated, so it never affects the score
8. Which kind of evaluation checks whether each object and attribute in a prompt actually appears?
   • Question-answering or detector-based benchmarks such as TIFA and GenEval
   • FID
   • Raw CLIP cosine similarity
   • Raising the guidance scale