Computer Vision

Tasks include classification, object detection, segmentation, tracking, and visual question answering. Each task asks for a different output, and none automatically provides a complete understanding of a scene.

Key takeaways

• Define the visual task and output.
• Test realistic capture conditions.
• Evaluate preprocessing and shortcuts.

Images become numerical arrays that encode pixels or other representations. A model learns patterns useful for its objective, but those patterns can include accidental correlations. A classifier may rely on a background rather than the object a developer intended it to recognize. Define the output precisely. Classification assigns labels to an image; detection locates object instances; segmentation labels pixels or regions. A model that identifies an object category may still fail to locate its boundary or distinguish several overlapping instances. Evaluate on realistic cameras, lighting, resolutions, viewpoints, and environments. Keep related images from the same scene or recording together when splitting data to avoid overly optimistic results. Inspect uncommon conditions and the cost of different mistakes. The application must also handle image quality, permissions, uncertainty, and downstream actions. A confident label is not proof that a scene is safe or that an inferred attribute is appropriate to use. Preserve the source image and meaningful review information when people need to check a result.

1. Construct a toy dataset where every training image of a red toy is on a white table and every blue toy is on a dark table.
2. Test the toys on swapped backgrounds and on an unseen surface.
3. If predictions follow the table rather than the toy, revise the data and evaluation rather than assuming the original accuracy measured the intended concept.

The invented setup illustrates a shortcut that a visually plausible demonstration can hide.