Epipolar Geometry and the Fundamental Matrix

When two cameras view the same scene from different positions, a 3D point projects to one pixel in each image. Finding the matching pixel by searching the entire second image is expensive and ambiguous. Epipolar geometry narrows the search: the match for a point in the first image should lie on a particular line in the second image, called its epipolar line. The line is determined by the two camera centers and the scene point’s viewing ray. The fundamental matrix F captures this relationship between image coordinates. In homogeneous coordinates, corresponding points x and x′ satisfy x′ᵀFx = 0; applying F to a point gives the corresponding epipolar line. The matrix describes the geometry between the two views without requiring camera calibration. With known camera intrinsics, the essential matrix expresses the related geometry in normalized camera coordinates. These matrices do not directly provide a complete scene model: point correspondences, camera assumptions, calibration, and triangulation still matter. In practice, a system detects candidate features, matches descriptors, estimates F robustly from tentative pairs, and checks geometric consistency. Mismatched features, moving objects, rolling shutter, lens distortion, and nearly planar scenes can weaken an estimate. A stereo system can then use rectification to align epipolar lines horizontally, making disparity search simpler. Depth estimation still depends on camera calibration and baseline, and uncertainty increases for distant points or weak texture. Treat F as a constraint for matching and reconstruction, not as depth by itself.