Image rectification

This process has several degrees of freedom and there are many strategies for transforming images to the common plane.

Computer stereo vision takes two or more images with known relative camera positions that show an object from different viewpoints.

However, in general settings (the camera does move towards the object or rotate) the epipolar lines are slanted.

Even with high-precision equipment, image rectification is usually performed because it may be impractical to maintain perfect coplanarity between cameras.

[2] If the images to be rectified are taken from camera pairs without geometric distortion, this calculation can easily be made with a linear transformation.

If the fundamental matrix is not known, it is necessary to find preliminary point correspondences between stereo images to facilitate its extraction.

For example, constraining the epipolar lines to be parallel with the horizontal axis means that epipoles must be mapped to the infinite point [1,0,0]T in homogeneous coordinates.

Poor choices of H and H' can result in rectified images that are dramatically changed in scale or severely distorted.

One advanced method is minimizing the disparity or least-square difference of corresponding points on the horizontal axis of the rectified image pair.

We will briefly outline and depict the results for a simple approach to find a H and H' projective transformation that rectify the image pair from the example scene.

[2] Next, we find the projective transformation H2 that takes the rotated image and twists it so that the horizontal axis aligns with the baseline.

If calculated correctly, this second transformation should map the e to infinity on the x axis (row 3, column 1 of 2D image set).

One strategy is to pick a plane parallel to the line where the two original optical axes intersect to minimize distortion from the reprojection process.

[11] Primary difficulties in the process occur The maps that are used with rectified images are non-topographical.

A camera (red) rotates about the blue axis by 5° to 90° (green), as the images are rectified by projection to the virtual image plane (blue). The virtual plane must be parallel to the stereo baseline (orange) and for visualization is located in the center of rotation. In this case, rectification is achieved by a virtual rotation of the red and green image planes, respectively, to be parallel to the stereo baseline.
The search for point 's match is restricted to the line in the right image. Since the images are not rectified, the line is slanted. After rectification it would be horizontal.
Model used for image rectification example.
3D view of example scene. The first camera's optical center and image plane are represented by the green circle and square respectively. The second camera has similar red representations.
Set of 2D images from example. The original images are taken from different perspectives (row 1). Using systematic transformations from the example (rows 2 and 3), we are able to transform both images such that corresponding points are on the same horizontal scan lines (row 4).