ArticleslgStudy

mathematics

Image rectification

Image rectification is a mathematics topic covered in the lgStudy science library. This page brings together a partial reference excerpt, illustrations, worked examples, real-world applications and a short study plan, so you can understand Image rectification rather than just read about it. In short: Image rectification is a transformation process used to project images onto a common image plane. This process has several degrees of freedom and there are many strategies for transforming images to the common plane.

Image rectification — main illustration
Image rectification — illustration

Key takeaways

  • Image rectification belongs to mathematics; place it in that map before memorising details.
  • Learn the definition first, then one example that makes the definition concrete.
  • Connect Image rectification to a quantity you can measure, compute or draw — that is where exam questions come from.
  • Reproduce the core statement of Image rectification from memory before moving on to harder problems.

Reference excerpt

Image rectification is a transformation process used to project images onto a common image plane. This process has several degrees of freedom and there are many strategies for transforming images to the common plane. Image rectification is used in computer stereo vision to simplify the problem of finding matching points between images (i.e. the correspondence problem), and in geographic information systems (GIS) to merge images taken from multiple perspectives into a common map coordinate system.

In computer vision

Computer stereo vision takes two or more images with known relative camera positions that show an object from different viewpoints. For each pixel it then determines the corresponding scene point's depth (i.e. distance from the camera) by first finding matching pixels (i.e. pixels showing the same scene point) in the other image(s) and then applying triangulation to the found matches to determine their depth. Finding matches in stereo vision is restricted by epipolar geometry: Each pixel's match in another image can only be found on a line called the epipolar line. If two images are coplanar, i.e. they were taken such that the right camera is only offset horizontally compared to the left camera (not being moved towards the object or rotated), then each pixel's epipolar line is horizontal and at the same vertical position as that pixel. However, in general settings (the camera does move towards the object or rotate) the epipolar lines are slanted. Image rectification warps both images such that they appear as if they have been taken with only a horizontal displacement and as a consequence all epipolar lines are horizontal, which slightly simplifies the stereo matching process. Note however, that rectification does not fundamentally change the stereo matching process: It searches on lines, slanted ones before and horizontal ones after rectification. Image rectification is also an equivalent (and more often used) alternative to perfect camera coplanarity. Even with high-precision equipment, image rectification is usually performed because it may be impractical to maintain perfect coplanarity between cameras. Image rectification can only be performed with two images at a time and simultaneous rectification of more than two images is generally impossible.

Transformation

If the images to be rectified are taken from camera pairs without geometric distortion, this calculation can easily be made with a linear transformation. X & Y rotation puts the images on the same plane, scaling makes the image frames be the same size and Z rotation & skew adjustments make the image pixel rows directly line up. The rigid alignment of the cameras needs to be known (by calibration) and the calibration coefficients are used by the transform. In performing the transform, if the cameras themselves are calibrated for internal parameters, an essential matrix provides the relationship between the cameras. The more general case (without camera calibration) is represented by the fundamental matrix. If the fundamental matrix is not known, it is necessary to find preliminary point correspondences between stereo images to facilitate its extraction.

Algorithms There are three main categories for image rectification algorithms: planar rectification, cylindrical rectification and polar rectification.

Implementation details All rectified images satisfy the following two properties:

All epipolar lines are parallel to the horizontal axis. Corresponding points have identical vertical coordinates. In order to transform the original image pair into a rectified image pair, it is necessary to find a projective transformation H. Constraints are placed on H to satisfy the two properties above. For example, constraining the epipolar lines to be parallel with the horizontal axis means that epipoles must be mapped to the infinite point [1,0,0]T in homogeneous coordinates. Even with these constraints, H still has four degrees of freedom. It is also necessary to find a matching H' to rectify the second image of an image pair. Poor choices of H and H' can result in rectified images that are dramatically changed in scale or severely distorted. There are many different strategies for choosing a projective transform H for each image from all possible solutions. One advanced method is minimizing the disparity or least-square difference of corresponding points on the horizontal axis of the rectified image pair. Another method is separating H into a specialized projective transform, similarity transform, and shearing transform to minimize image distortion. One simple method is to rotate both images to look perpendicular to the line joining their collective optical centers, twist the optical axes so the horizontal axis of each image points in the direction of the other image's optical center, and finally scale the smaller image to match for line-to-line correspondence. This process is demonstrated in the following example.

Example

Our model for this example is based on a pair of images that observe a 3D point P, which corresponds to p and p' in the pixel coordinates of each image. O and O' represent the optical centers of each camera, with known camera matrices M = K [ I 0 ] {\displaystyle M=K[I~0]} and M ′ = K ′ [ R T ] {\displaystyle M'=K'[R~T]} (we assume the world origin is at the first camera). We will briefly outline and depict the results for a simple approach to find a H and H' projective transformation that rectify the image pair from the example scene. First, we compute the epipoles, e and e' in each image:

… excerpt ends here. Continue reading the full article.

Illustrations

Image rectification: A camera (red) rotates about the blue axis by 5° to 90° (green), as the images are rectified by projection to the virtual image plane (blue). The virtual plane must be parallel to the stereo baseline (orange) and for visualization is located in the center of rotation. In this case, rectification is achieved by a virtual rotation of the red and green image planes, respectively, to be parallel to the stereo baseline.
A camera (red) rotates about the blue axis by 5° to 90° (green), as the images are rectified by projection to the virtual image plane (blue). The virtual plane must be parallel to the stereo baseline (orange) and for visualization is located in the center of rotation. In this case, rectification is achieved by a virtual rotation of the red and green image planes, respectively, to be parallel to the stereo baseline.
Image rectification: The search for point x L {\displaystyle \mathbf {x} _{L}} 's match is restricted to the line e R x R ¯ {\displaystyle {\overline {\mathbf {e} _{R}\mathbf {x} _{R}}}} in the right image. Since the images are not rectified, the line e R x R ¯ {\displaystyle {\overline {\mathbf {e} _{R}\mathbf {x} _{R}}}} is slanted. After rectification it would be horizontal.
The search for point x L {\displaystyle \mathbf {x} _{L}} 's match is restricted to the line e R x R ¯ {\displaystyle {\overline {\mathbf {e} _{R}\mathbf {x} _{R}}}} in the right image. Since the images are not rectified, the line e R x R ¯ {\displaystyle {\overline {\mathbf {e} _{R}\mathbf {x} _{R}}}} is slanted. After rectification it would be horizontal.
Image rectification: Model used for image rectification example
Model used for image rectification example
Image rectification: 3D view of example scene. The first camera's optical center and image plane are represented by the green circle and square respectively. The second camera has similar red representations.
3D view of example scene. The first camera's optical center and image plane are represented by the green circle and square respectively. The second camera has similar red representations.
Image rectification: Set of 2D images from example. The original images are taken from different perspectives (row 1). Using systematic transformations from the example (rows 2 and 3), we are able to transform both images such that corresponding points are on the same horizontal scan lines (row 4).
Set of 2D images from example. The original images are taken from different perspectives (row 1). Using systematic transformations from the example (rows 2 and 3), we are able to transform both images such that corresponding points are on the same horizontal scan lines (row 4).

Worked examples

Example 1 — a first encounter with Image rectification

Start with the simplest possible case. Write down what Image rectification claims or describes in one sentence, then invent the smallest concrete situation in which that sentence is true. In mathematics, the smallest case is usually a single object, a single equation or a single measurement. Check that every symbol or term in your sentence has a meaning in that case.

Example 2 — changing one variable

Take the situation from Example 1 and change exactly one quantity: double it, halve it, or set it to zero. Predict what should happen to Image rectification before you calculate. Comparing your prediction with the result is the fastest way to find out whether you understand the idea or only the words.

Example 3 — an exam-style question

Typical questions about Image rectification ask you to (a) state it precisely, (b) apply it to given data, and (c) explain a limitation. Practise writing all three answers in under five minutes; the third part is what separates a full-mark answer from an average one.

Applications of Image rectification

In research
Image rectification appears in mathematics research whenever the underlying quantities have to be modelled precisely. Papers usually cite it as a starting assumption and then explore where it breaks down.
In technology and industry
Engineering practice reuses Image rectification in design rules, simulations and safety margins. Knowing the idea lets you read a specification sheet and understand why the numbers look the way they do.
In the classroom
Image rectification is common in secondary-school and first-year university syllabi. It links to neighbouring topics Geographic information systems, Geometry in computer vision, Image processing, so understanding it makes those chapters shorter.
In everyday life
Look for Image rectification outside the textbook — in sport, cooking, traffic, electronics or the sky above you. An example you found yourself is remembered far longer than one you were given.

Affiliate

Preply — study more efficiently by working with a personal tutor. 50% off.

How to study Image rectification in 20 minutes

  1. Read the reference excerpt below once, without taking notes.
  2. Close the page and write down what Image rectification means in your own words.
  3. Compare your version with the excerpt and mark what you missed.
  4. Work through the three examples above with pen and paper.
  5. Explain Image rectification out loud to somebody else — or to Teacher Smith in the lgStudy chat.

Frequently asked questions

What is Image rectification in simple terms?

Image rectification is a transformation process used to project images onto a common image plane. This process has several degrees of freedom and there are many strategies for transforming images to the common plane.

Why does Image rectification matter?

Because it connects several mathematics ideas at once: it gives you a definition you can apply, a quantity you can calculate, and a way to check whether a result is plausible.

How should I study Image rectification?

Read the excerpt, restate it from memory, then work through the examples and applications listed on this page. The five-step study plan above takes about twenty minutes.

What does this page cover?

It gives you a compact reference excerpt plus original lgStudy explanations, examples, applications and study material on Image rectification.

Tags

  • Geographic information systems
  • Geometry in computer vision
  • Image processing

Keep exploring