|
|
|
|
|
The main task in stereo reconstruction is the matching of points in one image to the corresponding points in the other image. Normally this is done via correlation using epipolar geometry and other matching constraints to speed the process and maximise the reliability of matching.
For simplicity in this lab exercise we will do the matching of points manually.
Write a MATLAB function to the following specification (see the Note given at the end of the page):
% STEREO % % Usage: pt3D = stereo(im1, im2, C1, C2) % % Where: im1 and im2 are the two stereo images % C1 and C2 are the calibration matrices % for these two images respectively % % The function will prompt you to digitise some points in the first image % (finishing by clicking the right button). The function then % prompts you to digitise the equivalent points (which you must digitise % in exactly the same sequence) in the second image. % The function then solves the stereo equations % and returns the 3D coordinates of the points in pt3D. function pt3D = stereo(im1, im2, C1, C2)
You can use the disp or fprintf functions to print out messages on the screen and the digipts function to do the digitising. For example:
>> disp('Digitise some points in figure 1');
>> figure(1), imshow(im1);
>> [u1,v1] = digipts;
Note that digipts behaves similarly to MATLAB's
impixel but uses the cross-hair cursor from ginput
which I find easier to use.
If you have an N × 3 array of 3D points
p3D = [. . .
. . .
. . .
. . .];
and you want to draw a poly-line that, say, passes through points 2, 3, 5,
4, and 1 in that order from the p3D array you can do the following:
% Extract the points you want pts = [p3D(2,:); p3D(3,:); p3D(5,:); p3D(4,:); p3D(1,:)]; % Alternatively (note that it is optional to separate the numbers % inside the square brackets by commas), pts = p3D([2 3 5 4 1],:); % Extract X, Y and Z components and pass to the Matlab's line function: line(pts(:,1), pts(:,2), pts(:,3)); % To draw a closed polygon through the points 2, 3, 4, 1: indices = [2 3 4 1 2]; % repeat the 1st point to close the polygon line(p3D(indices,1), p3D(indices,2), p3D(indices,3));
When plotting your 3D reconstruction of the stereo view of the scene, use the following
>> axis equal % Keeps the aspect ratio of the objects regular.
>> box on % Overlays a box which helps in the 3D visualisation.
>> rotate3D on % Drag the mouse in the image to rotate the view.
>> grid on % You may find this useful too.
An example of the 3D reconstruction of a stereo pair
from a previous year's lab sheet (not
this year's)
is given below:
Write a MATLAB function to the following specification:
% HOMOGRAPHY % % Usage: H = homography(pts1, pts2) % % Where: pts1 and pts2 are both N x 2 matrices % containing the set of points related by % a plane-to-plane homography transformation. % H should be the 3 x 3 matrix that describes % the homography. function H = homography(pts1, pts2)The minimum number of pairs of points required for homography is 4. Your Matlab code should check whether this condition is satisfied and output an appropriate error message accordingly.
Given in posterA4.jpg
is an A4-sized poster. The physical size of a A4-page in portrait mode
is 297 mm (height) × 210 mm (width). Your task for this part of
the lab sheet is to select the four corner points of the poster in the
image (using the function digipts mentioned above) and create an image
that contains a perfect frontal view of the poster. Since we know the physical
size of the poster, the size of the output image should be appropriate
determined so that the aspect ratio of the poster is retained. You are
suggested to create an image of 891 (rows) × 630 (columns)
pixels to store the rectified version of the poster. Thus, the four
image corner points of the poster that you collect from digipts
should be mapped by the 3 × 3 homography H from your
homography function to the following (x,y)-coordinates
(the origin is at the top-left corner):
[1,1] (top-left),
[630,1] (top-right),
[630,891] (bottom-right), and
[1,891] (bottom-left).
When you display your output image for inspection in Matlab, do not forget to specify axis equal
to ensure the aspect ratio of the picture is retained.
Your output image should contain the rectified poster
and nothing else.
You would probably notice that edges of the picture frame are not straight. This is due to the distortion of the camera lens. This distortion can be corrected by a pre-processing step. In this lab sheet, we temporarily ignore lens distortion.
Note that when a point (x,y,1)T is mapped to a new point (x',y',1)T by H, both x' and y' are usually not whole numbers. To get the colour of a pixel at non-integer pixel coordinates, you will need to adopt a bilinear or bicubic interpolation on the four surrounding pixels. You will find the Matlab function maketform and imtransform useful for this exercise. An example about using these two functions can be found in the lecture note.
Your Matlab functions/scripts will be run and tested. So, do not forget to include all other Matlab functions (excluding the Matlab library functions but including Dr Peter Kovesi's) that are needed by your Matlab functions/scripts.
You might find it more tidy to save your MATLAB code and input/output test image files into subdirectories away from your HTML file. Ensure that all the links in your HTML file are correct. You should also provide some explanation in your HTML file to demonstrate your understanding about the problem and the work that you carried out.
Note: If you have the calibration matrices
computed for more than 2 images from the supplied set and prefer
to use them for the 3D reconstruction in this lab sheet, then your
function heading should be:
function pt3D = stereo(im1, im2, im3, C1, C2, C3)
if you use 3 images, and
function pt3D = stereo(im1, im2, im3, im4, C1, C2, C3, C4)
if you use 4 images. You will need to manually click all the matching points
in each image.
