首页 >
> 详细

2020 Computer Vision

Assignment 2: 3D Reconstruction

The process of recovering camera geometry and 3D scene structure is

a significant, perennial problem in computer vision. This assignment will

explore the challenges of recovering 3D geometry when errors are made in

estimating correspondences between two images in a stereo camera rig. The

main aims of this assignment are:

1. to understand the basics of projective geometry, including homoge-

neous co-ordinates and transformations;

2. to gain practical experience with triangulation and understand how

errors in image space manifest in three-dimensional space;

3. to analyse and report your results in a clear and concise manner.

This assignment relates to the following ACS CBOK areas: abstraction, de-

sign, hardware and software, data and information, HCI and programming.

1 Stereo cameras

This assignment will experiment with camera rigs modelled by two identi-

cal cameras that are separated by a Euclidean transformation. Individual

cameras are modelled by a 3 × 4 projection matrix that is parameterised

by internal, or ‘intrinsic’ parameters—such as focal length and principal

point—and external, or ‘extrinsic’ parameters for modelling the camera’s

position in the world. See Lecture 4 and Section 2.1.5 of the textbook for

more information on this.

The intrinsics matrix of the cameras is given by

K =

f 0 00 f 0

0 0 1

(1)

where f is the focal length of the cameras. The two camera projection

matrices are given by P1 = P and P2 = PM, where

P = K[I3×3|0], (2)

1

tr

v

Mv

world co-ordinate frame camera ('eye') co-ordinate frame

f

x

z

f

x

z

Figure 1: The camera extrinsics matrix transforms the scene so that the

camera remains at the origin. Overhead view shown.

and M is a 4 × 4 ‘extrinsics matrix’ that re-orientates the world relative

to the camera. Therefore, it is the inverse of the camera’s transformation

from its coordinate system into the world. This is illustrated in Figure

1 which depicts the same scene in world and eye co-ordinates, where the

transformation from eye co-ordinates into world co-ordinates is given by

M−1.

The geometry of the stereo camera rig demonstrating the projection of a

3D point in both cameras is illustrated in Figure 2, where the left camera is

located at the origin of the world co-ordinate system and the right camera

has been transformed by the relative transformation M−1. Homogeneous

3D points v are projected into both cameras giving pairs of homogeneous

image co-ordinates v′ = Pv and v′′ = PMv. Given the corresponding pro-

jections v′ and v′′ in the two views, the 3D point v can be reconstructed

with triangulation provided that the camera geometry is known. Point tri-

angulation (and, more broadly, ’bundle adjustment’ ) is, itself, a separate

process, but for this assignment you are free to use OpenCV’s or Matlab’s

triangulation function.

2 Task 1: Creating a camera

Your first task will investigate how projection matrices are constructed with

extrinsics matrices to position the camera in the world, and how they are

used to project 3D points onto images. Plotting points with scatter plots

will help visualise the effects of changing the projection matrices, but re-

member that you will first need to convert the points from homogeneous

co-ordinates into Cartesian co-ordinates. These different co-ordinate sys-

tems are described in Lecture 4 and Section 2.1 of the textbook.

2

ff

x

zy

t

v

v''

v'

Figure 2: Stereo camera configuration with the left camera at the world

origin and the right camera separated by an Euclidean transformation. Here,

point vecp is projected to p′ and p′′ in the left and right views respectively.

The source code included in Section 7 is a simple example of this. In

this code, a projection matrix P = [I3×3|0] corresponding to a focal length

of 1, is used to project four homogeneous 3D points with x = ±1, y =

±1, z = 1, w = 1. The resulting image is displayed as a scatter plot. In

this scenario you should expect see the four corners of a square because the

image co-ordinates will be [±11 ,

±1

1 ].

Study and run this code to make sure you understand what it is doing.

Next, modify the code to project the four homogeneous points defined by

x = ±1, y = −1, z = 2± 1, w = 1 and check the result.

1.1 In your report, show and comment on the effect of modifying the

focal length (refer to equation (1)) with both sets of points. How

would you obtain the same effect when operating a real camera?

Next, experiment with camera placement by viewing the projection un-

der a variety of extrinsic transformations: in particular translation and ro-

tation. Note that although mathematically you will always get a projection

of the scene onto the imaging plane, the image will be upside down if the

points are behind the camera (and any points with z = 0 will be at infinity).

You can check for such cases by plotting each point with a different colour.

To get you started with transforming the camera, begin with the trans-

lation matrix

T =

I3×3 t

0> 1

(3)

where t = [−1, 0, 0]> translates the scene 1 unit to the left—and hence

represents a camera one unit to the right of the camera defined only by

P. Plotting the set of vertices V = [±1,−1, 2 ± 1, 1]> should give you the

3

Figure 3: Left view. Figure 4: Right view.

graphs in Figure 3 and 4 for the left and right cameras respectively for a

camera with unit focal length.

You should also experiment with rotating the scene. The rotation matrix

about the y-axis is defined by

Ry =

cos(θ) 0 sin(θ) 0

0 1 0 0

− sin(θ) 0 cos(θ) 0

0 0 0 1

, (4)

and there are similar matrices for rotation about the x and z axes (as well as

rotating about arbitrary vectors). You can also combine chains of transfor-

mations, but note that the order matters. For example, the transformation

M = TRy first rotates the scene about the y-axes before it is translated,

whereas the transformation M = RyT will orbit the camera around the

origin.

When completing this task, remember that the transformation M−1

maps the eye co-ordinate frame, where the optical centre is at the origin, into

world co-ordinates. It may help your understanding to define scenes with

a very familiar structure, such as cubes, spheres and sets of parallel lines.

This will help you understand how the projection matrices are projecting

3D points onto image planes.

1.2 In your report, include multiple transformations, describing their

composition in terms of the type and order of operations, and show-

ing their effect on projection.

The main purpose of this exercise is to understand how M is used to

position the camera. You should experiment with these transformations

until you are comfortable with controlling the camera placement, because

understanding this concept will be invaluable for the next task.

4

xz

point cloud

f

b

Figure 5: Configuration for task 2.

x

z

a

point cloud

d

Figure 6: Cnfiguration for task 3.

3 Task 2: reconstruction from parallel cameras

This task will experiment with triangulating image points with known cam-

era geometry but imperfect point correspondences. Implement a pair of

stereo cameras sharing the same focal length and a fixed distance b (the

‘baseline’) between them. Generate a cloud of points within a sphere in

front of both cameras and project these points into both views. An illustra-

tion of this configuration is shown in Figure 5.

2.1 In your report, describe how you generated the point-cloud and what

considerations you gave to its construction.

Use OpenCV/Matlab’s triangulation function to reconstruct the 3D points

from each pair of corresponding image points and measure the residual (i.e.

3D distance) between the reconstructed 3D points and the ground truth.

This should be 0, since you are simply reversing the process of projection!

Next, add random Gaussian noise to the projected image point locations,

re-triangulate the image points and measure the residual again.

Repeat the experiment by systematically varying the baseline (ie. the

distance between the two cameras), focal length and amount and/or type of

noise. Although calculating the average error over all points may give some

insight into the errors introduced by noise, you are encouraged to find a more

nuanced analysis by visualising the original and reconstructed point-clouds

using a 3D scatter plot. Pay careful attention to where the reconstruction

diverges, and comment in your report on any pattern that emerges from

these experiments, and suggest a reason for these errors.

2.2 In your report, include at least one graph illustrating the relationship

between noise, focal length, and reconstruction error. Include an

explanation that describes how image noise affects triangulation and

how it is related to your results.

5

4 Task 3: reconstruction from converging cameras

Whereas the second task was concerned with two cameras with parallel op-

tical axes, this task will examine how converging axes affects the 3D re-

construction. For a given focal length and convergence angle, position the

second camera so the two cameras’ optical axes converge on a common point.

This configuration is illustrated in Figure 6, where the position of the second

camera is parameterised by the angle a and convergence distance d. Note

that you will need to rotate and translate the second camera. If you have

difficulty defining a suitable transformation, begin by positioning both cam-

eras at the origin and incrementally shift the second camera and observe

how the projection of a known1 3D scene changes.

3.1 Repeat the experiment from Task 2 by measuring the effect of vary-

ing convergence angle on the reconstructed noisy point-cloud projec-

tions. Include a graph of your experiments and compare them to the

results from Task 2. With reference to your discussion on error in

the previous task, suggest an explanation as to why these results are

or are not different from the results you saw in Task 2.

5 Task 4: Estimating pose (post-graduate students

only)

Tasks 2 and 3 assume that the relationship between the cameras are known.

In some applications we need to estimate this relationship from the point cor-

respondences which can be done by decomposing the Essential Matrix. Both

OpenCV and Matlab have methods for estimating and decomposing the Es-

sential Matrix, and we recommend you use the methods that check the ori-

entation of the camera with respect to the triangulated point clouds: the rel-

evant functions are ‘recoverPose’ in OpenCV and ‘relativeCameraPose’

in Matlab. The essential matrix is explained in Lecture 5 and Section 7.2 of

the textbook.

4.1 Repeat Task 3, but use the point correspondences to recover the

pose using the Essential Matrix and known intrinsics, and compare

the results against those collected in Task 3.

1ie. one that you are familiar with, such as a cube

6

6 Report

Hand in your report and supporting code/images via the MyUni page. Up-

load two files:

1. report.pdf, a PDF version of your report; and

2. code.zip a zip file containing your code.

This assignment is worth 40% of your overall mark for the course. It is

due on Monday May 11 at 11.59pm.

7 Task 1.1 Source code

To help you get started with the assignment, we have included the source

code that implements the first part of Task 1.

7.1 OpenCV/Python

import numpy as np

v=np . array ( [ [ 1 , 1 , −1, −1 ] , # x (3D scene po in t s )

[ 1 , −1, 1 , −1 ] , # y

[ 1 , 1 , 1 , 1 ] , # z

[ 1 , 1 , 1 , 1 ] ] ) # w

P=np . array ( [ [ 1 , 0 , 0 , 0 ] ,

[ 0 , 1 , 0 , 0 ] , # } p r o j e c t i o n matrix

[ 0 , 0 , 1 , 0 ] ] )

i=np . matmul (P, v ) ; # p ro j e c t i n to homogeneous co−ord

i=i [ 0 : 2 , : ] / i [ 2 ] # convert to Cartes ian co−ord

f i g=p l t . figure ( )

p l t . s c a t t e r ( i [ 0 , : ] , i [ 1 , : ] , c=’ r ’ , marker=’ s ’ )

p l t . show ( ) # behold : a square !

7.2 Matlab

v=[ 1 , 1 , −1, −1; % x (3D scene po in t s )

1 , −1, 1 , −1 ; % y

1 , 1 , 1 , 1 ; % z

1 , 1 , 1 , 1 ] ; % w

P=[ 1 , 0 , 0 , 0 ;

0 , 1 , 0 , 0 ; % } p ro j e c t i on matrix

0 , 0 , 1 , 0 ] ;

i=P∗v % pro j e c t in to homogeneous co−ord

i=i ( 1 : 2 , : ) . / i ( 3 , : ) % conver t to Cartes ian co−ord

s c a t t e r ( i ( 1 , : ) , i ( 2 , : ) ) % beho ld : a square !

hold on ; % but wait , there ’ s more :

i 2=v ’∗P’ % t h i s i s e qu i v a l en t . . .

i 2=i 2 ( : , 1 : 2 ) . / i 2 ( : , 3 )

s c a t t e r ( i 2 ( : , 1 ) , i 2 ( : , 2 ) )

John Bastian

联系我们

- QQ：99515681
- 邮箱：99515681@qq.com
- 工作时间：8:00-23:00
- 微信：codinghelp2

- Cs2461-10实验程序代做、代写java，C/C++，Python编程设 2021-03-02
- 代写program程序语言、代做python，C++课程程序、代写java编 2021-03-02
- Programming课程代做、代写c++程序语言、Algorithms编程 2021-03-02
- 代写csc1-Ua程序、代做java编程设计、Java实验编程代做 代做留学 2021-03-02
- 代做program编程语言、代写python程序、代做python设计编程 2021-03-02
- 代写data编程设计、代做python语言程序、Python课程编程代写 代 2021-03-02
- Cse 13S程序实验代做、代写c++编程、C/C++程序语言调试 代写留学 2021-03-02
- Mat136h5编程代做、C/C++程序调试、Python，Java编程设计 2021-03-01
- 代写ee425x实验编程、代做python，C++，Java程序设计 帮做c 2021-03-01
- Cscc11程序课程代做、代写python程序设计、Python编程调试 代 2021-03-01
- 代写program编程、Python语言程序调试、Python编程设计代写 2021-03-01
- 代做r语言编程|代做database|代做留学生p... 2021-03-01
- Data Structures代写、代做r编程课程、代做r程序实验 帮做ha 2021-03-01
- 代做data留学生编程、C++，Python语言代写、Java程序代做 代写 2021-03-01
- 代写aps 105编程实验、C/C++程序语言代做 代写r语言程序|代写py 2021-03-01
- Fre6831 Computational Finance 2021-02-28
- Sta141b Assignment 5 Interactive Visu... 2021-02-28
- Eecs2011a-F20 2021-02-28
- Comp-251 Final Asssessment 2021-02-28
- 代写cs1027课程程序、代做java编程语言、代写java留学生编程帮做h 2021-02-28