Machine Learning for Robotics
Assignment 1
ELEC0144 – 2022/2023
Page 2
Guidelines:
All deadlines are specified in Moodle, under the assessment section. Penalties
will be applied for late submissions in accordance with the guidelines:
https://www.ucl.ac.uk/academic-manual/chapters/chapter-4-assessment-
framework-taught-programmes/section-3-module-assessment#3.12
Please also be aware of UCL’s Academic Misconduct policy:
https://www.ucl.ac.uk/academic-manual/chapters/chapter-6-student-casework-
framework/section-9-student-academic-misconduct-procedure. Collaboration
with other teams via exchange of ideas, sharing of codes, re-using portions of the
reports etc. are not allowed and will be considered as collusion.
ELEC0144 – 2022/2023
Page 3
1 Assignment 1: Camera and Robot Calibration
1.1 Objective Summary
In this assignment, you are required to write Matlab codes for camera calibration, robot
calibration, image processing, and relating 2D pixel position to 3D position.
1.2 Task 1: Camera Calibration Given Simulated Checkerboard Data
In this task, you will write a code to calibrate a camera given some checkerboard data.
The checkerboard pattern was downloaded from Matlab (“checkerboardPattern.pdf”),
and was then printed on an A4 paper. The size of the checkerboard square was
measured and determined to be 22mm (Figure 1). This is an important information as it
will provide the position of the checkerboard points in World frame – The points would
be automatically [0, 0], [22, 0], [44, 0], …, [0, 22], [22, 22], [44, 22], so on and so forth.
Figure 1: Size of checkerboard square [Matlab2020A]
Six images of the checkerboard are then taken using the camera, with the checkerboard
held at six different positions and orientations. The files are “Image1.png” to
“Image6.png”, and are also shown here for your reference (Figure 2).
ELEC0144 – 2022/2023
Page 4
Figure 2: Examples of images taken by webcam.
Next, a code is written to extract the data points (Code 1). Remember, we need two
different types of data points: 1) the corner points in World frame for a single printout,
and 2) the corner points in pixel frame for 6 images.
Code 1: Code for data point extraction.
clear all
close all
clc
% General Settings
numImages = 6; % depends on the number of images you have
squareSize = 22; % in millimeters
% Read files
files = cell(1, numImages);
for i = 1:numImages
files{i} = fullfile(sprintf('Image%d.png', i));
end
% Display the image at working position
I = imread(files{1}); % assume image1 is the working position
figure; imshow(I);
title('Image at Working Position');
% Detect the checkerboard corners in the images.
[imagePoints, boardSize] = detectCheckerboardPoints(files);
% Generate the world coordinates of the checkerboard corners in the
% pattern-centric coordinate system, with the upper-left corner at
(0,0).
worldPoints = generateCheckerboardPoints(boardSize, squareSize);
ELEC0144 – 2022/2023
Page 5
If you run the code, the imagePoints and worldPoints will be generated in the Matlab
workspace. They will be in the form of [x, y] coordinates.
Your task now: Using the extracted data, write a code to calibrate the camera. You
should provide one set of intrinsic parameters as a K-matrix, as well as 6 sets of extrinsic
parameters which should include position and orientation.
Further task: Next, add the following code (Code 2) to your code. The code performs
camera calibration using Matlab’s toolbox.
Code 2: Matlab’s Camera Calibration Code
% Calibrate the camera using Matlab’s toolbox.
imageSize = [size(I, 1), size(I, 2)];
cameraParams = estimateCameraParameters(imagePoints, worldPoints, ...
'ImageSize', imageSize);
Run the code and see the results of “cameraParams” in Matlab workspace, then
compare and discuss the differences between your own results and the toolbox’s
results. Why are there differences? You might want to refer to Zhengyou Zhang’s paper
(Zhang2000) for some details.
Task 1: Write your own code to obtain the intrinsic and extrinsic parameters of the
camera, given checkerboard dataset. Then compare your results with Matlab
toolbox’s results.
What to write in your report:
Details of the algorithm, derivation, results, discussions and comparisons.
Additional files to submit:
Your Matlab code (m file) named “Vision_Task1.m”, which will be test run by
your module leader / tutors. The code should be properly commented.
Additionally: Save your intrinsic parameter matrix as “K.mat”, and the extrinsic
parameters of the first image, [1 2 3
] as “Tcw.mat” (cw stands for
camera-to-world), which you will need to use for Task 4 later.
ELEC0144 – 2022/2023
Page 6
1.3 Task 2: Robot Calibration Given Simulated TCP Data
In this task, you will write a code to calibrate the robot, i.e. to find the position and
orientation of the world frame with respect to the robot.
Six points on the checkerboard have been selected. Their positions according to the
World coordinate system (on the checkerboard itself) are:
Point 1 2 3 4 5 6
-22 0 22 44 66 154
-22 22 66 22 88 -22
The same six points have been “probed” by the robot using a sharp-tipped tool. The
positions according to the Robot coordinate system are:
Point 1 2 3 4 5 6
-121.6 -100.4 -79.16 -56.39 -35.55 54.32
222.4 178 133.6 177.2 110.9 219.3
-20 -20.04 -20.08 -19.96 -20.04 -19.7
Task 2: Write your own code to find the position and orientation of the World frame
with respect to the robot frame, given the positions of the checkerboard corner
points.
What to write in your report:
? Details of the algorithm, derivation, results and discussions.
Additional files to submit:
? Your Matlab code (m file) named “Vision_Task2.m”, which will be test run by
your module leader / tutors. The code should be properly commented.
Additionally: Save your result of position and orientation of world frame with respect to
robot frame as a single homogeneous transformation matrix named “Trw.mat” (rw
stands for robot-to-world), which you will need to use for Task 4 later.
ELEC0144 – 2022/2023
Page 7
1.4 Task 3: Image Processing given Simulated Image
In the “Task3Data” folder, you will find an image of “rectangular” object named
“Object.png”. It is of size 640x480 (Figure 3).
Figure 3: Camera view
Your task is to use image processing methods discussed in the lecture to determine the
position, orientation, circularity, edges and corners of the object.
Task 3: Write your own code to find the position, orientation, circularity, edges and
corners of the object.
What to write in your report:
? Details of the algorithm, derivation, results and discussions.
Additional files to submit:
Your Matlab code (m file) named “Vision_Task3.m”, which will be test run by
your module leader / tutors. The code should be properly commented.
Additionally: Save your result of x and y positions of the object, in the form of
[ 0], as “PObject.mat”, which you will need to use for Task 4 later.
ELEC0144 – 2022/2023
Page 8
1.5 Task 4: Relating Pixel Position to Robot Position using Simulated
Data
Before you start with task 4, please ensure that you have the following mat files from
earlier tasks:
K.mat from Task 1
Tcw.mat from Task 1
Trw.mat from Task 2
PObject.mat from Task 3
The data in PObject.mat are 2D positions of the object with respect to the camera pixel
frame. Now, use K.mat, Tcw.mat and Trw.mat to find the actual 3D positions of the
object with respect to the robot frame.
Task 4: Find the 3D positions of object with respect to robot frame.
What to write in your report:
Details of the algorithm, derivation, results and discussions.
Additional files to submit:
Your Matlab code (m file) named “Vision_Task4.m”, which will be test run by
your module leader / tutors. The code should be properly commented.
K.mat, Tcw.mat, Trw.mat, PObject.mat
ELEC0144 – 2022/2023
Page 9
1.6 What to Submit
Your Matlab codes, with proper comments. The code will be tested!
A written report which details the implementation of your calibration algorithm
and image processing, results, comparisons, discussions etc.
You should put everything (Matlab codes AND your written report) into a zip
folder, then submit the zip folder onto the submission point on Moodle. Note:
please do not submit .rar file – only .zip is allowed.
1.7 More about the Report
The report should have a cover page clearly indicating the following details:
Report title.
Team/Student name.
Full name, student number and email address for each team member.
Submission date.
The body of the report must be organized under the following section headings:
Executive summary
Task 1: Camera Calibration Given Simulated Checkerboard Data
Task 2: Robot Calibration Given Simulated TCP Data
Task 3: Image Processing given Simulated Image
Task 4: Relating Pixel Position to Robot Position using Simulated Data
Conclusion
The list of references should appear on separate pages. References should be formatted
using the IEEE Citation Style. It is extremely important that all third party sources of
information are properly credited and referenced in the correct manner. The inclusion
of any text or diagrams from websites or documents must be clearly indicated and
referenced.
Font size should be exactly 11 point. Recommended font type is Calibri or Arial. Text
should be both left and right aligned (justified text). All figures should have captions, axes
labels and legends where appropriate. Curves should be distinguishable even if printed in
black and white.
ELEC0144 – 2022/2023
Page 10
1.8 Marking Criteria
This assignment contributes 30% to the overall score of the module. The marking
criteria are described in the following table:
Criteria Mark
Weight
Task 1 code Code works properly, with good comments. 6%
Task 2 code Code works properly, with good comments. 4%
Task 3 code Code works properly, with good comments. 5%
Task 4 code Code works properly, with good comments. 3%
Report (Details) Details of the algorithm, derivation, discussions and
comparisons clear and comprehensive.
9%
Report (Format) English syntax and style, general organization and
formatting, figure, table and equation presentation
and use, literature citations are use all appropriate.