ELEC ENG 4061/7060 Image Processing Page 1 of 10
7060 Image Sensors Processing / 4061 Image Processing
Assignment 4
May 2020
(Assignment Value: 18%)
(note: there are no written questions for this assignment)
Overview
This assignment is intended to give you some hands-on experience with thresholding,
morphological cleaning, feature estimation and classification as covered in lectures. In this
assignment work you will be required to modify several functions which implement thresholding,
clean up and feature estimation stages of a classification system.
NOTE: Unlike previous assignments you may use any of the functions in the MATLAB image
processing toolbox except for graythresh (unless otherwise instructed by me – see comments in
red in exercise 4B) to achieve this.
Background
A manufacturer of different types of steel bolts, washers and attachment hooks has a approached
your company to develop an automated sorting system for their production lines to deal with the
large number of items that are currently manually sorted as they travel along a conveyor belt. The
intent is to develop a camera based classification system which can identify the metal parts.
However due to conditions on the production line space and lighting is limited and only small
noisy imagery of the parts can be obtained.
As your company’s key trouble-shooter, you have been assigned the task of developing a simple
demonstrator to show that such a system might work. To this end you have been given some
example imagery of some 6 metallic parts of the many dozen which they sell. Example images are
shown on the next page.
To solve this problem, you will need to firstly threshold and filter out the region containing the
metal parts and then implement a feature extraction step stage that extracts shape estimates
from the data. The thresholding will be performed using Otsu’s method and an Euler number
feature will be used to discriminate classes 2,4 and 6 from 1,3 and 5. It is up to you which of the
other simple shape measures covered in class you might use to separate out the other shapes.
Your resulting shape measures will then be fed into a KNN classifier which you have been given
to test if the approach works.
ELEC ENG 4061/7060 Image Processing Page 2 of 10
Class 1 Class 2
Class 3 Class 4
Class 5 Class 6
Above: example images of the 6 metallic parts (1 Bolt, 2 Eye Bolt, 3 Hook Bolt, 4 Carabiner, 5 U-
bolt and 6 Washer). Note the noise present in the captured images. The images in the dataset are
identified by the file prefixes bolt, eyeb, hook, loop, ubol and wash respectively.
Assignment Submission
Assignments are to be submitted via MyUni as a ZIP archive file containing the four key MATLAB
functions ( otsu_threshold.m, simple_euler.m, object_extract.m and feature_extract.m plus
any others you modify) and a Word or PDF document summarising your results with comments
on the performance of the processing steps employed.
Marks will be deducted for late submission.
IMPORTANT: DO NOT INCLUDE THE SUPPLIED IMAGE DATABASE IN YOUR ASSIGNMENT
SUBMISSION. I WILL DEDUCT MARKS IF YOU DO !!!
Source Materials
Source code, test functions and example imagery for this assignment are located on the MyUni
website (https://www.myuni.adelaide.edu.au).
The image data is contained in the noisy_data subdirectory and includes some 150 images per
object type (900 in all) which should give you plenty of examples to work with.
ELEC ENG 4061/7060 Image Processing Page 3 of 10
Exercise 4A (5%) – Image Thresholding (Otsu’s Method)
Modify the supplied function [B,th]=otsu_threshold(I) such that it thresholds an image using
Otsu’s method as described below. Here I is the input image, B is the binary image output and th
is the estimated threshold. An optional argument N can also be supplied which defines the
number of thresholds tested by the algorithm (see below).
Otsu’s thresholding method is a form of automatic class clustering based on maximising the
variance between the greylevel of pixels in the thresholded and un-thresholded regions of the
image whilst minimising the grey-level variance between pixels within each region.
That is, for all possible values of t, we week a threshold t that minimises:
)()()()( ttNttN oobb +
Where
)(tN b number of pixels below the threshold t
)(tb is the standard deviation of the values below t
)(tN o number of pixels above (or equal to) the threshold t
)(to is the standard deviation of the values above (or equal to) t
However, the solution for this turns out to be the same as maximising for the between class
variance. This can be easily calculated as follows:
For each potential threshold T (between the max and min values of the image)
1. Separate the pixels into two groups according to the current threshold T.
2. Find the mean of each group )(tb and )(to .
3. Square the difference between the two means ( )2)()( tt ob − .
4. Multiply this value by the number of pixels in one cluster )(tN b times the number
in the other )(tNo .
5. Record the result and the threshold T used
The best threshold is the value T associated with the largest calculated result.
Modify the function [B,th] = otsu_threshold(I) such that it implements the algorithm given above.
Test your algorithm on some example images including those from the test database and the
supplied function otsu_test.m
IMPORTANT: Do not use ‘greythresh’ or any other thresholding technique for this part of the
assignment or you will receive zero marks for part 4A.
ELEC ENG 4061/7060 Image Processing Page 4 of 10
Above: the Otsu algorithm given earlier applied to two test images (see otsu_test.m).
Exercise 4B (3%) – Region (Object) Extraction
Modify the supplied function B = object_extract(I) such that it thresholds the supplied image I
using Otsu’s method and returns a "cleaned up" binary image B representing the metal part.
The function should remove artefacts from the thresholded imagery such as point noise and
merge fragmented regions together to form a good representation of the irrigation object. You
will need to use morphological filters to achieve this.
Important Note: One problem you may face is in dealing with the excessive noise in the raw
imagery and this may adversely affect the threshold result. You may wish to reduce this noise
before considering the threshold step. So in effect the steps are:
1. Reduce noise
2. Threshold
3. Cleanup using morphological filtering (eg. erode/dilate/open/close etc)
The resulting binary image should ideally contain one connected region representing the plastic
part to make it easy for your feature extraction processing.
An example of this processing is given below (note: your own solutions may not look exactly like
this):
ELEC ENG 4061/7060 Image Processing Page 5 of 10
Illustrate your processing results using examples from the image datasets and write this up in your
assignment report. Take special note of any situations where the thresholding may be
problematic.
You can use the supplied test script object_extract_all to check your thresholding on the 6 sets
of test images (but be warned it will take some time to go through all 900 images).
Comments: The quality of the thresholding and cleanup stages will have an impact on any features
you estimate from the regions and hence the quality of the final classification performances in
exercises 4D. You may need to revisit this step once you have the classifier working to do some fine
tuning.
SPECIAL NOTE: If you are unable to get the Ostu thresholder going in part 4A you can temporarily
use the function greythresh() to obtain a threshold. If you do so please make it very clear in your
code and write up that you have done so otherwise marks will be deducted.
Exercise 4C (3.5%) – Feature Estimation (Euler Number)
As described in lectures the Euler number of a binary image is the number of binary regions minus
the number of holes.
There are several ways of estimating the Euler number for a binary image, but the easiest is to
estimate the number of convex and concave corner-like regions in the binary image by performing
a series of tests on 2x2 neighbourhoods around each pixel in the image. Geometrically, each
region or hole will have some factor of 4 of these.
This simple algorithm for 4-connected regions is as follows:
1. Set convex count and concave count to zero.
ELEC ENG 4061/7060 Image Processing Page 6 of 10
2. For each pixel (i,j) examine the local 2x2 patch of image data
a. If the 2x2 patch contains 1 set pixel increment the convex count by 1.
b. If the 2x2 patch contains 3 set pixels increment the concave count by 1.
c. If the 2x2 patch contains 2 set pixels diagonally opposite one another (ie [10;01]
or [01;10]) then increment the convex count by 2.
3. The final Euler number is the convex count minus the concave count divided by 4.
To loosely explain how this works. Imagine a simple box shape (Euler number = 1). The above
algorithm will only increment the convex count at each corner of the box giving a total value of 4
convex to 0 concave → (4-0)/4 = 1. Next imagine this same box now has a square hole in it (ie.
Euler=0). In this case the above algorithm will give a convex count of 4 and also a convex count of
4 at the 4 corners of the hole → (4-4)/4 =0. If instead there were 2 square holes rather than two
then the convex count would be 8 leading to an Euler number estimate of -1. Geometrically
speaking this reasoning can be extended for any number or shape of hole or object. In each case
the differences in count will always come down to a factor of 4, the only exception being when
the object is touching the image boundary. To handle this case a 1 pixel wide empty border is
usually added to the image before applying the above algorithm. Case (c) is included above to
improve the estimate in the presence of small adjacent holes etc.
Modify the supplied file simple_euler.m such that it implements the algorithm shown above.
Make sure you also add the 1 pixel boundary mentioned above before doing the calculation. Use
the script euler_test.m to check you code and write up your results.
Above: Example Euler number calculations (see euler_test).
Note: you do not need this solution to complete Exercise 4E. If you wish to use the Euler number
for exercise 4E then you may use regionprops() to get an estimate.
Euler Number 1 (est 1)
20 40 60
20
40
60
Euler Number 0 (est 0)
20 40 60
20
40
60
Euler Number 3 (est 3)
20 40 60
20
40
60
Euler Number 5 (est 5)
20 40 60 80 100 120
20
40
60
80
100
120
ELEC ENG 4061/7060 Image Processing Page 7 of 10
Exercise 4D (3.5%) – Feature Estimation (Eccentricity)
The eccentricity of an object can be used to distinguish between rounded and elongated objects.
There are several ways of computing such a measure. Here we will use an approximation based
on the central image moments discussed in class. The measure for eccentricity is as follows:
=
(2,0 − 0,2)
2
+ 41,1
2
(2,0 + 0,2)
2
From the lecture notes recall that the central moments are defined as follows:
Where xc and yc are the centres of mass defined as:
Modify the supplied function simple_eccentricity.m to implement this estimate of eccentricity
and test your results using the script eccentricity_test.m and write up your results. The supplied
code includes a call to meshgrid() to supply you with the x and y coordinates. An example output
is shown below:
Above: Example eccentricity calculations (see eccentricity_test).
NOTE: this estimate does not match the one used in regionprops() so don’t try comparing the two.
Note: you do not need this solution to complete Exercise 4E. If you wish to use eccentricity for
exercise 4E then you may use regionprops() to get an estimate.
ELEC ENG 4061/7060 Image Processing Page 8 of 10
Exercise 4E (3%) – Shape Measure Classification
As already noted the Euler number could be used as a simple shape measure to distinguish classes
2,4 and 6 from classes 1,3 and 5. However to distinguish between all 5 classes we will need to
make use of some of the other shape measures described in class.
Modify the function F=feature_extract(B) such that when it is given the cleaned up binary image
B it returns a row vector of shape measurements of your choice (which may or may not include
the Euler number).
To assist you many of the shape simple measures described in class can be constructed from
values returned by the MATLAB function regionprops() which works with binary images. Your task
is to figure out which ones can be used to successfully classify the 6 shapes.
The function regionprops takes a binary region B and returns a record with fields including such
measurements as Area, MajorAxis, MinorAxis EulerNumber etc. However, be warned that if your
object is fragmented into several regions the function will return estimates for each fragment
rather than a single result (make sure your solution for part 4B does the right thing).
Suggested measurements from the notes that you might like to try include:
• Euler number
• Minor axis / major axis
• Perimeter / square root of the area
• Area / convex area
Note your estimates should be, if possible, invariant to translation scale and rotation.
To test if your proposed shape measures might work, use the supplied function:
[F,C] = feature_extract_all();
If this runs to completion without producing an error (which it should unless you have a bug in
your feature_extract.m file) then the values F and C will contain the feature measurements and
class numbers for each of the 900 images in the dataset.
The function cplot(F,C) can then be used to generate scatterplots of the features you have
created. You can use tricks like cplot(F(:,4:5),C) to just plot features 4 and 5 of your data.
ELEC ENG 4061/7060 Image Processing Page 9 of 10
Above: illustration of feature clustering based on two possible shape measures (created using
cplot.m). Your own feature plots will differ from the example shown here.
Once you are satisfied that your feature measurements are working well enough you can use the
supplied KNN classifier script as a final test:
[C_est,C_true] = knn_classify_all(F,C,k);
Where F and C are previously calculated features and class numbers and ‘k’ is the number of
nearest neighbours to test to see if your solution works (eg. 3). C_est and C_true are the estimates
and true class labels based on a 30% train 70% test split of the data. An example screen shot of
the run-time plot and resulting confusion summary generated by this script are shown below.
ELEC ENG 4061/7060 Image Processing Page 10 of 10
Above: sample screenshots of knn_classify_all.m. Your own results and feature plots will differ
from what is shown here. Ideally the higher the numbers along the diagonal in the confusion
matrix the better.
Unlike most real-world problems you are ever likely to encounter, this dataset is relatively easy
to classify. If you get this working correctly, you should be able to easily achieve over 90% correct
classification (and probably into the high nineties).
Important: Plot histograms and/or scatterplots of the feature values for each class and record
your classification scores and put these in your report. Try different combinations of feature
measures and see what happens. What did you find were the best combinations?