辅导CSDS 391、辅导Matlab, python编程
CSDS 391 Introduction to Artificial Intelligence
Programming Assignment #2 (“P2”)
Total Points: 135 + 27 extra credit
In this assignment, you will implement two approaches to classification. The first is k-means clustering, the
second is training a linear classifier (a one-layer neural network). In both cases you will use the iris dataset. It
will be convenient to do this assignment in a language with good plotting libraries such as Matlab, python, or
julia. Submit your write-up as a pdf file. Put your code and any related files in a zip folder (please use zip and
not some other compression format). Submit those two files to Canvas so that your write up is visible within
canvas (and not contained within the zipped folder).
Write-up: Your write-up should answer the questions like in the written assignments. When relevant, include
the source code in the write-up if it is sufficiently short (say half a page or less), otherwise refer to the file.
For example, Exercise 3a asks you to write code to compute the mean-squared error, so you would just need to
show the lines for that function. For reference, see §19.6 of the textbook (or §18.6 in the 3rd edition).
Exercise 1. Clustering
a) Write a program that implements the k-means clustering algorithm on the iris data set. You should use 15 P.
the objective function and learning rule you derived in W4 Q1 (and your implementation will help you
uncover any errors, so it is helpful to do this before you turn in W4.)
b) Plot the value of the objective function as a function of the iteration 5 P.
D =
N∑
n=1
K∑
k=1
rn,k∥xn?μk∥2
to show that your learning rule and implementation minimizes this expression.
c) Plot the results of the learning process by showing the initial, intermediate, and converged cluster cen- 5 P.
ters overlaid on the data for k= 2 and k= 3.
d) Devise a method to plot the decision boundaries for this dataset using the optimized parameters. Explain 10 P.
your approach and plot your results.
Exercise 2. Linear decision boundaries
a) Inspect irisdata.csv which is provided with the assignment file on Canvas. Write a program (or use a 10 P.
library) that loads the iris data set and plots the 2nd and 3rd iris classes, similar to the plots shown in
lecture on neural networks (i.e. using the petal width and length dimensions).
b) Define a function that computes the output of simple one-layer neural network using a sigmoid non- 5 P.
linearity (linear classification with sigmoid non-linearity).
c) Write a function that plots the decision boundary for the non-linearity above overlaid on the iris data. 5 P.
Choose a boundary (by setting weight parameters by hand) that roughly separates the two classes. Use
an output of 0 to indicate the 2nd iris class, and 1 to indicate the 3rd.
d) Use a surface plot from a 3D plotting libary (e.g. in matlab or using matplotlib in python) to plot the 5 P.
output of your neural network over the input space. This should be similar to learning curve shown in
fig 19.17 (18.17 in 3rd ed.) of the textbook.
e) Show the output of your simple classifier using examples from the 2nd and 3rd Iris classes. Choose 5 P.
examples that are unambigious as well as those that are near the decision boundary.
Exercise 3. Neural networks
a) Write a program that calculates the mean-squared error of the iris data for the neural network defined 10 P.
above. The function should take three arguments: the data vectors, the parameters defining the neural
network, and the pattern classes.
1
b) Compute the mean squared error for two different settings of the weights (i.e. two dfiferent decision 5 P.
boundaries). Like above, select these by hand and choose settings that give large and small errors
respectively. Plot both boundaries on the dataset as above.
c) Give a mathematical derivation the gradient of the objective function above with respect to the neural 10 P.
network weights. You will have to use chain rule and the derivative of the sigmoid function as discussed
in class. Use w0 to represent the bias term. You should show and explain each step.
d) Show how the gradient can be written in both scalar and vector form. 5 P.
e) Write code that computes the summed gradient for an ensemble of patterns. Illustrate the gradient by 10 P.
showing (i.e. plotting) how the decision boundary changes for a small step.
Exercise 4. Learning a decision boundary through optimization
a) Using your code above, write a program that implements gradient descent to optimize the decision 5 P.
boundary for the iris dataset.
b) In your program, include code that shows the progress in two plots: the first should show the current 5 P.
decision boundary location overlaid on the data; the second should show the learning curve, i.e. a plot of
the objective function as a function of the iteration.
c) Run your code on the iris data set starting from a random setting of the weights. Note: you might need 10 P.
to restrict the degree of randomness so that the initial decision boundary is visible somewhere in the
plot. In your writeup, show the two output plots at the initial, middle, and final locations of the decision
boundary.
d) Explain how you chose the gradient step size. 5 P.
e) Explain how you chose a stopping criterion. 5 P.
Exercise 5. Extra credit: Using a machine learning toolbox
Use a machine learning toolbox such as keras or scikit-learn to generalize the network to perform classification +27 P.
on the full iris dataset. Your network should have one or more hidden layers and classify all three iris classes
using all four data dimensions. Show your results and explain your architecture.
In addition to implementing the neural network classifier, also think of an issue to explore. Write your own
mini-tutorial to report your results. Here are some examples of topics you can consider:
Illustrate how a multi-layer network makes a decision.
Explore different non-linearities.
Explore different network architectures and their classication and generalization performance.
Contrast two different machine learning algorithms (e.g. a neural network vs k-means clustering) and
how they perform on the full iris data set.
Explore algorithms for adjusting the learning rate.
There are many tutorials using different toolboxes available on the internet. You can follow any you find useful
to get started but do not just copy them for this extra credit portion of the assignment. You can make use of
material you find useful, but cite your sources in your writeup. The TAs will check for straight copying, so
please don’t do this.
If you do use material from a source, simplify it to focus on a single topic and understand it well enough that
you can explain it and modify it for our own purposes. Your write up should explain the ideas, code, and results
in your own words and make use of tables or figures.
The extra credit is worth 20% of the points, so it should be about 20% of the effort for the main part of the
assignment. Trivial examples will receive little or no credit. Your goal should be to teach yourself about
something about a neural network-related topic, and the write up should written as if you are explaining it to
one of your classmates.
2