首页 >
> 详细

COMP9417 19T3 Homework 2

Applying and Implementing Machine Learning

Introduction

The aim of this homework is to enable you to:

1. apply parameter search for machine learning algorithms implemented in the Python

sklearn machine learning library

2. answer questions based on your analysis and interpretation of the empirical results

of such applications, using your knowledge of machine learning

3. complete an implementation of a different version of a learning algorithm you have

previously seen

After completing this homework, you will be able to:

• set up a simple grid search over different hyper-parameter settings based on k-fold

cross-validation to obtain performance measures on different datasets

• compare the performance measures of different algorithm settings

• propose properties of algorithms and their hyper-parameters, or datasets, which may

lead to performance differences being observed

• suggest reasons for actual observed performance differences in terms of properties of

algorithms, parameter settings or datasets.

• read and understand incomplete code for a learning algorithm to the point of being able

to complete the implementation and run it successfully on a dataset.

There is a total of 5 marks available.

Deadline: Friday 1 November 2019, 5 PM

Submission will be via the Moodle page.

Late penalties: one mark will be deducted from the total for each day late, up to a total of five

days. If six or more days late, no marks will be given.

Recall the guidance regarding plagiarism in the course introduction: this applies to this

homework and if evidence of plagiarism is detected it may result in penalties ranging from loss

of marks to suspension.

There are two parts of questions in this homework, for the first part you should run a python

code using provided dataset and answer some questions related to that. For second question,

you should build k-nearest neighbour model using Australian credit card dataset.

Question 1

In this question, you should run the ‘comp9417_hw2.ipynb’ and the provided results will help

you to find the answer of the following questions.

Dataset

You can download the datasets required for the homework in the datasets folder. Note: you

will need to ensure the dataset files are in the same directory from which you are running this

notebook.

Package installation

Please Note: this homework uses some datasets in the Attribute-Relation File Format (.arff).

To load datasets from '.arff' formatted files, you will need to have installed the scipy.io package.

You can do this using pip at the command-line with the following code, if this package is not

installed yet.

pip install scipy

Part A - [0.5 marks]

To answer this question, you should run the python code in the notebook

‘comp9417_hw2.ipynb’. you will have a table similar to the bellow table which you should

copy and paste in your report as your answer for "Question Part A".

Copy/paste the output from running the code in the notebook.

DO NOT EDIT the table!!!

Part B - [0.5 marks]

In sklearn’s decision tree implementation, the parameter “max_depth” represents the maximum

depth of decision tree during the training procedure. The deeper the tree, the more splits it has,

and it captures more information about the data.

By increasing the value of the value of “max_depth” parameter we can expect this to:

--- (1) overfitting not changed by decreasing max_depth of the decision tree

--- (2) decrease overfitting by increasing max_depth of the decision tree

--- (3) increase overfitting by decreasing max_depth of the decision tree

--- (4) increase overfitting by increasing max_depth of the decision tree

Part C - [0.5 mark]

Looking at your table, the performance result for datasets with 50% noise and “max_depth”

parameter. Does finding the best parameter in grid search helps the decision tree model to

improve the test set accuracy compared to the default parameter settings? What is your answer?

--- (1) no

--- (2) yes, for 1/4 of the datasets

--- (3) yes, for 2/4 of the datasets

--- (4) yes, for 3/4 of the datasets

--- (5) yes, for 4/4 of the datasets

Question 2

In this practical question, you should develop a machine learning model using K-nearest

neighbours to predict which customer will be able to return the loan without any delinquency.

In order to build your model, you should use CreditCard.csv file in Datasets folder. Then you

should find the answer of the following questions:

Pre-processing

In the first step of this homework, you can start by applying min-max normalisation to all features (X1

– X14). You can test whether you did the normalisation correctly or not by checking the minimum and

maximum value for each of your features.

Min-Max normalization

� − min (�) �"#$ = max(�) − min (�)

After applying this normalisation, the minimum value of your feature will be 0 and the maximum value

will be 1.

Creating test and training sets

In this step, you have to create training and test sets. Please use the first 621 rows of the data

as training set and keep the 69 remaining one (from 622 to 690) as test set which we will use

later to evaluate the kNN model.

!!!!Please copy and paste all your codes in your submission file!!!!

Part A - [1 mark]

Implement a kNN classifier for Australian credit risk prediction using sklearn library. You

should set the n_neighbors =2 for training the model. What is your accuracy score for training

and test dataset?

Part B - [1 mark]

Find optimal number of neighbours by developing a search algorithm to find the optimal value

of k. You should find the optimal number of k in a range between 1 to 30 and finding optimal

value for number of k. please use AUC score to find the optimal number of neighbours.

Part C - [0.5 mark]

Plot the AUC score for all iterations (k: 1,…,30) in training and test sets. (one plot for training,

and one for test set).

Part D - [1 mark]

Compute precision and recall evaluation metrics for your kNN model with optimal number of

neighbours and another model that you have built in part A. Compare these metrics for these

two models.

联系我们

- QQ：99515681
- 邮箱：99515681@qq.com
- 工作时间：8:00-23:00
- 微信：codinghelp2

- Program课程作业代做、Python语言作业代写、Python编程作业调 2020-06-01
- 代做stats762作业、代写r编程设计作业、R课程设计作业代做、代写dat 2020-06-01
- Cosc 2666作业代写、C++程序设计作业调试、Programming作 2020-06-01
- Stats 731作业代写、代做c+,Java程序语言作业、代写python 2020-06-01
- Etf5952课程作业代写、Risk Analysis作业代做、Python 2020-06-01
- 41889留学生作业代做、代写ios Application作业、Java， 2020-06-01
- Sta2202作业代做、代写data课程作业、R程序语言作业调试、代写r课程 2020-06-01
- You Have Implemented A Simple Web Serv... 2020-05-31
- Stat 5511 Homework 4 2020-05-31
- Lab 7 2020-05-31
- 代写cosc 363作业、代做computer Graphics作业、代写c 2020-05-31
- Eie111课程作业代写、C++程序设计作业调试、代做c/C++语言作业、代 2020-05-31
- Math502作业代做、Mathematics课程作业代做、Java，Pyt 2020-05-31
- 代写sit114课程作业、代做data留学生作业、代写r编程设计作业、代做r 2020-05-31
- Envx3002作业代写、R程序语言作业调试、R课程设计作业代做、代写dat 2020-05-31
- Ee6435 Programming Homework 2020-05-30
- Computer Architecture Homework 3 2020-05-30
- Infs7450作业代做、Media Analytics作业代写、Pytho 2020-05-29
- 代写stats 782作业、代做r编程设计作业、代写data留学生作业、R课 2020-05-29
- 代写math223作业、R课程设计作业代做、代写data课程作业、R程序语言 2020-05-28