STATS 3DA3
Homework Assignment 6
Submission Deadline
• All submissions must be made before 10:00 PM on Wednesday, April 16, 2025.
Late Submissions
• Late Submission Penalty: A 15% deduction per day will be applied to assignments submitted after the deadline, including SAS accomdations.
• Late Submission Limit: Assignments submitted more than 72 hours late will receive a grade of zero, including SAS accomdations.
Submission Guidelines
• Format: Submit your work as a single PDF file via Avenue to Learn. You may submit individually or or as a group of up to three members.
Individual Submission
• Complete Questions 1—15.
• A GitHub repository is optional.
Group Submission
• Complete Questions 1–17.
• Group Size: Up to three members.
• Team Members’Contributions: For group submissions, you must complete Question 16, de- tailing each member’s contributions. This should correspond with the commit history in the GitHub repository.
- Note: While Question 16 is not graded, failure to include this information will result in the assignment not being graded.
- Example:
∗ Member A: Questions 1, 2, 4
∗ Member B: Questions 3, 5, 6
• GitHub Repository: You must include a link to a public GitHub repository showing the assignment’s version history.
- Note: While Question 17 is not graded, failure to provide this information will result in the assignment not being graded.
Assignment Standards
Please ensure your assignment adheres to the following standards for submission:
• Title Page Requirements: Each submission must include a title page featuring your group members’names and student IDs. Assignments without a title page will not be considered for grading.
• Formatting Preferences: The use of Quarto Jupyter Notebook for document preparation is highly recommended.
• Font and Spacing: Submissions must utilize an eleven-point font (Times New Roman or a similar font) with 1.5 line spacing. Ensure margins of at least 1 inch on all sides.
• Individual Work: While discussing homework problems with peers and other groups is per- mitted, the final written submission must be your group work.
• Submission Content: Do not include the assignment questions within your PDF. Instead, clearly mark each response with the corresponding question number. Screenshots are not an acceptable form of submission under any circumstances.
• Academic Writing: Ensure that your writing and any references used are appropriate for an undergraduate level of study.
• Originality Checks: Be aware that the instructor may use various tools, including online resources or in-person meetings, to verify the originality of submitted work.
Assignment Policy on the Use of Generative AI
• The use of Generative AI is not permitted in assignments, except for using GitHub Copilot as a coding assistant.
- If GitHub Copilot is used, you must clearly indicate this in the code comments.
• In alignment with McMaster academic integrity policy, it “shall be an offence knowingly to submit academic work for assessment that was purchased or acquired from another source”. This includes work created by generative AI tools. Also state in the policy is the following, “Contract Cheating is the act of”outsourcing of student work to third parties” with or without payment.” Using Generative AI tools is a form of contract cheating. Charges of academic dishonesty will be brought forward to the Office of Academic Integrity.
Heart Disease Classification Challenge
Overview
In this assignment, you will analyze the UCI Heart Disease Dataset, which contains medical records used to predict the presence of heart disease in patients. The dataset includes a mix of categorical and numerical variables, some missing values, and class imbalance.
For the context of data science methods for heart disease prediction, refer to - Detrano, R., et., al. (1989). International application of a new probability algorithm for the diagnosis of coro- nary artery disease. The American journal of cardiology, 64(5), 304-310. DOI:10.1016/0002- 9149(89)90524-9.
Dataset Information
The dataset is available at the UCI Machine Learning Repository:
https://archive.ics.uci.edu/ml/datasets/Heart+Disease
• Key Features:
– The dataset includes 303 observations with 13 features.
– Features include age, sex, chest pain type, resting blood pressure, cholesterol levels, fasting blood sugar, electrocardiographic results, and others.
– The response variable is num, which will be transformed to binary in the analysis.
Objectives
Analyze the dataset using two classification algorithms. Your analysis should include ex- ploratory data analysis, handling of missing values, feature selection, feature engineering, modeling, interpretation, and effective communication. The goal is to draw meaningful and well-supported conclusions from your analysis.
• Classifier requirement: At least one of the classifiers must be interpretable to allow for in-depth analysis and inference.
Assignment Questions
Address the following questions in your submission, providing detailed insights and conclusions based on your analysis.
1. Define and describe a classification problem using the dataset.
2. Apply any chosen data transformations, or explain why no transformations were necessary.
3. Provide a detailed description of the dataset, including variables, summaries, number of observations, data types, and distributions (include at least three statements).
4. Transform the response num into a binary outcome: 1 for heart disease and 0 for no heart disease. So combine 1, 2, 3, and 4 into 1 and 0 for 0. For Questions 4-16, use the transformed binary outcome.
5. Analyze relationships between variables and discuss their implications for feature selection or extraction (include at least two statements).
6. Drop the rows with the missing values. How many osbervations after dropping the missing values. Skip the outlier analysis.
7. Sub-group analysis: Explore potential sub-groups within the data using appropriate data science methods. Identify and visualize these sub-groups without using the labels and cate- gorical variables. Categroical variables already define sub groups so we don’t need to include them for this analysis.
8. Split 30% of the data for testing using a random seed of 1. Use the remaining 70% for training and model selection.
9. Identify the two classifiers you have chosen. Justify your selections based on the classifier requirement for this assignment.
10. Specify two metrics to compare classifier performance. Provide technical details on how each metric is computed.
11. Train two selected classifiers in (9) and identify optimal tuning parameters (if applicable) using the training set.
12. Apply a feature selection or extraction method to one of the classifiers in (9). Train this third classifier on the training set and identify optimal tuning parameters (if applicable) using the training set.
13. Use the selected metrics to evaluate three classifiers in (11) and (12) on the test set.
• Discuss your findings (at least two statements).
• Discuss the impact of feature selection or extraction on the performance of the classifier (at least one statement).
14. For the best interpretable model identified in (13), analyze and interpret the most important predictor variables in the context of the classification challenge (at least two statements).
15. [Bonus] Sub-group improvement strategy: If sub-groups were identified, propose and imple- ment a method to further improve the performance of one classifier. Compare the fourth classifier performance with the results from (13).
16. Team Contributions: Document each team member’s specific contributions to the ques- tions above. For group submissions, this should match the GitHub commit history. Individual submissions do not need to address this question.
17. Link to the public GitHub repository. This is optional for the individual submissions.
Notes
• Students can also choose one classifer not covered in the lectures.
Grading scheme
1. Answer [1]
2. Codes (variable type transformation, etc.) [1]
OR rationale for no transformation [1]
3. Codes [3] and three statements [2]
4. Codes for transforming the response variable [1]
5. Codes for association [2] and interpretation of figures or tables [2]
6. Codes [1]
answer [1]
7. Codes to identify sub groups [3] and Plot the sub groups [1]
8. Codes [1]
9. classifiers and justification [2]
10. Describe the two metrics [2]
11. Codes for training two classifiers [2]
Codes for tuning parameters (if any) [1]
12. Codes for feature selection or feature extraction [1]
Codes for training the third classifier with the selected or ex- tracted features [1]
Codes for tuning parameters (if any) [1]
13. Codes for evaluating three classifiers on the test set using two metrics in (10) [3]
Two statements for the findings [2]
One statement for the impact of feature selection or extraction
[1]
14. Codes finding the important variables [1]
Two statements for the analysis and interpretation of the most important predictor variables [2]
15. Codes for the sub-group improvement strategy (training and tun- ing parameters, if any) [Bonus 2]
Comparison of the performance with the results from (13) [Bonus 1]
Bonus 3 points will be added to the final grade
16. Document each team member’s specific contributions
17. Link to the public GitHub repository
The maximum point for this assignment is 39. We will convert this to 100%. The bonus 3 points will be added to the final grade.
All group members will receive the same grade if they contribute to the same.