MET CS 677 Kaggle Competition Guidelines (Online Class)
One of the requirements for MET CS 677 is the Kaggle Competition (15% of the grade). The competition must be done individually.
General Description
This is an opportunity for you to be creative in solving a data science problem.
Important: Competition is invite-only – DO NOT share invite link with other people.
Competition Description
For the competition you will need to solve the ML binary classification problem – predict which team will win Dota 2 game based on first 5 minutes of the game.
ML Metric we use for evaluating: ROC-AUC
You can use any ML technique, which we studied in the course. If you use something that we did not study – this is also great. Make sure to apply your data science skills.
How to make a submission on Kaggle
To make a submission on Kaggle you will need to train ML model in the Colab notebook and predict the test labels for the test set. Then you need to upload the CSV with predictions on Kaggle:
After you submit the CSV file it will give you PUBLIC roc-auc score of your prediction. You will have 5 attempts every day to try and increase the score.
Important: the final roc-auc will be computed automatically after competition ends on PRIVATE part of your data. If you don’t know what it means we will discuss it in details during one of the lectures. This is very important topic.
How to make submission on Blackboard
After competition ends you will need to screenshot your position on PRIVATE leaderboard. Do not send public leaderboard score, they don’t count for grading. You submission consists of two things:
- Screenshot of your position on Leaderboard
- The Colab Notebook which reproduces the ML model and produces the CSV file with predictions that gave you this position
It is very important to send correct Colab notebook which can be reproduced, because otherwise you won’t get credit for the competition.
Kaggle Competition Rubric
In order for you to receive the grade you must beat both simple and advanced baselines:
If you don’t beat the advanced baseline your grade will be 0.
After you beat the advanced baseline, your grade for Kaggle Competition is computed with the formula:
G = 505 - (position_private_leaderboard)*5 For example:
- If you have 1-st place on private leaderboard, you receive full grade: G= 505 - (position_private_ 1)*5 = 500
- If you have 5-th place on private leaderboard, you receive grade G = 505 - (position_private_5)*5 = 485
- If you have 45-th place on private leaderboard, you receive grade G = 505 - (45)*5 = 285