DS 2043 Data Processing Workshop I
Group Project
1.Introduction
In this project, you are supposed to model, design and implement a real world web application that is related to data analysis. For example, a simple music management system, a university course arrangement system, a library book management system, an e-business inventory management system, a train ticket booking system, etc. (you are encouraged to propose your own application. You should do the implementation using Python, Django framework and MySQL DBMS (We will provide tutorial on MySQL DBMS). Your web application should provide users reasonable operations based on your application.
This is a group-based project. Normally, Each group has 3 students, and you should choose your group via the grouping link on iSpace.
2.Workflow Stages
The project workflow goes through several stages described below.
1.Problem definition. Each group will need to select a topic and formulate the problem, e.g., music management, e-business inventory management or ticket booking, etc.
2.Data collection. Each group will need to collect the data using the web scraping techniques introduced in DS 2043 Data Processing Workshop I.
a.Note: You can also download existing datasets or generate synthetic datasets if necessary. However, at least some information are crawled by your group.
3.Wrangle, prepare, clean the data. The collected data may contains missing and noisy entries or in consistency schemas. Therefore, you need to perform data preprocessing and data cleaning techniques we learned in the class.
4.Data analysis and visualization. Analyze, identify patterns, and explore the data use data analysis and visualization methods introduced in DS 2043 Data Processing Workshop I.
5.Web development. Design and implement the website that provides a user interface to query the dataset or the data analytic result.
6.Presentation
This project is aimed to help students to learn how to formulate a simple database-related problem/task/application and to gain hands on experience on how to solve the problem by using methods, algorithms and techniques taught in class. The students will conduct a project on an interesting application and will present the obtained results. Students are encouraged to identify new problems/tasks/applications.
3.Assessment of the Project (Total: 100%)
Data collection, data preprocessing, data analysis and visualization 40%
Web design 20%
Documentations (reasonable assumptions and justifications.) 20%
Presentation (Organize ideas into clearly identifiable sections with all information presented in logical sequence) 20%
4.Project Definition
Each group submit a two or three pages document to propose the application you are going to work on. It should provide the following information:
The problem you are going to solve
The functionalities of your project
Your work schedule and checkpoints
The outcome of your project
5.Requirements
a)Deliverables:
1.All code for your application, including the web scraping code, data preprocessing and analysis code, and web development code.
2.Any auxiliary files needed by your program to create the database (e.g., .xml, .txt, etc).
3.The documentation of the whole project (.doc or .docx)， including presentation slide.
All the files should be zipped and named as:
GroupXX_studentID.zip, where .zip is the extension name of your file
Each group only needs to submit one bundle of your material by one student.
b)UI requirement:
Your website needs at least 5 pages, each page need to include navigation bars, logo, footers, etc.
Your website should allow user registration and login. At least two user types should be offered. For example, suppose you are building a bookstore website, your user types could be administrator, registered user and anonymous visitor.
Your website needs to include at least four features. The more features you have, the higher score you will earn. Feature is a workflow that can allow the user to perform a complete task. Take the example of bookstore, the features could be:
oA user searches books.
oAn admin adds new books.
oAn admin views purchasing history and statistical/analysis results.
o…
c)Project Documentation Report Requirement:
1.Project description (e.g., background, purpose, functionalities, etc.)
2.Data description and data collection.
3.Data preprocessing and data analysis.
4.Website design and feature implementation.
5.Difficulties you encountered when doing the project and how you handle them.