Assignment-2 | COMP9321 21T1 | WebCMS3
Assignment-2
Data Service for TV Shows
In this assignment, you are asked to develop a Flask-Restx data service that allows a client to read and store
some TV Shows, and allow the consumers to access the data through a REST API.
The assignment is based on The TV Maze API, which provides a detailed list of TV shows. You can explore the
TVmaze API using the following links
The source URL: (http://api.worldbank.org/v2/) http://api.tvmaze.com/shows (http://api.tvmaze.com/shows)
(http://api.worldbank.org/v2/) Documentations on API Call Structure: https://www.tvmaze.com/api
(https://www.tvmaze.com/api)
***Disclaimer: We are using an extremal API provided by TV Maze (https://www.tvmaze.com/). We want the
students to interact with real life web services to add to the learning experience and hence we are not responsible
nor liable for the wording or inclusion/exclusion of TV shows and descriptions within the API. This is a "building
your REST API" exercise and should be treated as such.
In this assignment, you are going to use the information provided in this API and add a few functionalities as listed
below:
Assignment Specification
Question-1: import a TV Show (2 marks)
This operation can be considered as an on-demand 'import' operation to get the details of a TV show and store it
in your application. The service will download the JSON data for the given TV show (by its name) ; You must use
sqlite for storing the data (the name of the database should be YOURZID.db ) locally after importing the TV show
.
You can use the following to query the API: http://api.tvmaze.com/search/shows?q=
(http://api.tvmaze.com/search/shows?q=TITLE) ?
For example, you can check the following query: http://api.tvmaze.com/search/shows?q=good%20girls
(http://api.tvmaze.com/search/shows?q=good%20girls)
To import a tv show, your API accepts a query parameter called "name".:
name : title for the tv show
After importing the collection, the service should return a response containing at least the following information:
Specification Make Submission Check Submission Collect Submission
4/2/2021 Assignment-2 | COMP9321 21T1 | WebCMS3
https://webcms3.cse.unsw.edu.au/COMP9321/21T1/resources/59281 2/15
id : a unique integer identifier automatically generated (this might be different than tvmase_id)
tvmaze-id : the id of the tv show in tvmaze API
last-update : the time the collection stored in the database
_links : the URL with which the imported collection can be retrieved (as shown in below example)
Important : For this assignment , you are asked to access the given Web content programmatically. Some Web
hosts do not allow their web pages to be accessed programmatically, some hosts may block your IP if you access
their pages too many times. During the implementation, download a few test pages and access the content locally
- try not to perform too many programmatically. Check the documentation of the tvmaze API to get insights about
their rate limiting policy.
Example:
POST /tv-shows/import?name=good girls
An example response [This is not what you store in DB, it is the call's response]
: 201 Created
{
"id" : 123,
"last-update": "2021-04-08-12:34:40",
"tvmaze-id" : 23542,
"_links": {
"self": {
"href": "http://[HOST_NAME]:[PORT]/tv-shows/123"
}
}
}
You must import a TV show if only the given name matches a valid TV show (good girls, Good Girls, GoodGirls
all match each other but they should not match a TV show like "Good Boys"). Be noted that the
TVMaze API provides a fuzzy search and hence tolerates typos, capital/small, dashes...etc. and at the same
time provides more results than the exact TV Show. You should only import the matching one (ignoring
cases, and any character except English alphabet and numbers ). In case there are more than one TV show
with the same one, it is up to you how to deal with it (e.g., importing both, importing the latest one, etc)
What and how you store the data in DB is up to you, but take a look at the rest of questions to know what
attributes you need to keep for each TV show
Do not get confused with having two identifiers ("id", and "tvmaze-id"); "id" is a unique identifier in your
service and all of your operations relay on this id to work; "tvmaze-id" is just a reference to the original data.
CLARIFICATION: You should never change an ID of a resource (do not update ids)
CLARIFICATION: You should replace [HOST_NAME]:[PORT] with correct values
A brief description about _links here : https://developer.wordpress.org/rest-api/using-the-rest-api/linking-andembedding
(https://developer.wordpress.org/rest-api/using-the-rest-api/linking-and-embedding/) /
Question 2 - Retrieve a TV Show (2 marks)
This operation retrieves a collection by its ID (the ID that is generated by your application) . The response of this
operation will show the details of TV show. Please see the provided response example below to see what
attributes should be included in the response. "_links" gives the links for previous, next, and current resource if
they exist . The next and previous resources are based on the sequential ID generated by your application.
4/2/2021 Assignment-2 | COMP9321 21T1 | WebCMS3
https://webcms3.cse.unsw.edu.au/COMP9321/21T1/resources/59281 3/15
The interface should look like as like below:
GET /tv-shows/{id}
Example Response returns: 200 OK
4/2/2021 Assignment-2 | COMP9321 21T1 | WebCMS3
https://webcms3.cse.unsw.edu.au/COMP9321/21T1/resources/59281 4/15
Question 3- Deleting a TV show (2 marks)
{
"tvmaze-id" :23542,
"id": 124,
"last-update": "2021-04-08-12:34:40",
"name": "Good Girls",
"type": "Scripted",
"language": "English",
"genres": [
"Drama",
"Comedy",
"Crime"
],
"status": "Running",
"runtime": 60,
"premiered": "2018-02-26",
"officialSite": "https://www.nbc.com/good-girls",
"schedule": {
"time": "22:00",
"days": [
"Sunday"
]
},
"rating": {
"average": 7.4
},
"weight": 100,
"network": {
"id": 1,
"name": "NBC",
"country": {
"name": "United States",
"code": "US",
"timezone": "America/New_York"
}
},
"summary": "
Good Girls follows three \"good girl\" suburban wives and mothers who
"_links": {
"self": {
"href": "http://[HOST_NAME]:[PORT]/tv-shows/124"
},
"previous": {
"href": "http://[HOST_NAME]:[PORT]/tv-shows/123"
},
"next": {
"href": "http://[HOST_NAME]:[PORT]/tv-shows/125"
}
}
}
4/2/2021 Assignment-2 | COMP9321 21T1 | WebCMS3
https://webcms3.cse.unsw.edu.au/COMP9321/21T1/resources/59281 5/15
This operation deletes an existing TV show from the database. The interface should look like as below:
DELETE /tv-shows/{id}
Returns: 200 OK
{
"message" :"The tv show with id 134 was removed from the database!",
"id": 134
}
Question 4 - Update a TV Show (2 marks)
This operation partially updates the details of a given TV Show.
The interface should look like the example below:
PATCH /tv-shows/{id}
{
"name": "Good Girls",
"language": "English",
"genres": [
"Drama",
"Comedy",
"Crime"
]
}
The above payload is just an example; it can contain any of the TV show attributes. Take a look at the example
response in Question 2 to know the existing attributes.
Returns: 200 OK
{
"id" : 123,
"last-update": "2021-04-08-12:34:50",
"_links": {
"self": {
"href": "http://[HOST_NAME]:[PORT]/tv-shows/123"
}
}
}
Question 5 - Retrieve the list of available TV Shows 4 marks)
This operation retrieves all available TV shows. The interface should look like as like below:
4/2/2021 Assignment-2 | COMP9321 21T1 | WebCMS3
https://webcms3.cse.unsw.edu.au/COMP9321/21T1/resources/59281 6/15
All four parameters are optional with default values being "order_by=+id", "page=1", and "page_size =100",
filter="id,name". "page" and "page_size" are used for pagination; "page_size" shows the number of TV shows per
page. "order_by" is a comma separated string value to sort the list based on the given criteria. The string consists
of two parts: the first part is a special character '+' or '-' where '+' indicates ordering ascendingly, and '-' indicates
ordering descendingly . The second part is an attribute name which is one of {id,name,runtime,premiered,ratingaverage}.
Here are some sample values of "order_by" :
+ratingaverage,+id
order by "rating-average ascending" and then "id ascending"
In this case sorti ng by "rating-average" has p riority over "id". This is similar to SQL order
by clause :
" rating-average ASC, id ASC "
-premiered order by "premiered descending"
"filter" is also another comma separated values (only consider= tvmaze_id ,id ,last-update ,name ,type ,language
,genres ,status ,runtime ,premiered ,officialSite ,schedule ,rating ,weight ,network ,summary), and show what
attribute should be shown for each TV show accordingly. Take a look at the following example to know how
response should be like:
GET /tv-shows?order_by=+id&page=1&page_size=100&filter=id,name
All four parameters are optional with default values being "order_by=+id", "page=1", and "page_size=100",
"filer=id,name"
Returns: 200 OK
GET /tv-shows?order_by= & page=1 & page_size=100 & filter=
4/2/2021 Assignment-2 | COMP9321 21T1 | WebCMS3
https://webcms3.cse.unsw.edu.au/COMP9321/21T1/resources/59281 7/15
Question 6 - get the statistics of the existing TV Show (3 marks)
This operations accepts a parameter called "format" which can be either "json" or "image". Depending on the
format your operation should provide the following information: In case when the the format is image, your
operation should return an image (can be in any image format) and the image illustrates the requested information
in a visualization (apply all your knowledge when creating the visualization such as choosing appropriate
visualization type and making sure that it is human readable, clear, and informative).
TV shows break down by an attribute determined by the "by" parameter; this parameter can be any of the
following TV show attributes: "language" (showing the percentage of TV shows per Language), "genres",
"status", and "type". In case of "genres", a TV show can have multiple values; you should come up with a
solution to visualise it. For instance you can think of h ow to visualise what percentage of movies belong to
both "Comedy" and "Crime" genres, etc.
Total Number of TV shows
Total Number of TV shows updated in the last 24 hours
The interface should look like as like below when the format is JSON:
GET /tv-shows/statistics?format=json&by=language
Returns: 200 OK
{
"page": 1,
"page-size": 100,
"tv-shows": [
{
"id" : 1,
"name" : "Good Girls"
},
{
"id" : 2,
"name" : "Brilliant Girls"
},
...
],
"_links": {
"self": {
"href": "http://[HOST_NAME]:[PORT]/tv-shows?order_by=+id&page=1&page_size=1000&filter=id
},
"next": {
"href": "http://[HOST_NAME]:[PORT]/tv-shows?order_by=+id&page=2&page_size=1000&filter=id
}
}
}
4/2/2021 Assignment-2 | COMP9321 21T1 | WebCMS3
https://webcms3.cse.unsw.edu.au/COMP9321/21T1/resources/59281 8/15
Resource created 16 days ago (Wednesday 17 March 2021, 01:08:22 PM), last modified a day ago (Thursday 01 April 2021, 10:30:28
AM).
{
"total": 1241,
"total-updated": 24,
"values" : { "English": 60.7, "French": 19.2, ... }
}
Notice:
TVMAZE API is only used in Question 1 for importing a new TV show. The rest of operations relay on the
existing TV Shows in your local database
There is not template code for this assignment, you submission should stick to the assignment
specifications.
Your submission will be marked manually.
You should adhere to the best design guidelines for REST API mentioned in the lecture (e.g., appropriate
responses based on JSON format with proper status codes, full API documentation : the generated
Swagger should be fully self-explanatory with operation summary, parameter descriptions, default values).
You should consider cases such as invalid titles or any invalid attempts to use the endpoint ( e.g. If the input
title doesn 't exist in the data source, return error 404)
You should return appropriate responses (JSON) in case of already imported collections
Your code must implemented in flask-restx and automatically generate swagger doc for testing the
endpoints.
Your code must be executable in CSE machines
Your code must not require installing any software (python libraries are fine)
Your code must be written in Python 3.5 or newer versions.
Your operations (API methods) must return appropriate responses in JSON format, and appropriate HTTP
response code! e.g., 500 Internal Server Error is inappropriate response!
Make sure you are using right datatypes in the database and in you API methods (e.g., not string for years
'2017')
Some of the response of some operations contain "_links", depending on the response this should include
links to "self", "next", and "previous" resources.
Submission:
The Deadline is Saturday the 3rd of April 2021 17:59
One and only one Python script file named " YOUR_ZID .py" which contains your code
How I can submit my assignment?
Go to the assignment page click on the "Make Submission" tab; pick your files which must be named
"YOUR_ZID.py". Make sure that the files are not empty, and submit the files together.
Can I submit my file after deadline?
Yes, you can. But 25% of your assignment will be deducted as a late penalty per day. In other words, if you
be late for more than 3 days, you will not be marked.
Comments
(/COMP9321/21T1/forums/search?forum_choice=resource/59281) (/COMP9321/21T1/forums/resource/59281)
4/2/2021 Assignment-2 | COMP9321 21T1 | WebCMS3
https://webcms3.cse.unsw.edu.au/COMP9321/21T1/resources/59281 9/15
Add a comment
Rongquan Han (/users/z5216960) 10 minutes from now (Fri Apr 02 2021 21:54:27 GMT+0800 (China
Standard Time))
Hi, for the return of question 2, do we need to include all these attritbutes? Thanks
Reply
Mohammadali Yaghoubzadehfard (/users/z5138589) about 3 hours from now (Sat Apr 03 2021
00:35:19 GMT+0800 (China Standard Time))
Include all attribues in the example provided in the question
Reply
Daniel Fan (/users/z5114117) about 4 hours ago (Fri Apr 02 2021 17:35:59 GMT+0800 (China Standard
Time))
Hi,
4/2/2021 Assignment-2 | COMP9321 21T1 | WebCMS3
https://webcms3.cse.unsw.edu.au/COMP9321/21T1/resources/59281 10/15
This question was asked by Hillary earlier:
https://webcms3.cse.unsw.edu.au/COMP9321/21T1/foru...
(https://webcms3.cse.unsw.edu.au/COMP9321/21T1/forums/2797387) , but I want to confirm
that the intention is for the "_links" created in question 5 to not work when pasted in the
browser/used as a http string ?
As noted by Hillary, the '+' character will be interpreted as a space when inputted as a url on the
browser. For this to work, the '+' needs to be encoded as '%2B'
e.g. from:
to
Swagger seems to do this encoding automatically, when using the swagger UI to try a request.
As such, none of the "_links" generated in question 5 involving '+' will work as per the spec
currently and will return 404 due to invalid input. Does this align with with the tutors/lecturer's
intentions?
Reply
"href": "http://[HOST_NAME]:[PORT]/tv-shows?order_by=+id&page=1&page_size=1000&filt
"href": "http://[HOST_NAME]:[PORT]/tv-shows?order_by=%2Bid&page=1&page_size=1000&f
Mohammadali Yaghoubzadehfard (/users/z5138589) about 3 hours from now (Sat Apr 03 2021
00:36:28 GMT+0800 (China Standard Time))
do not encode the links
Reply
Matias Alfredo Morales Armijo (/users/z5216410) about 5 hours ago (Fri Apr 02 2021 17:01:25
GMT+0800 (China Standard Time))
hi professor, I have a doubt about the assignment 2. Can I create a empty database or I need to
create programmatically the database?. thanks, Matias
Reply
Mohammadali Yaghoubzadehfard (/users/z5138589) about 3 hours from now (Sat Apr 03 2021
00:36:54 GMT+0800 (China Standard Time))
you need to do it programatically
Reply
Helin Wang (/users/z5272064) about 7 hours ago (Fri Apr 02 2021 14:58:22 GMT+0800 (China Standard
Time))
4/2/2021 Assignment-2 | COMP9321 21T1 | WebCMS3
https://webcms3.cse.unsw.edu.au/COMP9321/21T1/resources/59281 11/15
For Q6 is it ok to have a response like this.? Or we have to display the image in swagger?
Thanks.
Reply
Mohammadali Yaghoubzadehfard (/users/z5138589) about 3 hours from now (Sat Apr 03 2021
00:37:46 GMT+0800 (China Standard Time))
Your endpoint should return an image which can be shown in a browser
Reply
Vishaal Vageshwari (/users/z5161415) about 7 hours ago (Fri Apr 02 2021 14:17:48 GMT+0800 (China
Standard Time))
If we save a figure to the current directory for question 6 does the name of the image have to be
unique to me as a student, i.e. attaching my zID to the plot name - z1111111_plot.png.
Reply
Mohammadali Yaghoubzadehfard (/users/z5138589) about 3 hours from now (Sat Apr 03 2021
00:37:58 GMT+0800 (China Standard Time))
Yes
Reply
Vincent Shi (/users/z5182291) about 8 hours ago (Fri Apr 02 2021 13:26:26 GMT+0800 (China Standard
Time)), last modified 33 minutes from now (Fri Apr 02 2021 22:17:47 GMT+0800 (China Standard Time))
Hi, for Q4, should we:
1. Allow users to update network's id?
2. Allow users to update _links['self'], _links['previous'], and/or _links['next']?
My assumption is that we can't update these. Thanks in advance
4/2/2021 Assignment-2 | COMP9321 21T1 | WebCMS3
https://webcms3.cse.unsw.edu.au/COMP9321/21T1/resources/59281 12/15
Reply
Mohammadali Yaghoubzadehfard (/users/z5138589) about 3 hours from now (Sat Apr 03 2021
00:38:29 GMT+0800 (China Standard Time))
User cannot update those since they are not part of the resource
Reply
Vincent Shi (/users/z5182291) about 3 hours from now (Sat Apr 03 2021 00:42:52 GMT+0800
(China Standard Time)), last modified about 3 hours from now (Sat Apr 03 2021 00:43:35 GMT+0800
(China Standard Time))
Hi I understand _links are not part of the resource. But I think network should be part of
the resource right? If so, should we allow users to update network's id?
Reply
Alexander Georges (/users/z5115344) about 10 hours ago (Fri Apr 02 2021 11:37:15 GMT+0800 (China
Standard Time))
For Genre Statistics Q6, as we are returning percentages -> is it okay that percentages don't
add up to 100% (in the case that a show is multiple genres) -> or would you say that cross
sectional genres 'eg comedy & crime' is a genre in itself that should be represented?
I understand that there may be an opportunity to have creative license here, but not sure that
there is a clear cut answer (one that is better than the other). Any recommendations here?
Reply
Mohammadali Yaghoubzadehfard (/users/z5138589) about 3 hours from now (Sat Apr 03 2021
00:40:16 GMT+0800 (China Standard Time))
there is not a single correct answer, there are multiple... I would rather let you decide on your
question. DOES THAT MAKE SENSE?
Reply
Yangzi She (/users/z5142869) about 13 hours ago (Fri Apr 02 2021 09:04:12 GMT+0800 (China Standard
Time))
HI, the TypeError I got when I want to update the db
127.0.0.1 - - [02/Apr/2021 09:01:35] "PATCH /tv-shows/1 HTTP/1.1" 500 -
Traceback (most recent call last):
File "/Users/apple/untitled/venv/lib/python3.7/site-packages/flask/app.py", line 2464, in __call__
return self.wsgi_app(environ, start_response)
File "/Users/apple/untitled/venv/lib/python3.7/site-packages/flask/app.py", line 2450, in wsgi_app
response = self.handle_exception(e)
4/2/2021 Assignment-2 | COMP9321 21T1 | WebCMS3
https://webcms3.cse.unsw.edu.au/COMP9321/21T1/resources/59281 13/15
File "/Users/apple/untitled/venv/lib/python3.7/site-packages/flask_restplus/api.py", line 584, in
error_router
return original_handler(e)
File "/Users/apple/untitled/venv/lib/python3.7/site-packages/flask/app.py", line 1867, in
handle_exception
reraise(exc_type, exc_value, tb)
File "/Users/apple/untitled/venv/lib/python3.7/site-packages/flask/_compat.py", line 38, in reraise
raise value.with_traceback(tb)
File "/Users/apple/untitled/venv/lib/python3.7/site-packages/flask/app.py", line 2447, in wsgi_app
response = self.full_dispatch_request()
File "/Users/apple/untitled/venv/lib/python3.7/site-packages/flask/app.py", line 1952, in
full_dispatch_request
rv = self.handle_user_exception(e)
File "/Users/apple/untitled/venv/lib/python3.7/site-packages/flask_restplus/api.py", line 584, in
error_router
return original_handler(e)
File "/Users/apple/untitled/venv/lib/python3.7/site-packages/flask/app.py", line 1821, in
handle_user_exception
reraise(exc_type, exc_value, tb)
File "/Users/apple/untitled/venv/lib/python3.7/site-packages/flask/_compat.py", line 38, in reraise
raise value.with_traceback(tb)
File "/Users/apple/untitled/venv/lib/python3.7/site-packages/flask/app.py", line 1950, in
full_dispatch_request
rv = self.dispatch_request()
File "/Users/apple/untitled/venv/lib/python3.7/site-packages/flask/app.py", line 1936, in
dispatch_request
return self.view_functions[rule.endpoint](**req.view_args)
File "/Users/apple/untitled/venv/lib/python3.7/site-packages/flask_restplus/api.py", line 329, in
wrapper
return self.make_response(data, code, headers=headers)
File "/Users/apple/untitled/venv/lib/python3.7/site-packages/flask_restplus/api.py", line 350, in
make_response
resp = self.representations[mediatype](data, *args, **kwargs)
File "/Users/apple/untitled/venv/lib/python3.7/site-packages/flask_restplus/representations.py",
line 25, in output_json
dumped = dumps(data, **settings) + "\n"
4/2/2021 Assignment-2 | COMP9321 21T1 | WebCMS3
https://webcms3.cse.unsw.edu.au/COMP9321/21T1/resources/59281 14/15
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/json/__init__.py", line
238, in dumps
**kw).encode(obj)
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/json/encoder.py", line
201, in encode
chunks = list(chunks)
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/json/encoder.py", line
438, in _iterencode
o = _default(o)
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/json/encoder.py", line
179, in default
raise TypeError(f'Object of type {o.__class__.__name__} '
TypeError: Object of type set is not JSON serializable
Reply
Kan-Lin Lu (/users/z3417618) about 7 hours ago (Fri Apr 02 2021 14:55:08 GMT+0800 (China
Standard Time))
This might help:
https://stackoverflow.com/questions/8230315/how-to...
(https://stackoverflow.com/questions/8230315/how-to-json-serialize-sets)
Especially:
JSON (http://www.json.org/) notation has only a handful of native datatypes (objects, arrays,
strings, numbers, booleans, and null), so anything serialized in JSON needs to be expressed
as one of these types.
Make sure the returning dictionary is of the right datatype.
Reply
Yangzi She (/users/z5142869) about 6 hours ago (Fri Apr 02 2021 15:30:25 GMT+0800 (China
Standard Time))
Thank you very much!
Reply
Yangzi She (/users/z5142869) about 16 hours ago (Fri Apr 02 2021 05:59:31 GMT+0800 (China Standard
Time))
Hi, for Q4, when I want to update the db, I get TypeError:
4/2/2021 Assignment-2 | COMP9321 21T1 | WebCMS3
https://webcms3.cse.unsw.edu.au/COMP9321/21T1/resources/59281 15/15
Load More Comments
Can you give me a hint? I've been stuck here for a long time.
Reply
Mohammadali Yaghoubzadehfard (/users/z5138589) about 13 hours ago (Fri Apr 02 2021 08:40:37
GMT+0800 (China Standard Time))
You should provide a full stack of the error; this will not help to debug
Reply
Hasam Hasam (/users/z5206651) about 16 hours ago (Fri Apr 02 2021 05:45:11 GMT+0800 (China
Standard Time))
Just a bit confused with the deadline, and the announcement made about CSE servers being
down - so we can submit on Tuesday 6PM without any deduction to the final mark? But after
Tuesday 6PM the assignment will not be marked?
Reply
Mohammadali Yaghoubzadehfard (/users/z5138589) about 13 hours ago (Fri Apr 02 2021 08:31:02
GMT+0800 (China Standard Time))
Yes
Reply