COMP90015 Distributed Systems
Project 2 - Decentralized Chat
Aaron Harwood
School of Computing and Information Systems
The University of Melbourne
2021 Semester II
Aaron Harwood (School of Computing and Information Systems?The University of Melbourne)COMP90015 Distributed Systems 2021 Semester II 1 / 33
Synopsis
In this assignment you will transform your existing chat program into a
decentralized chat application.
You may reuse as much of your project 1 code as you wish. You may rewrite
everything if you wish as well. Feedback from project 1 assessment can be used
to help improve your project 2 code and report.
The basic operation and protocol from project 1 will be the same in project 2,
however instead of a client and server there will be only one process which we
will call a peer. The peer will act as both a client and a server in the
decentralized system.
The intended operation of the chat peer is described in this slide set.
You are required to also implement one extended feature as described next.
Aaron Harwood (School of Computing and Information Systems ? The University of Melbourne)COMP90015 Distributed Systems 2021 Semester II 2 / 33
Extended Feature I
As well as the base protocol that everyone must implement, each group
must choose one extended feature from the list of three choices below,
design the protocol as an extension to the base protocol and implement in
their peer:
Room Migration – Rooms can be moved from one peer to another. E.g. before
a peer quits it can send its rooms to other peers. All peers that are currently in
those rooms should automatically disconnect and reconnect to the room when
it has been relocated. The peer that receives a room becomes the owner of that
room. Selection of which peers to send rooms to should be based on ensuring
that all existing peers in those rooms can connect to the new peer where their
room is migrated to. Rooms should not become “lost”, e.g. if the peer that is
receiving a room also quits in the middle of receiving it and therefore does not
migrate it further. Of course, if there are no neighboring peers to migrate a
room to then the room is lost if the peer quits. The user of a peer should also
be able to issue a command to migrate a room on a room-by-room basis.
Aaron Harwood (School of Computing and Information Systems ? The University of Melbourne)COMP90015 Distributed Systems 2021 Semester II 3 / 33
Extended Feature II
Shout – A Shout command can be used by any user currently joined in a
room. The command causes a provided message to be delivered to all rooms
on all peers of the network. All peers in the network includes all peers for
which there is a path of connections from the shouting peer, not just those
peers that are directly reachable by the shouting peer. The order of delivery at
all rooms on all peers of the network must be FIFO, i.e. that the order of the
messages shouted by a given user matches the order of those messages
delivered at each room at each peer, regardeless of whether some peers have
connected and/or disconnected in between. When the shouted message is
delivered to a room, the identity of the shouter should be listed as “IP:port
shouted”, where IP and port are the values as seen by the peer hosting the
room that the shouting peer is currently in.
Aaron Harwood (School of Computing and Information Systems ? The University of Melbourne)COMP90015 Distributed Systems 2021 Semester II 4 / 33
Extended Feature III
File Attachment – Attach a file or files to the message. Users should have the
option to download specific attachments or download all attachments for a
given received message. The download should happen by one of two
mechanisms: (i) directly connecting to the peer that sent the message, if the
peer indicates (as metadata in the message) this is allowed (e.g. the peer
would typically have a public IP address for this to work), or else the peer that
sent the message uploads the attachments to the peer maintaining the room
and peers receiving the message download it from that peer. When a peer is
uploading or downloading files it should not block the user from issuing further
commands. Note: be carefull when implementing functionality that access the
local file system - this creates a strong potential vulnerability.
Aaron Harwood (School of Computing and Information Systems ? The University of Melbourne)COMP90015 Distributed Systems 2021 Semester II 5 / 33
Chat Peer
The base functionality that all groups must implement is briefly:
Each peer is started by a user who interacts with the peer on the command line
in the same way as interacting with the client in project 1.
Each peer will maintain the chat rooms that are created by its user. Users can
only create rooms on their own peer. As explained shortly, the create and
delete room commands don’t require message passing because they are only
executed locally.
There is no “main hall” room in project 2. A peer that connects to another
peer must first obtain a list of rooms and then join one. The exact session
protocol is discussed further in.
Peers will provide a list of neihboring peers with IP address and port number,
that allows other peers to search the network of peers for chat rooms, using an
iterative search mechanism.
Peers will allow other peers to connect to them, to search for chat rooms and to
join chat rooms and chat in them. If a peer does not have a public IP address
then only peers on the same private network as the peer can connect to it.
Aaron Harwood (School of Computing and Information Systems ? The University of Melbourne)COMP90015 Distributed Systems 2021 Semester II 6 / 33
Overview of peer protocol I
Protocol aspects taken from Project 1:
List – (same as proj1) respond with a list of rooms that the peer is maintaining
Join – (same as proj1) allow a peer to join a room
Who – (same as proj1) provide a list of who is in the room
Kick – (similar to proj1) the kicked user is forcibly disconnected from the peer, and
blocked from reconnecting. This is effectively a local command that does not require
message passing, as explained below, since only the owner of a room can kick others and
rooms created on a peer can only be created by the user of that peer.
Quit – (same as proj1) disconnect from the peer
Message – (same as proj1) the message is sent to all peers in the room
Protocol aspects that are very different to Project 1:
The identity protocol and associated messages from project 1 will not be used. Rather a
peer’s IP and port number combination, e.g. 192.168.1.10:3000, as seen by the peer
hosting the room, will be used and shown as the peer’s identity. The peer cannot change
its identity unless it disconnects and reconnects from a different IP address and/or a
different port number.
The CreateRoom functionality will exist but will only be accepted if the command was
given by the user of the peer, which means that effectively no actual messages are
required since this command will never be accepted by any other peers. It can therefore
be directly implemented in the peer without message passing.
The Delete functionality will exist but similar to CreateRoom, the delete command is a
local command only that does not require message passing.
Additional protocol aspects:
Aaron Harwood (School of Computing and Information Systems ? The University of Melbourne)COMP90015 Distributed Systems 2021 Semester II 7 / 33
Overview of peer protocol II
ListNeighbors – a peer can ask a peer for a list of peers’ network addresss that are
connected to it (of course, connected peers that are joined in a room also have their
network address exposed however not all connected peers must be joined in a room)
SearchNetwork – a local command that causes the peer to do a breadth first search of all
peers available to it (using a combination of ListNeighbors and RoomList messages, to
gather a list of chat rooms over all accessible peers and show that to the user
Connect – a local command that causes the peer to make a TCP connection to the peer
as given by the user, i.e. the user gives the IP and port number
Help – a local command that lists all of the commands available to the user with their
syntax that shows how to use them
Aaron Harwood (School of Computing and Information Systems ? The University of Melbourne)COMP90015 Distributed Systems 2021 Semester II 8 / 33
Initial connection to a peer
From this point on, for the purposes of discussion a peer that is
connecting to another peer will be called the client and the other peer will
be called the server.
When a peer process is started by the user, it does not connect to any other
peer initially.
The user must use the Connect command (local command) to cause the peer
to make a TCP connection to another peer. The user must know the IP and
port number of the peer to be connected to.
When the connection is established from client to server, the server simply
waits for the client to send a command.
While connected:
The user can use the List and other commands to find rooms, join a room and start
chatting, very much the same way as project 1. The biggest differences are that the user’s
identity is its IP and port number (as seen by the server) and that there is no “main hall”.
The user can use the ListNeighbors and SearchNetwork command (the operation of
these commands is explained further in) to search the broader network of peers.
The user must use the Quit command to disconnect from the peer. The user can then
connect to another peer.
When not connected to another peer, all of the commands issued are with
respect to the local peer’s room list, including the local commands
CreateRoom, Delete and Kick.
The peer process quits (disconnecting any connections first) when the standard
input is closed, using Ctrl-D on the terminal.
Aaron Harwood (School of Computing and Information Systems ? The University of Melbourne)COMP90015 Distributed Systems 2021 Semester II 9 / 33
Client Command Line Interface
The client must accept input from standard input. Each line of input
(terminated by a newline) is interpreted by the client as either a command or a
message. If the line of input starts with a hash character “#” then it is
interpreted as a command, otherwise it is interpreted as a message.
The client must write all output to standard output. Since chat is
asynchronous, messages may arrive at any time and will be written to standard
output when they arrive. For an extended feature the command line interface
should provide ways to interact with the extended feature.
If some data such as streaming video needs to be displayed, a GUI window can
be opened to display that. It should be controllable from the command line
and may also be controllable from the GUI window itself.
Aaron Harwood (School of Computing and Information Systems ? The University of Melbourne)COMP90015 Distributed Systems 2021 Semester II 10 / 33
JSON Format
All protocol messages, i.e. sent between client and server, will be in the form
of newline terminated JSON objects.
A JSON library must be used to marshal JSON output and to unmarshal JSON
objects. Do not implement JSON (un)marshalling yourself.
All data written to the TCP connection must be UTF8 encoded.
When implementing an extended feature you may dynamically change to
another data format other than JSON for that feature if you wish, but the base
protocol must be implemented in JSON and the base protocol must continue
to work regardless of whether the feature is used or not. E.g. you may even use
a separate TCP connection to transmit binary data when required and have
additional default ports for those aspects. You may add some fields to the base
protocol messages if you need to support your extended feature, but you
cannot change any of the existing fields.
Aaron Harwood (School of Computing and Information Systems ? The University of Melbourne)COMP90015 Distributed Systems 2021 Semester II 11 / 33
Connect command
The peer to connect to is not given on the command line. Rather to make
a connection to another peer:
>#connect 142.250.70.238:4444
[] 192.168.1.10:38283>
The room is empty [] (no room) and the identity is the IP and port number as
seen by the remote peer.
Note this example is a little contrived: the peer with the public IP address
142.250.70.238 would not see the private IP address 192.168.1.10, but rather
would see the public IP address of the router that the privately connected peer
is getting access through to the public network. Your actual output will differ
according to the specific networks that you test on. The examples in this
project specification continue with these contrived IP addresses.
If the port number is not provided then the default port number is used. As
well, the connect command should allow the user to optionally specify which
port they would like the TCP connection to use locally when making the
connection:
>#connect 142.250.70.238:4444 5000
[] 192.168.1.10:5000>
Aaron Harwood (School of Computing and Information Systems ? The University of Melbourne)COMP90015 Distributed Systems 2021 Semester II 12 / 33
Help command
The help command should just list the available commands and their
syntax, e.g. like:
>#help
#help - list this information
#connect IP[:port] [local port] - connect to another peer
#quit - disconnect from a peer
...
You can simply provide a short description of all the commands like above.
Aaron Harwood (School of Computing and Information Systems ? The University of Melbourne)COMP90015 Distributed Systems 2021 Semester II 13 / 33
Join Room Protocol
The client may at any time send a Join message to the server to indicate
that the client wishes to change their current room to the indicated room.
{
"type":"join",
"roomid":"comp90015"
}
E.g.:
[] 192.168.1.10:38283>#join comp90015
Note the format above shows [] to indicate that the client having identity
192.168.1.10:38283 as seen by the server is not in a room. We can use
the empty string "" to represent not in a room. Text messages sent when
not in a room are ignored.
Aaron Harwood (School of Computing and Information Systems ? The University of Melbourne)COMP90015 Distributed Systems 2021 Semester II 14 / 33
Join Room Protocol
When receiving a Join message the server must follow exactly this
protocol:
If roomid is invalid or non existent then client’s current room will not change.
The special room given by "" indicates that the client wants to leave the room
but not join another room. I.e. stay connected but not be in any room.
Otherwise the client’s current room will change to the requested room.
If the room did not change then the server will send a RoomChange message
only to the client that requested the room change.
If the room did change, then the server will send a RoomChange message to all
clients currently in the requesting client’s current room and the requesting
client’s requested room.
E.g.:
{
"type":"roomchange",
"identity":"192.168.1.10:38283",
"former":"",
"roomid":"comp90015"
}
Aaron Harwood (School of Computing and Information Systems ? The University of Melbourne)COMP90015 Distributed Systems 2021 Semester II 15 / 33
Join Room Protocol
The client will determine from the message whether the request was
successful or not and will output relevant information:
If the request was not successful: “The requested room is invalid or
non existent.”
If the request was successful then either “192.168.1.10:4444 moved to
comp90015” if the client was not already in a room or “192.168.1.10:4444
moved from roomid to comp90015” if the client was already in a room
called roomid in this example.
And on the command line:
[] 192.168.1.10:38283>#join comp90015
[] 192.168.1.10:38283>
192.168.1.10:38283 moved to comp90015
[comp90015] 192.168.1.10:38283>
Aaron Harwood (School of Computing and Information Systems ? The University of Melbourne)COMP90015 Distributed Systems 2021 Semester II 16 / 33
Room Contents Message I
The RoomContents message lists all client identities currently in the room:
{
"type":"roomcontents",
"roomid":"comp90015",
"identities":["192.168.1.10:38283","192.168.1.10:5000","142.250.70.238:4444"],
}
There is no need to indicate an owner since the owner of the room is
always the peer that is sending the message.
The client may at any time send a Who message to request a
RoomContents message from the server for a given room:
{
"type":"who",
"roomid":"comp90015"
}
Aaron Harwood (School of Computing and Information Systems ? The University of Melbourne)COMP90015 Distributed Systems 2021 Semester II 17 / 33
Room Contents Message II
e.g.
[comp90025] 192.168.1.10:38283>#who comp90015
Note that obviously the server knows the identity of every client
connection, so the identity of the sender is not required.
Aaron Harwood (School of Computing and Information Systems ? The University of Melbourne)COMP90015 Distributed Systems 2021 Semester II 18 / 33
Room List Message
The RoomList message lists all room ids and the count of identities in
each room:
{
"type":"roomlist",
"rooms":[{"roomid":"comp90025","count":5},
{"roomid":"comp90015","count":7},
{"roomid":"CovidWillComeToAnEnd","count":4}]
}
The client may at any time send a List message to request a RoomList
message from the server:
{
"type":"list"
}
Aaron Harwood (School of Computing and Information Systems ? The University of Melbourne)COMP90015 Distributed Systems 2021 Semester II 19 / 33
Room Information at the Client
The client displays information regarding rooms and room lists whenever
they arrive. Examples have been given on previous slides.
Aaron Harwood (School of Computing and Information Systems ? The University of Melbourne)COMP90015 Distributed Systems 2021 Semester II 20 / 33
Create Room
Only the user of the peer can create a room on that peer, using the
CreateRoom command. There is no message associated with this
command. Rather the logic is implemented locally and is the same as in
project 1:
If the requested name of the room is valid and does not already exist then it is
created – the room name must contain alphanumeric characters only, start
with an upper or lower case letter, have at least 3 characters and at most 32
characters.
The peer outputs either e.g. “Room jokes created.” or “Room jokes is
invalid or already in use.” depending on whether the local command
worked or not, for the example “jokes” room.
Aaron Harwood (School of Computing and Information Systems ? The University of Melbourne)COMP90015 Distributed Systems 2021 Semester II 21 / 33
Room Ownership
The owner of a room is always the peer that created it. It can only be
created by the user of that peer. Therefore there is no metadata required
to maintain the owner of each room. For the purposes of discussion, the
“owner” of a room always refers to the peer that created it.
Aaron Harwood (School of Computing and Information Systems ? The University of Melbourne)COMP90015 Distributed Systems 2021 Semester II 22 / 33
Delete Room
The owner of a room can at any time issue a Delete command. Similarly
to the create room and kick user, this command does not generate
messages and the logic is simply executed locally.
The peer will first treat this as if all users of the room had sent a
RoomChange message to the empty room "". Then the peer will delete the
room. The peer can print whether the room was successfully deleted or
not.
Aaron Harwood (School of Computing and Information Systems ? The University of Melbourne)COMP90015 Distributed Systems 2021 Semester II 23 / 33
Messages
The client can at any time send a Message:
{
"type":"message",
"content":"Hi there!"
}
A message received by the server is relayed to all clients that are within
the same room as the client who sent the message. The id of the sender is
appended to the message:
{
"type":"message",
"identity":"192.168.1.10:38283",
"content":"Hi there!"
}
Clients will never receive messages for rooms that they are not in and
messages sent by a client that is not in a room are ignored (silently
dropped).
Aaron Harwood (School of Computing and Information Systems ? The University of Melbourne)COMP90015 Distributed Systems 2021 Semester II 24 / 33
Quit
The client can at any time send a Quit message:
{
"type":"quit"
}
The server will remove the user from their current room, sending an
appropriate RoomChange message to all clients in that room. The roomid
of the RoomChange message will be an empty string.
When the server sends the RoomChange event to the disconnecting client,
then it can close the connection.
When the client that is disconnecting receives the RoomChange message,
then it can close the connection.
Aaron Harwood (School of Computing and Information Systems ? The University of Melbourne)COMP90015 Distributed Systems 2021 Semester II 25 / 33
Abrupt Disconnections
If a client gets disconnected (e.g. if the TCP connection is broken), then
the server treats this as if the client had sent a Quit message. The server
should send appropriate messages to other clients in the system and set
room ownership appropriately.
Aaron Harwood (School of Computing and Information Systems ? The University of Melbourne)COMP90015 Distributed Systems 2021 Semester II 26 / 33
List Neighbors Protocol
A client may request the server to list its neighbors by sending a list
neighbors message:
{
"type":"listneighbors"
}
The server responds with a list of peers that are currently connected (not
including its own network address, or the address of the client that issued
the request):
{
"neighbors":["192.168.1.10:5000","142.250.70.238:4444"]
}
The list may be empty.
Aaron Harwood (School of Computing and Information Systems ? The University of Melbourne)COMP90015 Distributed Systems 2021 Semester II 27 / 33
Search Network
The SearchNetwork command is a local command. When the user issues this
command, the peer is required to automatically crawl over all of the accesible
peers in the network (using a breadth-first recursive strategy), connecting to
each peer (using a connection that is separate to the peer’s existing connection
if there is one) and find the rooms available in each peer using a List
command and other peers to search using the ListNeighbors request. The
output should be a room list for each peer that is found, e.g.:
[] 192.168.1.10:38283>#searchnetwork
[] 192.168.1.10:38283>
192.168.1.10:5000
gameworld 10 users
142.250.70.238:4444
googlehut 52 users
alumni 20 users
104.215.148.63:4444
microsoftroom 28 users
In this example, the peer 104.215.148.63:4444 was discovered after listing
the neighbors of one of the peers that was already known. The user can then
decide whether to manually connect to one of these other peers or not, using
the Connect command.
Aaron Harwood (School of Computing and Information Systems ? The University of Melbourne)COMP90015 Distributed Systems 2021 Semester II 28 / 33
Technical aspects
Use Java 1.11.
All message formats should be JSON encoded. Use a JSON library for this.
The format can be different for the extended feature.
Your program should be cleanly finished by terminating all running threads.
Package everything into a single runnable jar file, for the peer.
Your peer should be executable exactly as follows where the port option -p
gives the port that the peer listens on for incomming connections and the port
option -i gives the port that the peer uses to make connections to other peers
when the user issues the Connect command:
$> java -jar chatpeer.jar [-p port] [-i port]
Use command line option parsing (e.g. using the args4j library or your
choice).
The default server port listened on should be 4444. A command line option
can override this. The default port used to make connections to other peers
when the user uses the Connect command can be whatever port is used by the
OS (i.e. an emphemeral port), and therefore may change each time the user
disconnects and reconnects.
Terminating the standard input (Ctrl-D) should terminate the peer.
Aaron Harwood (School of Computing and Information Systems ? The University of Melbourne)COMP90015 Distributed Systems 2021 Semester II 29 / 33
Your Report I
Only 1 report is required per team.
Use 10pt font, double column, 1 inch margin all around. Put both team
members’ login names and student numbers at the top of the report.
In a few sentence, include a statement of the contribution from each team
member.
In about 1000 words: Discuss your design and implementation of your chosen
extended feature. Use formal diagrams to help with your explanation. Provide
an analysis/critical discussion of your extended feature implementation with
respect to relevant selected challenges of distributed systems.
In about 500 words: Develop/discuss a model that describes the decentralized
chat application, which includes your extended feature, in terms of the
communication patterns and network topology that arises over time as new
peers connect to exsisting peers and old peers disconnect, and with respect to
different commands that users can issue. E.g. a peer can only have one
existing connection to another peer at any point in time, which limits the kind
of patterns that we can see, other than when e.g. a breadth first search is
being conducted, which is limited by reachability between machines. Discuss all
of these aspects with respect to your model.
Aaron Harwood (School of Computing and Information Systems ? The University of Melbourne)COMP90015 Distributed Systems 2021 Semester II 30 / 33
Your Report II
In about 500 words: Develop a failure model for the decentralized chat
application which includes your extended feature, using basic assumptions as
discussed in the lecture, including e.g. assumptions of failure rate of a peer
process. Use the model to make some basic statements about how fault
tolerant the decentralized chat application is (or is not as the case may be) to
process failure and network failure. You may consider what happens when
there are N peers connected into one large connected component and N
becomes very large.
Aaron Harwood (School of Computing and Information Systems ? The University of Melbourne)COMP90015 Distributed Systems 2021 Semester II 31 / 33
Assessment
Assessment for this project is worth 20% of the total subject assessment.
The project assessment is split between the software implementation (50%)
and the report (50%):
Software assessment is comprised of two parts:
Up to 65% of software marks for an implementation that compiles to provide a chat peer jar file, as
also submitted, that provides the minimal amount of functionality: client can connect and join a
room, can send and receive chat messages and can quit. If this minimal functionality does not work
then at most 65% will be awarded for the software, based on what aspects do appear to work
and/or code inspection.
Additional 15% of software marks are awarded based on correct implementation of remaining base
protocol, including ListNeighbors and SearchNetwork. These marks are only awarded if the
functionality above all works as expected.
Final 20% of the software marks are awarded based on a correct implementation of the extended
feature.
When some part of an implementation does not work, up to 25% of the marks for that part of the
implementation may be awarded based on code inspection. In this case, well presented code has a
better chance of attracting more marks.
Report:
Assessment is based on presentation, demonstration of critical thinking, correctness, being concise,
consistent in terminology, and complete in addressing the required criteria on the previous slide.
Assessment is weighted based on the word limitations for each criteria.
Aaron Harwood (School of Computing and Information Systems ? The University of Melbourne)COMP90015 Distributed Systems 2021 Semester II 32 / 33
Submission
You need to submit the following (only one person needs to submit per
team) via LMS:
Your report in PDF format only.
Your chatpeer.jar file.
Your source files in a .ZIP or .TAR archive only.