GRPC留学生讲解、C++编程语言调试、辅导c/c++、辅导system
解析R语言编程|讲解SPSS
GRPC and Distributed Systems
Forward
In this project, you will design and implement a simple distributed file system (DFS). First, you will develop several file transfer
protocols using gRPC and Protocol Buffers. Next, you will incorporate a weakly consistent synchronization system to manage
cache consistency between multiple clients and a single server. The system should be able to handle both binary and textbased
files.
Your source code will use a combination of C++14, gRPC, and Protocol Buffers to complete the implementation.
Setup
You can clone the code in the Project 4 repository with the command:
git clone https://github.gatech.edu/gios-sum-19/pr4.git
Submission Instructions
Submit all code through the submit.py script given at the top level of the repository. For instructions on how to submit
individual components of the assignment, see the instructions below.
For this assignment, you may submit your code up to 10 times in 24 hours. After the deadline, we download your last
submission before the deadline, review your submission, and assign a grade. There is no limit to the number of times you
may submit your readme-student.md file.
After submitting, you may double-check the results of your submission by visiting the Udacity/GT autograding website and
going to the student portal.
Readme
Throughout the project, we encourage you to keep notes on what you have done, how you have approached each part of
the project, and what resources you used in preparing your work. We have provided you with a prototype file, readmestudent.md
that you should use throughout the project.
You may submit your readme-student.md file with the command:
python submit.py readme
At the prompt, please provide your GT username and password.
Raw Blame History7/15/2019 pr4/readme.md at master · gios-sum-19/pr4
https://github.gatech.edu/gios-sum-19/pr4/blob/master/readme.md 2/7
If this is your first time submitting, the program will then ask you if you want to save the JWT. If you reply yes, it will save a
token on your filesystem so that you don't have to provide your username and password each time that you submit.
The Udacity site will store your readme-student.md file in a database, where it will be used during grading. The submit script
will acknowledge receipt of your README file. For this project, like in Project 3, you only need to submit one README file for
both parts of the project.
Note: you may choose to submit a PDF version of this file (readme-student.pdf) in place of the markdown version. The
submission script will automatically detect and submit this file if it is present in the project root directory. If you submit both
files, we will give the PDF version preference.
Directions
Part 1 - Building the RPC protocol service
In Part 1, you will build a series of remote procedure calls (RPC) and message types that will fetch, store, list, and get
attributes for files on a remote server. The SunRPC implementation (ONC RPC) and XDR interface definition language (IDL)
you learned about in the lectures are currently deprecated in favor of more modern implementations of the concepts behind
RPC and IDL. There are many RPC replacements, including TI-RPC, Finagle, Thrift, and Cap'n Proto. However, in this
assignment, we will use gRPC for RPC services and Protocol Buffers as the definition language. The core gRPC library is
written in C but supports multiple languages, including C++, Java, Go, and others. It is actively developed and in use at
several organizations, such as Google, Square, Netflix, Juniper, Cisco, and Dropbox. In this assignment, we will use the gRPC
C++ API.
Part 1 Goals
The goal of part 1 is to generate an RPC service that will perform the following operations on a file:
Fetch a file from a remote server and transfer its contents via gRPC
Store a file to a remote server and transfer its data via gRPC
List all files on the remote server:
For this assignment, the server is only required to contain files in a single directory; it is not necessary to manage
nested directories.
The file listing should include the file name and the modified time (mtime) of the file data in seconds from the
epoch.
Get the following attributes for a file on the remote server:
Size
Modified Time
Creation Time
The client should be able to request each of the operations described above for binary and text-based files. The server will
respond to those requests using the gRPC service methods you specify in your proto buffer definition file.
Finally, the client should recognize when the server has timed out. gRPC can signal the server using a deadline timeout
as described in this gRPC article. You should ensure that your client recognizes this timeout signal.
Part 1 Sequence Diagram
A sequence diagram of the expected interactions in part 1 is available in the docs/part1-sequence.pdf file of this repository.
Protocol Buffers and gRPC7/15/2019 pr4/readme.md at master · gios-sum-19/pr4
https://github.gatech.edu/gios-sum-19/pr4/blob/master/readme.md 3/7
To begin part 1, you should first familiarize yourself with the basics of using Protocol Buffers. In particular, you should focus
on the use of RPC service definitions and message type definitions for the request and response types that are used by the
RPC services.
You will then create your protocol in the dfs-service.proto file inside the project repository. There are several required
services and message types described in the proto file that you should implement, but you may add as many additional
methods and/or message types that you deem necessary. What you name those services and message types is also up to
your discretion.
To autogenerate the gRPC and Protocol Buffer class and header files, we’ve provided a Makefile command that will take care
of that for you. When you are ready to generate your protobuf/gRPC files, run the following command from the root of the
repository:
make protos
You will find the results of that command in the part1/proto-src directory of the repository. You should familiarize yourself
with the results in that directory, but you won’t need to, and should not, make any changes to those files. Your job will be to
override the service methods in your dfslib-servernode-p1.cpp source file.
Once you have familiarized yourself with Protocol Buffers, you should next familiarize yourself with the C++ API for gRPC. In
particular, pay close attention to how the server implementation overrides methods, and the client makes calls to the RPC
service for streaming message types.
You do not need to concern yourself with asynchronous gRPC, we will only be working with the synchronous version in
this project.
Part 1 Structure
All of the part 1 files are available in the part1 directory. You will find several source files in that directory, but you are only
responsible for adjusting and submitting the difslib-* files inside part1 . The rest of the source files provide the
supporting structure for the program. You may change any of the other source files for your testing purposes, but they will
not be submitted as a part of your grade.
In each of the files to be modified, you will find additional instructions and hints on how you should approach the contents
of that file. The following comment marker precedes each tip in the source code:
// STUDENT INSTRUCTION:
Source code file descriptions:
src/dfs-client-p1.[cpp,h] - the CLI executable for the client side.
src/dfs-server-p1.[cpp,h] - the CLI executable for the server side.
src/dfs-clientnode.[cpp,h] - the parent class for the client node library file that you will override. All of the methods
you will override are documented in the dfslib-clientnode-p1.h file you will modify.
src/dfs-utils.h - A header file of utilities used by the executables. You may change this, but note that this file is not
submitted to Bonnie. There is a separate dfs-shared file you may use for your utilities.
dfs-service.proto - TO BE MODIFIED Add your proto buffer service and message types to this file, then run the make
protos command to generate the source.
dfslib-servernode-p1.[cpp,h] - TO BE MODIFIED - Override your gRPC service methods in this file by adding them to
the DFSServerImpl class. The service method signatures can be found in the proto-src/dfs-service.grpc.pb.h file
generated by the make protos command you ran earlier.7/15/2019 pr4/readme.md at master · gios-sum-19/pr4
https://github.gatech.edu/gios-sum-19/pr4/blob/master/readme.md 4/7
dfslib-clientnode-p1.[cpp,h] - TO BE MODIFIED - Add your client-side calls to the gRPC service in this file. We’ve
provided the basic structure and method calls expected by the client executable. However, you may add any additional
declarations and definitions that you deem necessary.
dfslib-shared-p1.[cpp,h] - TO BE MODIFIED - Add any shared code or utilities that you need in this file. The shared
header is available on both the client and server side.
Part 1 Compiling and Running
To compile the source code in Part 1, you may use the Makefile in the root of the repository and run:
make part1
Or, you may change to the part1 directory and run make .
For a list of all make commands available, run make in the root of the repository.
To run the executables, see the usage instructions in their respective files.
In most cases, you'll start the server with:
./bin/dfs-server-p1
The client is then used to fetch, store, list, and stat files. For example:
./bin/dfs-client-p1 fetch gt-campanile.jpg
Part 1 Submitting
To submit part 1, run the following from the root of the repository:
python submit.py part1
Part 2 - Completing the DFS
Now that you have a working gRPC service, we will turn our focus towards completing a rudimentary DFS. For this
assignment, we’ll apply a weakly consistent cache strategy to the RPC calls you already created in Part 1. This is similar to the
approach used by the Andrew File System (AFS). To keep things simple, we’ll focus on whole-file caching for the client-side
and a simple lock strategy on the server-side to ensure that only one client may write to the server at any given time.
Part 2 Goals
For this assignment, your consistency model should adhere to the following expectations:
Whole-file caching. The client should cache whole files on the client-side (i.e., do not concern yourself with partial file
caches). Read and write operations should be applied to local files and only update the server when a file has been
modified or created on the client.
One Creator/Writer per File (i.e., writer locks). Clients should request a file write lock from the server before pushing. If
they are not able to obtain a lock, the attempt should fail. The server should keep track of which client holds a lock to a
particular file, then release that lock after the client has successfully stored a file to the server.
CRC Checksums. To determine if a file has changed between the server and the client, you may use a CRC checksum
function we have provided for you. Any differences in the checksums between the server and the client constitute a
change. The CRC function requires a file path and a table that we’ve already set up for you. The return value will be a
uint32_t value that you can use in your gRPC message types. An example of how to obtain the checksum follows:7/15/2019 pr4/readme.md at master · gios-sum-19/pr4
https://github.gatech.edu/gios-sum-19/pr4/blob/master/readme.md 5/7
std::uint32_t crc = file_checksum(filepath, this->crc_table);
Note that you can copy the code from part1 for your Store, Fetch, List, and Stat methods to part2. However, please note
that you will most likely need to adjust those methods to meet the requirements of the DFS implementation.
Date based Sequences. If a file has changed between the server and the client, then the last modified timestamp should
win. In other words, if the server has a more recent timestamp, then the client should fetch it from the server. If the client
has a more recent timestamp, then the client should store it on the server. If there is no change in the modified time for
a file, the client should do nothing. Keep in mind, when storing data, you must first request a write lock as described
earlier.
Part 2 Structure
The file structure for Part 2 is identical to Part 1. As with Part 1, you are only responsible for adjusting the dfslib-* files in
the part2 directory.
Two threads on the client side will run concurrently. You will need to synchronize these threads and their access to the server.
The watcher thread uses the inotify system commands to monitor client directories for you. We’ve already provided the
structural components to manage the events. However, you will need to make the appropriate changes to ensure that file
notification events coordinate with the synchronization timer described next. An event callback function is provided in the
dfslib-clientnode-p2.cpp file to assist you with that; read the STUDENT INSTRUCTION comments carefully as they provide
some hints on how to manage that process.
The sync thread uses a simple timer to connect with the server every 3 seconds, similar to how NFS connects to a server. The
client should request a list of files from the server, along with their last modified time (mtime), then synchronize the files
between the client and the server. The client should fetch, store, or do nothing based on the goals discussed earlier.
Part 2 Sequence Diagram
A sequence diagram of the expected interactions for part 2 is available in the docs/part2-sequence.pdf file of this repository.
Part 2 Compiling
To compile the source code in Part 2, you may use the Makefile in the root of the repository and run:
make part2
Or, you may change to the part2 directory and run make .
For a list of all make commands available, run make in the root of the repository.
To run the executables, see the usage instructions in their respective files.
In most cases, you'll start the server with something similar to the following:
./bin/dfs-server-p2
The client should then mount to the client path. For example:
./bin/dfs-client-p2 mount
The above command will mount the client to the mnt/client directory, then start the watcher and sync timer threads.
Changes to the mount path should then synchronize to the server and any other clients that have mounted.
As in Part 1, the client should also continue to accept individual commands, such as fetch, store, list, and stat.
Part 2 Submitting7/15/2019 pr4/readme.md at master · gios-sum-19/pr4
https://github.gatech.edu/gios-sum-19/pr4/blob/master/readme.md 6/7
To submit part 2, run the following from the root of the repository:
python submit.py part2
References
Relevant lecture material
P4L1 Remote Procedure Calls
gRPC and Protocol Buffer resources
gRPC C++ Reference
Protocol Buffers 3 Language Guide
gRPC C++ Examples
gRPC C++ Tutorial
Protobuffers Scalar types
gRPC Status Codes
gRPC Deadline
Rubric
Your project will be graded at least on the following items:
Interface specification (.proto)
Service implementation
gRPC initiation and binding
Proper handling of deadline timeouts
Proper clean up of memory and gRPC resources
Proper communication with the server
Proper request and management of write locks
Proper synchronization of files between multiple clients and a single server
Insightful observations in the Readme file and suggestions for improving the class for future semesters
gRPC Implementation (40 points)
Full credit requires: code compiles successfully, does not crash, files fully transmitted, basic safety checks, and proper use of
gRPC - including the ability to get, store, and list files, along with the ability to recognize a timeout. Note that the automated
tests will test some of these automatically, but graders may execute additional tests of these requirements.
DFS Implementation (50 points)
Full credit requires: code compiles successfully, does not crash, files fully transmitted, basic safety checks, proper use of
gRPC, write locks properly handled, cache properly handled, synchronization of sync and inotify threads properly handled,
and synchronization of multiple clients to a single server. Note that the automated tests will test some of these automatically,
but graders may execute additional tests of these requirements.
README (10 points + 5 point extra credit opportunity)
Clearly demonstrates your understanding of what you did and why - we want to see your design and your explanation
of the choices that you made and why you made those choices. (4 points)
A description of the flow of control for your code; we strongly suggest that you use graphics here, but a thorough
textual explanation is sufficient. (2 points)7/15/2019 pr4/readme.md at master · gios-sum-19/pr4
https://github.gatech.edu/gios-sum-19/pr4/blob/master/readme.md 7/7
A brief explanation of how you implemented and tested your code. (2 points)
References any external materials that you consulted during your development process (2 points)
Suggestions on how you would improve the documentation, sample code, testing, or other aspects of the project (up to
5 points extra credit available for noteworthy suggestions here, e.g., actual descriptions of how you would change
things, sample code, code for tests, etc.) We do not give extra credit for simply reporting an issue - we're looking for
actionable suggestions on how to improve things.
Questions
For all questions, please use the class Piazza forum or the class Slack channel so that TA's and other students can assist you.