Skeleton of communication description
parent
430c5ecb0c
commit
f4ce641714
@ -0,0 +1,87 @@
|
||||
# Communication
|
||||
|
||||
This section gives detailed overview about communication in ReCodEx solution. Basic concept is captured on following image:
|
||||
|
||||
![Communication Img](https://github.com/ReCodEx/GlobalWiki/raw/master/images/Backend_Connections.png)
|
||||
|
||||
Red connections are through ZeroMQ sockets, Blue are through WebSockets and Green are through HTTP.
|
||||
|
||||
## Internal worker communication
|
||||
|
||||
## Broker - Worker communication
|
||||
|
||||
When a worker is started, it registers itself with the broker by sending the
|
||||
`init` command followed by its hardware group and headers that describe its
|
||||
capabilities (such as the number of threads it can run simultaneously,
|
||||
languages it can work with...). The headers are expected to be in following
|
||||
format: `header_name=value`. Every header shall be in a separate frame.
|
||||
|
||||
Whenever the broker receives an assignment suitable for the worker, it just
|
||||
forwards the evaluation request message it originally received from the
|
||||
frontend. The worker has to:
|
||||
|
||||
- Download the archive containing the submission and an isoeval configuration
|
||||
file
|
||||
- Download any supplementary files based on the configuration file, such as test
|
||||
inputs or helper programs (This can be done on demand, using a `fetch` command
|
||||
in the assignment configuration)
|
||||
- Download the source codes of the student's submission
|
||||
- Evaluate the submission according to the assignment's configuration
|
||||
- Upload the results of the evaluation to the file server
|
||||
- Notify the broker that the evaluation is finished
|
||||
|
||||
Thanks to this message structure, it's possible to cache the configuration file
|
||||
and only download the student's submissions when the same assignment is
|
||||
evaluated repeatedly for different students (a common case for homeworks and
|
||||
classroom assignments).
|
||||
|
||||
After finishing the evaluation, worker notifies the broker of this fact by
|
||||
sending:
|
||||
|
||||
- The `done` command
|
||||
- The job id
|
||||
|
||||
This allows the broker to reliably distribute messages - if a worker doesn't
|
||||
succeed in processing a request (it doesn't respond in a time limit), the
|
||||
request can be sent to another worker.
|
||||
|
||||
## Worker - File Server communication
|
||||
|
||||
## Broker - Monitor communication
|
||||
|
||||
## Broker - Frontend communication
|
||||
|
||||
The communication between the frontend and the workers is mediated by a broker
|
||||
that passes jobs to workers capable of processing them.
|
||||
|
||||
### Assignment evaluation request
|
||||
|
||||
The frontend must send a multipart message that contains the following frames:
|
||||
|
||||
- The `eval` command
|
||||
- The job id (in ASCII representation -- we avoid endianness issues and also
|
||||
support alphabetic ids)
|
||||
- A frame for each header (e.g. `hwgroup=group_1`)
|
||||
- An URL of the archive that contains the submitted files and isoeval
|
||||
configuration
|
||||
- An URL where the worker should store the result of the evaluation
|
||||
|
||||
If the broker is capable of routing the request to a worker, it responds with
|
||||
`accept`. Otherwise (for example when the requirements specified by the headers
|
||||
cannot be met), it responds with `reject`.
|
||||
|
||||
Note that we will need to store the job ID and the assignment configuration
|
||||
somewhere close to the submitted files so it's possible to check how a
|
||||
submission was evaluated. The job ID will likely be a part of the submission's
|
||||
path. The configuration could be linked there under some well-known name.
|
||||
|
||||
### Notifying the frontend about evaluation progress
|
||||
|
||||
The script that requested the evaluation will have exited by the time a worker
|
||||
processes the request. This issue remains to be resolved.
|
||||
|
||||
## File Server - Frontend communication
|
||||
|
||||
## Monitor - Browser communication
|
||||
|
||||
## Frontend - Browser communication
|
Loading…
Reference in New Issue