notes on communication between the components

master
Teyras 9 years ago
parent 3c235dd96e
commit 54d3801eca

@ -87,4 +87,61 @@ For user friendly access and modifying tasks following information should be sto
- for every hash name one human readable filename - for every hash name one human readable filename
### Conclusion ### Conclusion
Files are internally stored by their `sha1sum` hashes, so it's easy to implement versioning and get rid of files with duplicity content. **Worker** also uses files by their hashes, which is great for local caching without worries about actual version number of given file. On the other hand, **Database** keeps info about human readable names, so to users (teachers) in **WebApp** are files presented in a friendly way. Files are internally stored by their `sha1sum` hashes, so it's easy to implement versioning and get rid of files with duplicity content. **Worker** also uses files by their hashes, which is great for local caching without worries about actual version number of given file. On the other hand, **Database** keeps info about human readable names, so to users (teachers) in **WebApp** are files presented in a friendly way.
## Frontend - broker communication
The communication between the frontend and the workers is mediated by a broker
that passes jobs to workers capable of processing them.
### Assignment evaluation request
The frontend must send a multipart message that contains the following frames:
- The `eval` command
- The job id (ASCII or network byte order - to be specified)
- A frame for each header (e.g. `hwgroup=group_1`)
- A hash code of the assignment's configuration file
- Hash codes of files submitted by the user, each in a separate frame
If the broker is capable of routing the request to a worker, it responds with
`ack`. Otherwise (for example when the requirements specified by the headers
cannot be met), it responds with `nack`.
### Notifying the frontend about evaluation progress
The script that requested the evaluation will have exited by the time a worker
processes the request. This issue remains to be resolved.
## Broker - worker communication
When a worker is started, it registers itself with the broker by sending the
`init` command followed by headers that describe its capabilities (such as the
number of threads it can run simultaneously, its hardware group, languages it
can work with...).
Whenever the broker receives an assignment suitable for the worker, it just
forwards the evaluation request message it originally received from the
frontend. The worker has to:
- Download the assignment configuration file
- Download any supplementary files based on the configuration file, such as test
inputs or helper programs
- Download the source codes of the student's submission
- Evaluate the submission according to the assignment's configuration
- Upload the results of the evaluation to the file server
Thanks to this message structure, it's possible to cache the configuration file
and only download the student's submissions when the same assignment is
evaluated repeatedly for different students (a common case for homeworks and
classroom assignments).
After finishing the evaluation, worker notifies the broker of this fact by
sending:
- The `done` command
- The job id
This allows the broker to reliably distribute messages - if a worker doesn't
succeed in processing a request (it doesn't respond in a time limit), the
request can be sent to another worker.

Loading…
Cancel
Save