Updated Architecture (markdown)

master
Petr Stefan 8 years ago
parent ef3367ff36
commit 430c5ecb0c

@ -77,71 +77,3 @@ their hashes, which is great for local caching without worries about actual
version number of given file. On the other hand, **Database** stores information version number of given file. On the other hand, **Database** stores information
about human readable names, so that the files are presented in a friendly way to about human readable names, so that the files are presented in a friendly way to
users (teachers) in **WebApp**. users (teachers) in **WebApp**.
## Frontend - broker communication
The communication between the frontend and the workers is mediated by a broker
that passes jobs to workers capable of processing them.
### Assignment evaluation request
The frontend must send a multipart message that contains the following frames:
- The `eval` command
- The job id (in ASCII representation -- we avoid endianness issues and also
support alphabetic ids)
- A frame for each header (e.g. `hwgroup=group_1`)
- An URL of the archive that contains the submitted files and isoeval
configuration
- An URL where the worker should store the result of the evaluation
If the broker is capable of routing the request to a worker, it responds with
`accept`. Otherwise (for example when the requirements specified by the headers
cannot be met), it responds with `reject`.
Note that we will need to store the job ID and the assignment configuration
somewhere close to the submitted files so it's possible to check how a
submission was evaluated. The job ID will likely be a part of the submission's
path. The configuration could be linked there under some well-known name.
### Notifying the frontend about evaluation progress
The script that requested the evaluation will have exited by the time a worker
processes the request. This issue remains to be resolved.
## Broker - worker communication
When a worker is started, it registers itself with the broker by sending the
`init` command followed by its hardware group and headers that describe its
capabilities (such as the number of threads it can run simultaneously,
languages it can work with...). The headers are expected to be in following
format: `header_name=value`. Every header shall be in a separate frame.
Whenever the broker receives an assignment suitable for the worker, it just
forwards the evaluation request message it originally received from the
frontend. The worker has to:
- Download the archive containing the submission and an isoeval configuration
file
- Download any supplementary files based on the configuration file, such as test
inputs or helper programs (This can be done on demand, using a `fetch` command
in the assignment configuration)
- Download the source codes of the student's submission
- Evaluate the submission according to the assignment's configuration
- Upload the results of the evaluation to the file server
- Notify the broker that the evaluation is finished
Thanks to this message structure, it's possible to cache the configuration file
and only download the student's submissions when the same assignment is
evaluated repeatedly for different students (a common case for homeworks and
classroom assignments).
After finishing the evaluation, worker notifies the broker of this fact by
sending:
- The `done` command
- The job id
This allows the broker to reliably distribute messages - if a worker doesn't
succeed in processing a request (it doesn't respond in a time limit), the
request can be sent to another worker.

Loading…
Cancel
Save