From 430c5ecb0c1f6eea641fc2124b785d0c38075c43 Mon Sep 17 00:00:00 2001 From: Petr Stefan Date: Tue, 14 Jun 2016 19:44:21 +0200 Subject: [PATCH] Updated Architecture (markdown) --- Architecture.md | 68 ------------------------------------------------- 1 file changed, 68 deletions(-) diff --git a/Architecture.md b/Architecture.md index a275624..2869b13 100644 --- a/Architecture.md +++ b/Architecture.md @@ -77,71 +77,3 @@ their hashes, which is great for local caching without worries about actual version number of given file. On the other hand, **Database** stores information about human readable names, so that the files are presented in a friendly way to users (teachers) in **WebApp**. - -## Frontend - broker communication - -The communication between the frontend and the workers is mediated by a broker -that passes jobs to workers capable of processing them. - -### Assignment evaluation request - -The frontend must send a multipart message that contains the following frames: - -- The `eval` command -- The job id (in ASCII representation -- we avoid endianness issues and also - support alphabetic ids) -- A frame for each header (e.g. `hwgroup=group_1`) -- An URL of the archive that contains the submitted files and isoeval - configuration -- An URL where the worker should store the result of the evaluation - -If the broker is capable of routing the request to a worker, it responds with -`accept`. Otherwise (for example when the requirements specified by the headers -cannot be met), it responds with `reject`. - -Note that we will need to store the job ID and the assignment configuration -somewhere close to the submitted files so it's possible to check how a -submission was evaluated. The job ID will likely be a part of the submission's -path. The configuration could be linked there under some well-known name. - -### Notifying the frontend about evaluation progress - -The script that requested the evaluation will have exited by the time a worker -processes the request. This issue remains to be resolved. - -## Broker - worker communication - -When a worker is started, it registers itself with the broker by sending the -`init` command followed by its hardware group and headers that describe its -capabilities (such as the number of threads it can run simultaneously, -languages it can work with...). The headers are expected to be in following -format: `header_name=value`. Every header shall be in a separate frame. - -Whenever the broker receives an assignment suitable for the worker, it just -forwards the evaluation request message it originally received from the -frontend. The worker has to: - -- Download the archive containing the submission and an isoeval configuration - file -- Download any supplementary files based on the configuration file, such as test - inputs or helper programs (This can be done on demand, using a `fetch` command - in the assignment configuration) -- Download the source codes of the student's submission -- Evaluate the submission according to the assignment's configuration -- Upload the results of the evaluation to the file server -- Notify the broker that the evaluation is finished - -Thanks to this message structure, it's possible to cache the configuration file -and only download the student's submissions when the same assignment is -evaluated repeatedly for different students (a common case for homeworks and -classroom assignments). - -After finishing the evaluation, worker notifies the broker of this fact by -sending: - -- The `done` command -- The job id - -This allows the broker to reliably distribute messages - if a worker doesn't -succeed in processing a request (it doesn't respond in a time limit), the -request can be sent to another worker.