**Worker's** main role is securely _compile_, _run_ and _evaluate_ given submit against model solutions provided by author of each task. It is logicaly divided into three objects:
- **Message Frontend** communicates with **WebApp** using messaging queue [ZeroMQ](http://zeromq.org/). It receives new submits, operates the evaluation through **Work API** and reports progress back.
- **Worker Core** can do all evaluating steps and is responsible for security of them. Sandbox [Isolate](https://github.com/ioi/isolate) is used.
- **File Server Frontend** ensures via **File API** access to files on **File Server**, where are stored testing inputs and corresponding outputs for each task and other required files. It's possible to upload files, too.
- **File Server Frontend** provides access to files on **File Server** via
**File API**, where testing inputs and corresponding outputs for each task and
other required files are stored. It's possible to upload files, too.
### Default worker configuration
Worker should have some default configuration which is applied to worker itself or may be used in given jobs (implicitly if something is missing, or explicitly with special variables). This configuration should be hardcoded and can be rewritten by explicitly declared configuration file. Format of this configuration is yaml like in the job config.
@ -71,8 +73,11 @@ To meet these requirements, **Storage** and **Database** must be set as bellow.
├── submits
│ └── user_id
│ └── advanced_dot_net_1
│ ├── bf216fa9274261628f4d952a103c6cfd1cbbc587
│ └── e6ae49bbfda4a8bb57aceeb64fb117990b226ca5
│ └── submit_id
│ ├── eval.yml
│ └── source.cs
├── submit_archives
│ └── submit_id.tar.gz
├── tasks
│ ├── a
│ │ ├── a014ed2abb56371bfaf2b4298a85d5dfb56509ed
@ -86,10 +91,11 @@ To meet these requirements, **Storage** and **Database** must be set as bellow.
There are subdirectories _user_id_ and _advanced_dot_net_1_ which groups
submits by users and courses the submits are for. This structure is easy to
maintain for new and deleted users.
- **tasks** contains all files for all existing task in ReCodEx. To avoid too
many files in one directory, files are separated to subfolders by first
character of their name.
- **submit_archives** contains the student submissions in compressed archives so
that they can be easily downloaded by workers.
- **tasks** contains supplementary files (such as test inputs or helper
programs) for all existing task in ReCodEx. To avoid too many files in one
directory, files are separated to subfolders by first character of their name.
- **temp** directory is dedicated to temporary storing outputs of programs on teachers' demand. This directory will be erased by cron job on daily basis.
### Database
@ -117,11 +123,12 @@ that passes jobs to workers capable of processing them.
The frontend must send a multipart message that contains the following frames:
- The `eval` command
- The job id (ASCII or network byte order - to be specified)
- The job id (in ASCII representation -- we avoid endianness issues and also
support alphabetic ids)
- A frame for each header (e.g. `hwgroup=group_1`)
- An URL of the file server where the submitted files are stored
-A hash code of the assignment's configuration file
- Hash codes of files submitted by the user, each in a separate frame
- An URL of the archive that contains the submitted files and isoeval
configuration
- An URL where the worker should store the result of the evaluation
If the broker is capable of routing the request to a worker, it responds with
`accept`. Otherwise (for example when the requirements specified by the headers
@ -148,7 +155,8 @@ Whenever the broker receives an assignment suitable for the worker, it just
forwards the evaluation request message it originally received from the
frontend. The worker has to:
- Download the assignment configuration file
- Download the archive containing the submission and an isoeval configuration
file
- Download any supplementary files based on the configuration file, such as test
inputs or helper programs (This can be done on demand, using a `fetch` command
Teyras
9 years ago