Typographic fixes

master
Petr Stefan 8 years ago
parent b3964943f2
commit b93f2058e6

@ -1,6 +1,6 @@
# Assignments # Assignments
Assignments are programming tasks that can be tested and evaluated by a worker after user submits his/hers solution. An assignment is described by a YAML file that contains information on how to build, run and test it. One submitted assignment is called a (worker) job. Assignments are programming tasks that can be tested and evaluated by a worker after user submits a solution. An assignment is described by a YAML file that contains information on how to build, run and test it. One submitted assignment is called a (worker) job.
## Basics ## Basics
@ -182,7 +182,7 @@ This configuration example is written in YAML and serves only for demonstration
submission: # happy hippoes fence submission: # happy hippoes fence
job-id: hippoes job-id: hippoes
language: c language: c
file-collector: http://localhost:9999/tasks file-collector: http://localhost:9999/exercises
log: true log: true
hw-groups: hw-groups:
- group1 - group1

@ -20,7 +20,7 @@ The broker's responsibilites are:
## Architecture ## Architecture
The broker uses our ZeroMQ _reactor_ to bind events on sockets to handler classes. The broker uses our ZeroMQ _reactor_ to bind events on sockets to handler classes.
There are currently two handlers - one that handles the main functionality and There are currently two handlers -- one that handles the main functionality and
another one that sends status reports to the REST API asynchronously so that the another one that sends status reports to the REST API asynchronously so that the
broker does not have to wait for HTTP requests which can take a lot of time, broker does not have to wait for HTTP requests which can take a lot of time,
especially when some kind of error happens on the server. especially when some kind of error happens on the server.
@ -29,7 +29,7 @@ especially when some kind of error happens on the server.
The `worker_registry` class is used to store information about workers, their The `worker_registry` class is used to store information about workers, their
status and the jobs in their queue. It can look up a worker using the headers status and the jobs in their queue. It can look up a worker using the headers
received with a request. It also uses a basic load balancing algorithm - the received with a request. It also uses a basic load balancing algorithm -- the
workers are contained in a queue and whenever one of them receives a job, it is workers are contained in a queue and whenever one of them receives a job, it is
moved to the back, which makes it less likely to receive another job soon. moved to the back, which makes it less likely to receive another job soon.
@ -38,8 +38,8 @@ receive a `done` message from it.
### Error handling ### Error handling
**Job failure** - we recognize two ways a job can fail - an internally and **Job failure** -- we recognize two ways a job can fail -- an internally and
externally. An internal failure is the worker's fault - for example when it externally. An internal failure is the worker's fault -- for example when it
cannot download a file needed for the evaluation for some reason. An external cannot download a file needed for the evaluation for some reason. An external
error is for example when the job configuration is malformed. Note that we do not error is for example when the job configuration is malformed. Note that we do not
consider a student entering an incorrect solution a job failure. consider a student entering an incorrect solution a job failure.
@ -48,19 +48,19 @@ Jobs that failed internally are reassigned until a limit on the amount of
reassingments (configurable with the `max_request_failures` option) is reached. reassingments (configurable with the `max_request_failures` option) is reached.
External failures are reported to the frontend immediately. External failures are reported to the frontend immediately.
**Worker failure** - when a worker crash is detected, we attempt to reassign its **Worker failure** -- when a worker crash is detected, we attempt to reassign its
current job and also all the jobs from its queue. Because the current job might current job and also all the jobs from its queue. Because the current job might
be the reason of the crash, its reassignment is also counted towards the be the reason of the crash, its reassignment is also counted towards the
`max_request_failures` limit (the counter is shared). If there is no worker that `max_request_failures` limit (the counter is shared). If there is no worker that
could process a job (i.e. it cannot be reassigned), the job is reported as could process a job (i.e. it cannot be reassigned), the job is reported as
failed to the frontend via REST API. failed to the frontend via REST API.
**Broker failure** - when the broker itself crashed and is restarted, workers **Broker failure** -- when the broker itself crashed and is restarted, workers
will reconnect automatically. However, all jobs in their queues are lost. If a will reconnect automatically. However, all jobs in their queues are lost. If a
worker manages to finish a job and notifies the "new" broker, the report is worker manages to finish a job and notifies the "new" broker, the report is
forwarded to the frontend. The same goes for external failures. Jobs that fail forwarded to the frontend. The same goes for external failures. Jobs that fail
internally cannot be reassigned, because the "new" broker does not know their internally cannot be reassigned, because the "new" broker does not know their
headers - they are reported as failed immediately. headers -- they are reported as failed immediately.
## Installation ## Installation

@ -86,7 +86,7 @@ Group entity contains data about a group. They can be hierarchical, reference to
GroupMembership represents relation between group and its members. Also it contains additional data, so this entity is not autogenerated by Doctrine. Essential parts are keys to user and group entities (_user_ and _group_ keys). Additional data: GroupMembership represents relation between group and its members. Also it contains additional data, so this entity is not autogenerated by Doctrine. Essential parts are keys to user and group entities (_user_ and _group_ keys). Additional data:
- _status_ -- user status in the group, one of _requested_, _accepted_ and _rejected_ - _status_ -- user status in the group, one of _requested_, _accepted_ and _rejected_
- _type_ -- type of user access, one of _student_, _supervisor_ and _*_ for all - _type_ -- type of user access, one of _student_, _supervisor_ and * for all
- _requestedAt_ -- date and time the join request was received (or NULL) - _requestedAt_ -- date and time the join request was received (or NULL)
- _joindedAt_ -- date and time the request for joining was accepted (or NULL) - _joindedAt_ -- date and time the request for joining was accepted (or NULL)
- _rejectedAt_ -- date and time the request for joining was rejected (or NULL) - _rejectedAt_ -- date and time the request for joining was rejected (or NULL)
@ -156,7 +156,7 @@ Permission entity holds permissions for user roles. API endpoints can have a per
- _role_ -- key of the role this permission is assigned to - _role_ -- key of the role this permission is assigned to
- _resource_ -- resource name, usually same for all endpoints in one presenter; from the example above resource is _exercises_ string - _resource_ -- resource name, usually same for all endpoints in one presenter; from the example above resource is _exercises_ string
- _action_ -- action, usually different for each endpoint, wildcard can be - _action_ -- action, usually different for each endpoint, wildcard can be
specified as _*_; in the example above, _view-all_ is the action specified as *; in the example above, _view-all_ is the action
- _isAllowed_ -- flag if the rule is allowed of denied - _isAllowed_ -- flag if the rule is allowed of denied
## ReferenceExerciseSolution ## ReferenceExerciseSolution

@ -43,7 +43,7 @@ Configurable variables are saved in _group_vars/all.yml_ file. Syntax is basic k
- _mysql_recodex_username_ -- MySQL username for ReCodEx API access. - _mysql_recodex_username_ -- MySQL username for ReCodEx API access.
- _mysql_recodex_password_ -- Password for the user above. - _mysql_recodex_password_ -- Password for the user above.
- _admin_email_ -- Email of administrator. Used when configuring Apache webserver. - _admin_email_ -- Email of administrator. Used when configuring Apache webserver.
- _recodex_hostname -- Hostname where the API and web app will be accessible. For example "recodex.projekty.ms.mff.cuni.cz". - _recodex_hostname_ -- Hostname where the API and web app will be accessible. For example "recodex.projekty.ms.mff.cuni.cz".
- _webapp_node_addr_ -- IP address of NodeJS server running web app. Defaults to "127.0.0.1" and should not be changed. - _webapp_node_addr_ -- IP address of NodeJS server running web app. Defaults to "127.0.0.1" and should not be changed.
- _webapp_node_port_ -- Port to above. - _webapp_node_port_ -- Port to above.
- _webapp_public_addr_ -- Public address, where web server for web app will listen. Defaults to "*". - _webapp_public_addr_ -- Public address, where web server for web app will listen. Defaults to "*".
@ -101,4 +101,4 @@ One of the most important aspects of ReCodEx instance is security. It is crucial
- Hide broker, workers and fileserver behind firewall, private subnet or IPsec tunnel. They are not required to be reached from public internet, so it is better keep them isolated. - Hide broker, workers and fileserver behind firewall, private subnet or IPsec tunnel. They are not required to be reached from public internet, so it is better keep them isolated.
- Keep your server updated and well configured. For automatic installation of security updates on CentOS system refer to `yum-cron` package. Configure SSH and Apache to use only strong ciphers, some recommendations can be found [here](https://bettercrypto.org/static/applied-crypto-hardening.pdf). - Keep your server updated and well configured. For automatic installation of security updates on CentOS system refer to `yum-cron` package. Configure SSH and Apache to use only strong ciphers, some recommendations can be found [here](https://bettercrypto.org/static/applied-crypto-hardening.pdf).
- Do not put actually used credentials on web, for example do not commit your passwords (in Ansible variables file) on GitHub. - Do not put actually used credentials on web, for example do not commit your passwords (in Ansible variables file) on GitHub.
- Regularly check logs for anomalies. - Regularly check logs for anomalies.

@ -55,7 +55,7 @@ Current system is old, but robust. There were no major security incidents during
rapid development in web technologies opens new horizons of how web interface rapid development in web technologies opens new horizons of how web interface
can be made. can be made.
- **web api** -- CodEx offers a very limited XML API based on outdated - **web api** -- CodEx offers a very limited XML API based on outdated
technologies that isn't sufficient for users who would like to create custom technologies that is not sufficient for users who would like to create custom
interfaces such as a command line tool or mobile application. interfaces such as a command line tool or mobile application.
- **sandboxing** -- MO-Eval sandbox is based on principle of monitoring system - **sandboxing** -- MO-Eval sandbox is based on principle of monitoring system
calls and blocking the bad ones. This can be easily done for single-threaded calls and blocking the bad ones. This can be easily done for single-threaded
@ -136,12 +136,12 @@ principles. Modifying CodEx is also not an option -- the required scope of a new
solution is too big. To sum up, a new evaluation system has to be written, with solution is too big. To sum up, a new evaluation system has to be written, with
only small parts of reused code from CodEx (for example judges). only small parts of reused code from CodEx (for example judges).
The new project is **ReCodEx - ReCodEx Code Examiner**. The name should point to The new project is **ReCodEx -- ReCodEx Code Examiner**. The name should point to
CodEx, previous evaluation solution, but also reflect new approach to solve CodEx, previous evaluation solution, but also reflect new approach to solve
issues. **Re** as part of the name means redesigned, rewritten, renewed or issues. **Re** as part of the name means redesigned, rewritten, renewed or
restarted. restarted.
Official assignment of the project is available [here](http://www.ksi.mff.cuni.cz/sw-projekty/zadani/recodex.pdf) (only in czech). Most notable features are following: Official assignment of the project is available [here](http://www.ksi.mff.cuni.cz/sw-projekty/zadani/recodex.pdf) (only in Czech). Most notable features are following:
- modern HTML5 web frontend written in Javascript using a suitable framework - modern HTML5 web frontend written in Javascript using a suitable framework
- REST API implemented in PHP, communicating with database, backend and file server - REST API implemented in PHP, communicating with database, backend and file server

@ -4,7 +4,7 @@ This article will describe in execution flow of submission from the point of sub
## Web Application ## Web Application
First thing user has to submit his/hers solution to web application which provides interface to upload multiple files and then submit them. More detailed description follows: First thing users have to do is to submit their solutions to web application which provides interface to upload multiple files and then submit them. More detailed description follows:
1. user upload file by file into prepared submit form 1. user upload file by file into prepared submit form
2. after uploading all files connected to assignment user clicks on submit button 2. after uploading all files connected to assignment user clicks on submit button
@ -87,4 +87,4 @@ Web api is notified about job status from broker. After that api is deciding if
Web application has only a simple work to do. It has to obtain results from web api. More detailed description follows: Web application has only a simple work to do. It has to obtain results from web api. More detailed description follows:
1. web application sends request for results of particular submission 1. web application sends request for results of particular submission
2. if results are available than they are sent back to web application and displayed to user 2. if results are available then they are sent back to web application and displayed to user

@ -66,7 +66,7 @@ Superadmin is user with the most priviledges.
### Users management ### Users management
There are only few roles to which users can belong in ReCodEx. Basically there are only three: _student_, _supervisor_, and _superadmin_. Base role is student which is assigned to every registered user. Roles are stored in database alongside other information about user. One user always has only one role at the time. At first startup of ReCodEx administrator should create his/hers account and then change role in database by hand. After that manual intervention into database should never be needed. There are only few roles to which users can belong in ReCodEx. Basically there are only three: _student_, _supervisor_, and _superadmin_. Base role is student which is assigned to every registered user. Roles are stored in database alongside other information about user. One user always has only one role at the time. At first startup of ReCodEx administrator should create his account and then change role in database by hand. After that manual intervention into database should never be needed.
There is a little catch in groups and instances management. Groups can have admins and supervisors. This setting is valid only per one particular group and has to be separated from basic role system. This implies that supervisor in one group can be student in another and simultaneously have global supervisor role. Changing role from student to supervisor and back is done automatically by application and should not be managed by hand in database! Previously stated information can be applied to instances as well, but instances can only have admins. There is a little catch in groups and instances management. Groups can have admins and supervisors. This setting is valid only per one particular group and has to be separated from basic role system. This implies that supervisor in one group can be student in another and simultaneously have global supervisor role. Changing role from student to supervisor and back is done automatically by application and should not be managed by hand in database! Previously stated information can be applied to instances as well, but instances can only have admins.

@ -99,7 +99,7 @@ $ composer install
## Configuration and usage ## Configuration and usage
The API can be configured in `config.neon` and `config.local.neon` files in `app/config` directory. The first file is predefined by authors and should not be modified. The second one is not present and could be created from `config.local.neon.example` template in the config directory. Local configuration have higher precedence, so it will override default values from `config.neon`. The API can be configured in `config.neon` and `config.local.neon` files in `app/config` directory. The first file is predefined by authors and should not be modified. The second one is not present and could be created by copying `config.local.neon.example` template in the config directory. Local configuration have higher precedence, so it will override default values from `config.neon`.
### Configurable items ### Configurable items

@ -100,7 +100,7 @@ The worker installation process is composed of following steps:
- create config file `/etc/recodex/worker/config-1.yml` - create config file `/etc/recodex/worker/config-1.yml`
- create systemd unit file `/etc/systemd/system/recodex-worker@.service` - create systemd unit file `/etc/systemd/system/recodex-worker@.service`
- put main binary to `/usr/bin/recodex-worker` - put main binary to `/usr/bin/recodex-worker`
- put judges binaries to `/usr/bin/recodex-judge-normal`, `/usr/bin/recodex-judge-shuffle` and `/usr/bin/recodex-judge-filter` - put judges binaries to `/usr/bin/` directory
- create system user and group `recodex` with `/sbin/nologin` shell (if not already existing) - create system user and group `recodex` with `/sbin/nologin` shell (if not already existing)
- create log directory `/var/log/recodex` - create log directory `/var/log/recodex`
- set ownership of config (`/etc/recodex`) and log (`/var/log/recodex`) directories to `recodex` user and group - set ownership of config (`/etc/recodex`) and log (`/var/log/recodex`) directories to `recodex` user and group
@ -149,27 +149,27 @@ Worker should have some default configuration which is applied to worker itself
Mandatory items are bold, optional italic. Mandatory items are bold, optional italic.
- **worker-id** - unique identification of worker at one server. This id is used by _isolate_ sanbox on linux systems, so make sure to meet isolate's requirements (default is number from 1 to 999). - **worker-id** -- unique identification of worker at one server. This id is used by _isolate_ sanbox on linux systems, so make sure to meet isolate's requirements (default is number from 1 to 999).
- **broker-uri** - URI of the broker (hostname, IP address, including port, ...) - **broker-uri** -- URI of the broker (hostname, IP address, including port, ...)
- _broker-ping-interval_ - time interval how often to send ping messages to broker. Used units are milliseconds. - _broker-ping-interval_ -- time interval how often to send ping messages to broker. Used units are milliseconds.
- _max-broker-liveness_ - specifies how many pings in a row can broker miss without making the worker dead. - _max-broker-liveness_ -- specifies how many pings in a row can broker miss without making the worker dead.
- _headers_ - map of headers specifies worker's capabilities - _headers_ -- map of headers specifies worker's capabilities
- _env_ - list of enviromental variables which are sent to broker in init command - _env_ -- list of enviromental variables which are sent to broker in init command
- _threads_ - information about available threads for this worker - _threads_ -- information about available threads for this worker
- **hwgroup** - hardware group of this worker. Hardware group must specify worker hardware and software capabilities and it is main item for broker routing decisions. - **hwgroup** -- hardware group of this worker. Hardware group must specify worker hardware and software capabilities and it is main item for broker routing decisions.
- _working-directory_ - where will be stored all needed files. Can be the same for multiple workers on one server. - _working-directory_ -- where will be stored all needed files. Can be the same for multiple workers on one server.
- **file-managers** - addresses and credentials to all file managers used (eq. all different frontends using this worker) - **file-managers** -- addresses and credentials to all file managers used (eq. all different frontends using this worker)
- **hostname** - URI of file manager - **hostname** -- URI of file manager
- _username_ - username for http authentication (if needed) - _username_ -- username for http authentication (if needed)
- _password_ - password for http authentication (if needed) - _password_ -- password for http authentication (if needed)
- _file-cache_ - configuration of caching feature - _file-cache_ -- configuration of caching feature
- _cache-dir_ - path to caching directory. Can be the same for multiple workers. - _cache-dir_ -- path to caching directory. Can be the same for multiple workers.
- _logger_ - settings of logging capabilities - _logger_ -- settings of logging capabilities
- _file_ - path to the logging file with name without suffix. `/var/log/recodex/worker` item will produce `worker.log`, `worker.1.log`, ... - _file_ -- path to the logging file with name without suffix. `/var/log/recodex/worker` item will produce `worker.log`, `worker.1.log`, ...
- _level_ - level of logging, one of `off`, `emerg`, `alert`, `critical`, `err`, `warn`, `notice`, `info` and `debug` - _level_ -- level of logging, one of `off`, `emerg`, `alert`, `critical`, `err`, `warn`, `notice`, `info` and `debug`
- _max-size_ - maximal size of log file before rotating - _max-size_ -- maximal size of log file before rotating
- _rotations_ - number of rotation kept - _rotations_ -- number of rotation kept
- _limits_ - default sandbox limits for this worker. All items are described in assignments section in job configuration description. If some limits are not set in job configuration, defaults from worker config will be used. In such case the worker's defaults will be set as the maximum for the job. Also, limits in job configuration cannot exceed limits from worker. - _limits_ -- default sandbox limits for this worker. All items are described in assignments section in job configuration description. If some limits are not set in job configuration, defaults from worker config will be used. In such case the worker's defaults will be set as the maximum for the job. Also, limits in job configuration cannot exceed limits from worker.
#### Example config file #### Example config file

Loading…
Cancel
Save