From b93f2058e643ba7b560fce49b789f9eb5bfa1488 Mon Sep 17 00:00:00 2001 From: Petr Stefan Date: Thu, 24 Nov 2016 13:21:23 +0100 Subject: [PATCH] Typographic fixes --- Assignments.md | 4 ++-- Broker.md | 14 +++++++------- Database.md | 4 ++-- Installation.md | 4 ++-- Introduction.md | 6 +++--- Submission-flow.md | 4 ++-- User-documentation.md | 2 +- Web-API.md | 2 +- Worker.md | 44 +++++++++++++++++++++---------------------- 9 files changed, 42 insertions(+), 42 deletions(-) diff --git a/Assignments.md b/Assignments.md index 5310873..cd520bf 100644 --- a/Assignments.md +++ b/Assignments.md @@ -1,6 +1,6 @@ # Assignments -Assignments are programming tasks that can be tested and evaluated by a worker after user submits his/hers solution. An assignment is described by a YAML file that contains information on how to build, run and test it. One submitted assignment is called a (worker) job. +Assignments are programming tasks that can be tested and evaluated by a worker after user submits a solution. An assignment is described by a YAML file that contains information on how to build, run and test it. One submitted assignment is called a (worker) job. ## Basics @@ -182,7 +182,7 @@ This configuration example is written in YAML and serves only for demonstration submission: # happy hippoes fence job-id: hippoes language: c - file-collector: http://localhost:9999/tasks + file-collector: http://localhost:9999/exercises log: true hw-groups: - group1 diff --git a/Broker.md b/Broker.md index 6af22dd..5c98830 100644 --- a/Broker.md +++ b/Broker.md @@ -20,7 +20,7 @@ The broker's responsibilites are: ## Architecture The broker uses our ZeroMQ _reactor_ to bind events on sockets to handler classes. -There are currently two handlers - one that handles the main functionality and +There are currently two handlers -- one that handles the main functionality and another one that sends status reports to the REST API asynchronously so that the broker does not have to wait for HTTP requests which can take a lot of time, especially when some kind of error happens on the server. @@ -29,7 +29,7 @@ especially when some kind of error happens on the server. The `worker_registry` class is used to store information about workers, their status and the jobs in their queue. It can look up a worker using the headers -received with a request. It also uses a basic load balancing algorithm - the +received with a request. It also uses a basic load balancing algorithm -- the workers are contained in a queue and whenever one of them receives a job, it is moved to the back, which makes it less likely to receive another job soon. @@ -38,8 +38,8 @@ receive a `done` message from it. ### Error handling -**Job failure** - we recognize two ways a job can fail - an internally and -externally. An internal failure is the worker's fault - for example when it +**Job failure** -- we recognize two ways a job can fail -- an internally and +externally. An internal failure is the worker's fault -- for example when it cannot download a file needed for the evaluation for some reason. An external error is for example when the job configuration is malformed. Note that we do not consider a student entering an incorrect solution a job failure. @@ -48,19 +48,19 @@ Jobs that failed internally are reassigned until a limit on the amount of reassingments (configurable with the `max_request_failures` option) is reached. External failures are reported to the frontend immediately. -**Worker failure** - when a worker crash is detected, we attempt to reassign its +**Worker failure** -- when a worker crash is detected, we attempt to reassign its current job and also all the jobs from its queue. Because the current job might be the reason of the crash, its reassignment is also counted towards the `max_request_failures` limit (the counter is shared). If there is no worker that could process a job (i.e. it cannot be reassigned), the job is reported as failed to the frontend via REST API. -**Broker failure** - when the broker itself crashed and is restarted, workers +**Broker failure** -- when the broker itself crashed and is restarted, workers will reconnect automatically. However, all jobs in their queues are lost. If a worker manages to finish a job and notifies the "new" broker, the report is forwarded to the frontend. The same goes for external failures. Jobs that fail internally cannot be reassigned, because the "new" broker does not know their -headers - they are reported as failed immediately. +headers -- they are reported as failed immediately. ## Installation diff --git a/Database.md b/Database.md index daa2502..07b6d49 100644 --- a/Database.md +++ b/Database.md @@ -86,7 +86,7 @@ Group entity contains data about a group. They can be hierarchical, reference to GroupMembership represents relation between group and its members. Also it contains additional data, so this entity is not autogenerated by Doctrine. Essential parts are keys to user and group entities (_user_ and _group_ keys). Additional data: - _status_ -- user status in the group, one of _requested_, _accepted_ and _rejected_ -- _type_ -- type of user access, one of _student_, _supervisor_ and _*_ for all +- _type_ -- type of user access, one of _student_, _supervisor_ and * for all - _requestedAt_ -- date and time the join request was received (or NULL) - _joindedAt_ -- date and time the request for joining was accepted (or NULL) - _rejectedAt_ -- date and time the request for joining was rejected (or NULL) @@ -156,7 +156,7 @@ Permission entity holds permissions for user roles. API endpoints can have a per - _role_ -- key of the role this permission is assigned to - _resource_ -- resource name, usually same for all endpoints in one presenter; from the example above resource is _exercises_ string - _action_ -- action, usually different for each endpoint, wildcard can be - specified as _*_; in the example above, _view-all_ is the action + specified as *; in the example above, _view-all_ is the action - _isAllowed_ -- flag if the rule is allowed of denied ## ReferenceExerciseSolution diff --git a/Installation.md b/Installation.md index 62e0f3d..903b617 100644 --- a/Installation.md +++ b/Installation.md @@ -43,7 +43,7 @@ Configurable variables are saved in _group_vars/all.yml_ file. Syntax is basic k - _mysql_recodex_username_ -- MySQL username for ReCodEx API access. - _mysql_recodex_password_ -- Password for the user above. - _admin_email_ -- Email of administrator. Used when configuring Apache webserver. -- _recodex_hostname -- Hostname where the API and web app will be accessible. For example "recodex.projekty.ms.mff.cuni.cz". +- _recodex_hostname_ -- Hostname where the API and web app will be accessible. For example "recodex.projekty.ms.mff.cuni.cz". - _webapp_node_addr_ -- IP address of NodeJS server running web app. Defaults to "127.0.0.1" and should not be changed. - _webapp_node_port_ -- Port to above. - _webapp_public_addr_ -- Public address, where web server for web app will listen. Defaults to "*". @@ -101,4 +101,4 @@ One of the most important aspects of ReCodEx instance is security. It is crucial - Hide broker, workers and fileserver behind firewall, private subnet or IPsec tunnel. They are not required to be reached from public internet, so it is better keep them isolated. - Keep your server updated and well configured. For automatic installation of security updates on CentOS system refer to `yum-cron` package. Configure SSH and Apache to use only strong ciphers, some recommendations can be found [here](https://bettercrypto.org/static/applied-crypto-hardening.pdf). - Do not put actually used credentials on web, for example do not commit your passwords (in Ansible variables file) on GitHub. -- Regularly check logs for anomalies. \ No newline at end of file +- Regularly check logs for anomalies. diff --git a/Introduction.md b/Introduction.md index f96289e..f9881c3 100644 --- a/Introduction.md +++ b/Introduction.md @@ -55,7 +55,7 @@ Current system is old, but robust. There were no major security incidents during rapid development in web technologies opens new horizons of how web interface can be made. - **web api** -- CodEx offers a very limited XML API based on outdated - technologies that isn't sufficient for users who would like to create custom + technologies that is not sufficient for users who would like to create custom interfaces such as a command line tool or mobile application. - **sandboxing** -- MO-Eval sandbox is based on principle of monitoring system calls and blocking the bad ones. This can be easily done for single-threaded @@ -136,12 +136,12 @@ principles. Modifying CodEx is also not an option -- the required scope of a new solution is too big. To sum up, a new evaluation system has to be written, with only small parts of reused code from CodEx (for example judges). -The new project is **ReCodEx - ReCodEx Code Examiner**. The name should point to +The new project is **ReCodEx -- ReCodEx Code Examiner**. The name should point to CodEx, previous evaluation solution, but also reflect new approach to solve issues. **Re** as part of the name means redesigned, rewritten, renewed or restarted. -Official assignment of the project is available [here](http://www.ksi.mff.cuni.cz/sw-projekty/zadani/recodex.pdf) (only in czech). Most notable features are following: +Official assignment of the project is available [here](http://www.ksi.mff.cuni.cz/sw-projekty/zadani/recodex.pdf) (only in Czech). Most notable features are following: - modern HTML5 web frontend written in Javascript using a suitable framework - REST API implemented in PHP, communicating with database, backend and file server diff --git a/Submission-flow.md b/Submission-flow.md index 9b3513b..7936e33 100644 --- a/Submission-flow.md +++ b/Submission-flow.md @@ -4,7 +4,7 @@ This article will describe in execution flow of submission from the point of sub ## Web Application -First thing user has to submit his/hers solution to web application which provides interface to upload multiple files and then submit them. More detailed description follows: +First thing users have to do is to submit their solutions to web application which provides interface to upload multiple files and then submit them. More detailed description follows: 1. user upload file by file into prepared submit form 2. after uploading all files connected to assignment user clicks on submit button @@ -87,4 +87,4 @@ Web api is notified about job status from broker. After that api is deciding if Web application has only a simple work to do. It has to obtain results from web api. More detailed description follows: 1. web application sends request for results of particular submission -2. if results are available than they are sent back to web application and displayed to user \ No newline at end of file +2. if results are available then they are sent back to web application and displayed to user diff --git a/User-documentation.md b/User-documentation.md index 26d001d..495bcaa 100644 --- a/User-documentation.md +++ b/User-documentation.md @@ -66,7 +66,7 @@ Superadmin is user with the most priviledges. ### Users management -There are only few roles to which users can belong in ReCodEx. Basically there are only three: _student_, _supervisor_, and _superadmin_. Base role is student which is assigned to every registered user. Roles are stored in database alongside other information about user. One user always has only one role at the time. At first startup of ReCodEx administrator should create his/hers account and then change role in database by hand. After that manual intervention into database should never be needed. +There are only few roles to which users can belong in ReCodEx. Basically there are only three: _student_, _supervisor_, and _superadmin_. Base role is student which is assigned to every registered user. Roles are stored in database alongside other information about user. One user always has only one role at the time. At first startup of ReCodEx administrator should create his account and then change role in database by hand. After that manual intervention into database should never be needed. There is a little catch in groups and instances management. Groups can have admins and supervisors. This setting is valid only per one particular group and has to be separated from basic role system. This implies that supervisor in one group can be student in another and simultaneously have global supervisor role. Changing role from student to supervisor and back is done automatically by application and should not be managed by hand in database! Previously stated information can be applied to instances as well, but instances can only have admins. diff --git a/Web-API.md b/Web-API.md index e36cb4a..86eb7d6 100644 --- a/Web-API.md +++ b/Web-API.md @@ -99,7 +99,7 @@ $ composer install ## Configuration and usage -The API can be configured in `config.neon` and `config.local.neon` files in `app/config` directory. The first file is predefined by authors and should not be modified. The second one is not present and could be created from `config.local.neon.example` template in the config directory. Local configuration have higher precedence, so it will override default values from `config.neon`. +The API can be configured in `config.neon` and `config.local.neon` files in `app/config` directory. The first file is predefined by authors and should not be modified. The second one is not present and could be created by copying `config.local.neon.example` template in the config directory. Local configuration have higher precedence, so it will override default values from `config.neon`. ### Configurable items diff --git a/Worker.md b/Worker.md index 9aefac1..7999ea3 100644 --- a/Worker.md +++ b/Worker.md @@ -100,7 +100,7 @@ The worker installation process is composed of following steps: - create config file `/etc/recodex/worker/config-1.yml` - create systemd unit file `/etc/systemd/system/recodex-worker@.service` - put main binary to `/usr/bin/recodex-worker` -- put judges binaries to `/usr/bin/recodex-judge-normal`, `/usr/bin/recodex-judge-shuffle` and `/usr/bin/recodex-judge-filter` +- put judges binaries to `/usr/bin/` directory - create system user and group `recodex` with `/sbin/nologin` shell (if not already existing) - create log directory `/var/log/recodex` - set ownership of config (`/etc/recodex`) and log (`/var/log/recodex`) directories to `recodex` user and group @@ -149,27 +149,27 @@ Worker should have some default configuration which is applied to worker itself Mandatory items are bold, optional italic. -- **worker-id** - unique identification of worker at one server. This id is used by _isolate_ sanbox on linux systems, so make sure to meet isolate's requirements (default is number from 1 to 999). -- **broker-uri** - URI of the broker (hostname, IP address, including port, ...) -- _broker-ping-interval_ - time interval how often to send ping messages to broker. Used units are milliseconds. -- _max-broker-liveness_ - specifies how many pings in a row can broker miss without making the worker dead. -- _headers_ - map of headers specifies worker's capabilities - - _env_ - list of enviromental variables which are sent to broker in init command - - _threads_ - information about available threads for this worker -- **hwgroup** - hardware group of this worker. Hardware group must specify worker hardware and software capabilities and it is main item for broker routing decisions. -- _working-directory_ - where will be stored all needed files. Can be the same for multiple workers on one server. -- **file-managers** - addresses and credentials to all file managers used (eq. all different frontends using this worker) - - **hostname** - URI of file manager - - _username_ - username for http authentication (if needed) - - _password_ - password for http authentication (if needed) -- _file-cache_ - configuration of caching feature - - _cache-dir_ - path to caching directory. Can be the same for multiple workers. -- _logger_ - settings of logging capabilities - - _file_ - path to the logging file with name without suffix. `/var/log/recodex/worker` item will produce `worker.log`, `worker.1.log`, ... - - _level_ - level of logging, one of `off`, `emerg`, `alert`, `critical`, `err`, `warn`, `notice`, `info` and `debug` - - _max-size_ - maximal size of log file before rotating - - _rotations_ - number of rotation kept -- _limits_ - default sandbox limits for this worker. All items are described in assignments section in job configuration description. If some limits are not set in job configuration, defaults from worker config will be used. In such case the worker's defaults will be set as the maximum for the job. Also, limits in job configuration cannot exceed limits from worker. +- **worker-id** -- unique identification of worker at one server. This id is used by _isolate_ sanbox on linux systems, so make sure to meet isolate's requirements (default is number from 1 to 999). +- **broker-uri** -- URI of the broker (hostname, IP address, including port, ...) +- _broker-ping-interval_ -- time interval how often to send ping messages to broker. Used units are milliseconds. +- _max-broker-liveness_ -- specifies how many pings in a row can broker miss without making the worker dead. +- _headers_ -- map of headers specifies worker's capabilities + - _env_ -- list of enviromental variables which are sent to broker in init command + - _threads_ -- information about available threads for this worker +- **hwgroup** -- hardware group of this worker. Hardware group must specify worker hardware and software capabilities and it is main item for broker routing decisions. +- _working-directory_ -- where will be stored all needed files. Can be the same for multiple workers on one server. +- **file-managers** -- addresses and credentials to all file managers used (eq. all different frontends using this worker) + - **hostname** -- URI of file manager + - _username_ -- username for http authentication (if needed) + - _password_ -- password for http authentication (if needed) +- _file-cache_ -- configuration of caching feature + - _cache-dir_ -- path to caching directory. Can be the same for multiple workers. +- _logger_ -- settings of logging capabilities + - _file_ -- path to the logging file with name without suffix. `/var/log/recodex/worker` item will produce `worker.log`, `worker.1.log`, ... + - _level_ -- level of logging, one of `off`, `emerg`, `alert`, `critical`, `err`, `warn`, `notice`, `info` and `debug` + - _max-size_ -- maximal size of log file before rotating + - _rotations_ -- number of rotation kept +- _limits_ -- default sandbox limits for this worker. All items are described in assignments section in job configuration description. If some limits are not set in job configuration, defaults from worker config will be used. In such case the worker's defaults will be set as the maximum for the job. Also, limits in job configuration cannot exceed limits from worker. #### Example config file