|
|
|
@ -1,17 +1,20 @@
|
|
|
|
|
# System configuration
|
|
|
|
|
|
|
|
|
|
This section describes configuration of ReCodEx components. Bold items in lists
|
|
|
|
|
describing the values are mandatory, italic ones are optional.
|
|
|
|
|
|
|
|
|
|
## Worker
|
|
|
|
|
|
|
|
|
|
Worker should have some default configuration which is applied to worker itself
|
|
|
|
|
or may be used in given jobs (implicitly if something is missing, or explicitly
|
|
|
|
|
with special variables). This configuration should be hardcoded and can be
|
|
|
|
|
Worker have a default configuration which is applied to worker itself or is used
|
|
|
|
|
in given jobs (implicitly if something is missing, or explicitly with special
|
|
|
|
|
variables). This configuration is hardcoded in worker sources and can be
|
|
|
|
|
rewritten by explicitly declared configuration file. Format of this
|
|
|
|
|
configuration is yaml with similar structure to job configuration.
|
|
|
|
|
configuration is YAML file with similar structure as job configuration. The
|
|
|
|
|
default location is `/etc/recodex/worker/config-<N>.yml` where `N` is identifier
|
|
|
|
|
of the particular worker instance.
|
|
|
|
|
|
|
|
|
|
### Configuration items
|
|
|
|
|
|
|
|
|
|
Mandatory items are bold, optional italic.
|
|
|
|
|
|
|
|
|
|
- **worker-id** -- unique identification of worker at one server. This id is
|
|
|
|
|
used by _isolate_ sanbox on linux systems, so make sure to meet isolate's
|
|
|
|
|
requirements (default is number from 1 to 999).
|
|
|
|
@ -45,7 +48,7 @@ Mandatory items are bold, optional italic.
|
|
|
|
|
...
|
|
|
|
|
- _level_ -- level of logging, one of `off`, `emerg`, `alert`, `critical`,
|
|
|
|
|
`err`, `warn`, `notice`, `info` and `debug`
|
|
|
|
|
- _max-size_ -- maximal size of log file before rotating
|
|
|
|
|
- _max-size_ -- maximal size of log file before rotating in bytes
|
|
|
|
|
- _rotations_ -- number of rotation kept
|
|
|
|
|
- _limits_ -- default sandbox limits for this worker. All items are described in
|
|
|
|
|
assignments section in job configuration description. If some limits are not
|
|
|
|
@ -99,17 +102,17 @@ limits:
|
|
|
|
|
|
|
|
|
|
### Isolate sandbox
|
|
|
|
|
|
|
|
|
|
New feature in version 1.3 is possibility of limit Isolate box to one or more
|
|
|
|
|
cpu or memory node. This functionality is provided by _cpusets_ kernel mechanism
|
|
|
|
|
and is now integrated in isolate. It is allowed to set only `cpuset.cpus` and
|
|
|
|
|
`cpuset.mems` which should be just fine for sandbox purposes. As kernel
|
|
|
|
|
functionality further description can be found in manual page of _cpuset_ or in
|
|
|
|
|
Linux documentation in section `linux/Documentation/cgroups/cpusets.txt`. As
|
|
|
|
|
previously stated this settings can be applied for particular isolate boxes and
|
|
|
|
|
has to be written in isolate configuration. Standard configuration path should
|
|
|
|
|
be `/usr/local/etc/isolate` but it may depend on your installation process.
|
|
|
|
|
Configuration of _cpuset_ in there is really simple and is described in example
|
|
|
|
|
below.
|
|
|
|
|
New feature of the 1.3 version is a possibility of limit Isolate box to one or
|
|
|
|
|
more CPU or memory nodes. This functionality is provided by _cpusets_ kernel
|
|
|
|
|
mechanism and is now integrated into Isolate. It is allowed to set only
|
|
|
|
|
`cpuset.cpus` and `cpuset.mems` which should be just fine for sandbox purposes.
|
|
|
|
|
As a kernel functionality further description can be found in manual page of
|
|
|
|
|
_cpuset_ or in Linux documentation in section
|
|
|
|
|
`linux/Documentation/cgroups/cpusets.txt`. As previously stated this settings
|
|
|
|
|
can be applied for particular Isolate boxes and has to be written in Isolate
|
|
|
|
|
configuration. Standard configuration path should be `/usr/local/etc/isolate`
|
|
|
|
|
but it may depend on your installation process. Configuration of _cpuset_ in
|
|
|
|
|
there is described in example below.
|
|
|
|
|
|
|
|
|
|
```
|
|
|
|
|
box0.cpus = 0 # assign processor with ID 0 to isolate box with ID 0
|
|
|
|
@ -121,12 +124,12 @@ box2.mems = 4-7 # assign range of memory nodes
|
|
|
|
|
box3.cpus = 1,2,3 # assign list of processors to isolate box 3
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
- **cpuset.cpus:** Cpus limitation will restrict sandboxed program only to
|
|
|
|
|
- _cpuset.cpus:_ Cpus limitation will restrict sandboxed program only to
|
|
|
|
|
processor threads set in configuration. On hyperthreaded processors this means
|
|
|
|
|
that all virtual threads are assignable, not only the physical ones. Value can
|
|
|
|
|
be represented by single number, list of numbers separated by commas or range
|
|
|
|
|
with hyphen delimiter.
|
|
|
|
|
- **cpuset.mems:** This value is particularly handy on NUMA systems which has
|
|
|
|
|
- _cpuset.mems:_ This value is particularly handy on NUMA systems which has
|
|
|
|
|
several memory nodes. On standard desktop computers this value should always
|
|
|
|
|
be zero because only one independent memory node is present. As stated in
|
|
|
|
|
`cpus` limitation there can be single value, list of values separated by comma
|
|
|
|
@ -134,10 +137,10 @@ box3.cpus = 1,2,3 # assign list of processors to isolate box 3
|
|
|
|
|
|
|
|
|
|
## Broker
|
|
|
|
|
|
|
|
|
|
### Configuration items
|
|
|
|
|
The default location for broker configuration file is
|
|
|
|
|
`/etc/recodex/broker/config.yml`.
|
|
|
|
|
|
|
|
|
|
Description of configurable items in broker's config. Mandatory items are bold,
|
|
|
|
|
optional italic.
|
|
|
|
|
### Configuration items
|
|
|
|
|
|
|
|
|
|
- _clients_ -- specifies address and port to bind for clients (frontend
|
|
|
|
|
instance)
|
|
|
|
@ -200,9 +203,8 @@ logger:
|
|
|
|
|
|
|
|
|
|
## Monitor
|
|
|
|
|
|
|
|
|
|
Configuration file is located in subdirectory `monitor` of standard ReCodEx
|
|
|
|
|
configuration folder `/etc/recodex/`. It is in YAML format as all of the other
|
|
|
|
|
configurations. Format is very similar to configurations of broker or workers.
|
|
|
|
|
Configuration file is located in directory `/etc/recodex/monitor/` by default.
|
|
|
|
|
It is in YAML format as all of the other configurations.
|
|
|
|
|
|
|
|
|
|
### Configuration items
|
|
|
|
|
|
|
|
|
@ -245,11 +247,17 @@ logger:
|
|
|
|
|
|
|
|
|
|
## Cleaner
|
|
|
|
|
|
|
|
|
|
The default location for cleaner configuration file is
|
|
|
|
|
`/etc/recodex/cleaner/config.yml`.
|
|
|
|
|
|
|
|
|
|
### Configuration items
|
|
|
|
|
|
|
|
|
|
- **cache-dir** -- directory which cleaner manages
|
|
|
|
|
- **file-age** -- file age in seconds which are considered outdated and will be deleted
|
|
|
|
|
- **file-age** -- file age in seconds which is considered as outdated and will
|
|
|
|
|
be marked for deletion
|
|
|
|
|
|
|
|
|
|
### Example configuration
|
|
|
|
|
|
|
|
|
|
```{.yml}
|
|
|
|
|
cache-dir: "/tmp"
|
|
|
|
|
file-age: "3600" # in seconds
|
|
|
|
@ -258,11 +266,11 @@ file-age: "3600" # in seconds
|
|
|
|
|
## REST API
|
|
|
|
|
|
|
|
|
|
The API can be configured in `config.neon` and `config.local.neon` files in
|
|
|
|
|
`app/config` directory. The first file is predefined by authors and should not
|
|
|
|
|
be modified. The second one is not present and could be created by copying
|
|
|
|
|
`config.local.neon.example` template in the config directory. Local
|
|
|
|
|
configuration have higher precedence, so it will override default values from
|
|
|
|
|
`config.neon`.
|
|
|
|
|
`app/config` directory of the API project source tree. The first file is
|
|
|
|
|
predefined by authors and should not be modified. The second one is not present
|
|
|
|
|
and could be created by copying `config.local.neon.example` template in the
|
|
|
|
|
config directory. Local configuration have higher precedence, so it will
|
|
|
|
|
override default values from `config.neon`.
|
|
|
|
|
|
|
|
|
|
### Configurable items
|
|
|
|
|
|
|
|
|
@ -410,6 +418,10 @@ doctrine:
|
|
|
|
|
|
|
|
|
|
## Web application
|
|
|
|
|
|
|
|
|
|
The location for configuration of the web application is in root of the project
|
|
|
|
|
source tree. The name have to be `.env` and can be created by copying template
|
|
|
|
|
`.env-example` file.
|
|
|
|
|
|
|
|
|
|
### Configurable items
|
|
|
|
|
|
|
|
|
|
Description of configurable options. Bold are required values, optional ones are
|
|
|
|
@ -424,7 +436,7 @@ in italics.
|
|
|
|
|
|
|
|
|
|
### Example configuration file
|
|
|
|
|
|
|
|
|
|
```
|
|
|
|
|
```{.yml}
|
|
|
|
|
NODE_ENV=production
|
|
|
|
|
API_BASE=https://recodex.projekty.ms.mff.cuni.cz:4000/v1
|
|
|
|
|
PORT=8080
|
|
|
|
|