LEdoian
/
recodex-wiki
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
You cannot select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
fixes
master
Branches Tags
${ item.name }
Create tag ${ searchTerm }
Create branch ${ searchTerm }
from 'ff08b96403'
${ noResults }
recodex-wiki/System-configuration.md

456 lines
16 KiB
Markdown
Raw Blame History

# System configuration
This section describes configuration of ReCodEx components. Bold items in lists
describing the values are mandatory, italic ones are optional.
## Worker
Worker have a default configuration which is applied to worker itself or is used
in given jobs (implicitly if something is missing, or explicitly with special
variables). This configuration is hardcoded in worker sources and can be
rewritten by explicitly declared configuration file. Format of this
configuration is YAML file with similar structure as job configuration. The
default location is `/etc/recodex/worker/config-<N>.yml` where `N` is identifier
of the particular worker instance.
### Configuration items
- **worker-id** -- unique identification of worker at one server. This id is
used by _isolate_ sandbox on linux systems, so make sure to meet requirements
of the isolate (default is number from 1 to 999).
- _worker-description_ -- human readable description of this worker
- **broker-uri** -- URI of the broker (hostname, IP address, including port,
...)
- _broker-ping-interval_ -- time interval how often to send ping messages to
broker. Used units are milliseconds.
- _max-broker-liveness_ -- specifies how many pings in a row can broker miss
without making the worker dead.
- _headers_ -- map of headers specifies worker's capabilities
- _env_ -- list of environmental variables which are sent to broker in init
command
- _threads_ -- information about available threads for this worker
- **hwgroup** -- hardware group of this worker. Hardware group must specify
worker hardware and software capabilities and it is main item for broker
routing decisions.
- _working-directory_ -- where will be stored all needed files. Can be the same
for multiple workers on one server.
- **file-managers** -- addresses and credentials to all file managers used (eq.
all different frontends using this worker)
- **hostname** -- URI of file manager
- _username_ -- username for http authentication (if needed)
- _password_ -- password for http authentication (if needed)
- _file-cache_ -- configuration of caching feature
- _cache-dir_ -- path to caching directory. Can be the same for multiple
workers.
- _logger_ -- settings of logging capabilities
- _file_ -- path to the logging file with name without suffix.
`/var/log/recodex/worker` item will produce `worker.log`, `worker.1.log`,
...
- _level_ -- level of logging, one of `off`, `emerg`, `alert`, `critical`,
`err`, `warn`, `notice`, `info` and `debug`
- _max-size_ -- maximal size of log file before rotating in bytes
- _rotations_ -- number of rotation kept
- _limits_ -- default sandbox limits for this worker. All items are described in
assignments section in job configuration description. If some limits are not
set in job configuration, defaults from worker config will be used. In such
case the worker's defaults will be set as the maximum for the job. Also,
limits in job configuration cannot exceed limits from worker.
- **max-output-length** -- used for `tasks.{task}.sandbox.output` option, defined
in bytes, applied to both stdout and stderr and is not divided, both will get
this value
- **cleanup-submission** -- if set to true, then files produced during evaluation
of submission will be deleted at the end, extra caution is advised because this
setup can cause extensive disk usage
### Example config file
```{.yml}
worker-id: 1
broker-uri: tcp://localhost:9657
broker-ping-interval: 10 # milliseconds
max-broker-liveness: 10
headers:
env:
- c
- cpp
threads: 2
hwgroup: "group1"
working-directory: /tmp/recodex
file-managers:
- hostname: "http://localhost:9999" # port is optional
username: "" # can be ignored in specific modules
password: "" # can be ignored in specific modules
file-cache: # only in case that there is cache module
cache-dir: "/tmp/recodex/cache"
logger:
file: "/var/log/recodex/worker" # w/o suffix - actual names will
# be worker.log, worker.1.log,...
level: "debug" # level of logging
max-size: 1048576 # 1 MB; max size of file before log rotation
rotations: 3 # number of rotations kept
limits:
time: 5 # in secs
wall-time: 6 # seconds
extra-time: 2 # seconds
stack-size: 0 # normal in KB, but 0 means no special limit
memory: 50000 # in KB
parallel: 1
disk-size: 50
disk-files: 5
environ-variable:
ISOLATE_BOX: "/box"
ISOLATE_TMP: "/tmp"
bound-directories:
- src: /tmp/recodex/eval_5
dst: /evaluate
mode: RW,NOEXEC
```
### Isolate sandbox
New feature of the 1.3 version is a possibility of limit Isolate box to one or
more CPU or memory nodes. This functionality is provided by _cpusets_ kernel
mechanism and is now integrated into Isolate. It is allowed to set only
`cpuset.cpus` and `cpuset.mems` which should be just fine for sandbox purposes.
As a kernel functionality further description can be found in manual page of
_cpuset_ or in Linux documentation in section
`linux/Documentation/cgroups/cpusets.txt`. As previously stated this settings
can be applied for particular Isolate boxes and has to be written in Isolate
configuration. Standard configuration path should be `/usr/local/etc/isolate`
but it may depend on your installation process. Configuration of _cpuset_ in
there is described in example below.
```
box0.cpus = 0 # assign processor with ID 0 to isolate box with ID 0
box0.mems = 0 # assign memory node with ID 0
# if not set, linux by itself will decide where should
# the sandboxed programs run at
box2.cpus = 1-3 # assign range of processors to isolate box 2
box2.mems = 4-7 # assign range of memory nodes
box3.cpus = 1,2,3 # assign list of processors to isolate box 3
```
- _cpuset.cpus:_ Cpus limitation will restrict sandboxed program only to
processor threads set in configuration. On hyperthreaded processors this means
that all virtual threads are assignable, not only the physical ones. Value can
be represented by single number, list of numbers separated by commas or range
with hyphen delimiter.
- _cpuset.mems:_ This value is particularly handy on NUMA systems which has
several memory nodes. On standard desktop computers this value should always
be zero because only one independent memory node is present. As stated in
`cpus` limitation there can be single value, list of values separated by comma
or range stated with hyphen.
## Broker
The default location for broker configuration file is
`/etc/recodex/broker/config.yml`.
### Configuration items
- _clients_ -- specifies address and port to bind for clients (frontend
instance)
- _address_ -- hostname or IP address as string (`*` for any)
- _port_ -- desired port
- _workers_ -- specifies address and port to bind for workers
- _address_ -- hostname or IP address as string (`*` for any)
- _port_ -- desired port
- _max_liveness_ -- maximum amount of pings the worker can fail to send
before it is considered disconnected
- _max_request_failures_ -- maximum number of times a job can fail (due to
e.g. worker disconnect or a network error when downloading something from
the fileserver) and be assigned again
- _monitor_ -- settings of monitor service connection
- _address_ -- IP address of running monitor service
- _port_ -- desired port
- _notifier_ -- details of connection which is used in case of errors and good
to know states
- _address_ -- address where frontend API runs
- _port_ -- desired port
- _username_ -- username which can be used for HTTP authentication
- _password_ -- password which can be used for HTTP authentication
- _logger_ -- settings of logging capabilities
- _file_ -- path to the logging file with name without suffix.
`/var/log/recodex/broker` item will produce `broker.log`, `broker.1.log`,
...
- _level_ -- level of logging, one of `off`, `emerg`, `alert`, `critical`,
`err`, `warn`, `notice`, `info` and `debug`
- _max-size_ -- maximal size of log file before rotating
- _rotations_ -- number of rotation kept
### Example config file
```{.yml}
# Address and port for clients (frontend)
clients:
address: "*"
port: 9658
# Address and port for workers
workers:
address: "*"
port: 9657
max_liveness: 10
max_request_failures: 3
monitor:
address: "127.0.0.1"
port: 7894
notifier:
address: "127.0.0.1"
port: 8080
username: ""
password: ""
logger:
file: "/var/log/recodex/broker" # w/o suffix - actual names will be
# broker.log, broker.1.log, ...
level: "debug" # level of logging
max-size: 1048576 # 1 MB; max size of file before log rotation
rotations: 3 # number of rotations kept
```
## Monitor
Configuration file is located in directory `/etc/recodex/monitor/` by default.
It is in YAML format as all of the other configurations.
### Configuration items
Description of configurable items, bold ones are required, italics ones are
optional.
- _websocket_uri_ -- URI where is the endpoint of WebSocket connection. Must be
visible to the clients (directly or through public proxy)
- string representation of IP address or a hostname
- port number
- _zeromq_uri_ -- URI where is the endpoint of ZeroMQ connection from broker.
Could be hidden from public internet.
- string representation of IP address or a hostname
- port number
- _logger_ -- settings of logging
- _file_ -- path with name of log file. Defaults to
`/var/log/recodex/monitor.log`
- _level_ -- logging level, one of "debug", "info", "warning", "error" and
"critical"
- _max-size_ -- maximum size of log file before rotation in bytes
- _rotations_ -- number of rotations kept
### Example configuration file
```{.yml}
---
websocket_uri:
- "127.0.0.1"
- 4567
zeromq_uri:
- "127.0.0.1"
- 7894
logger:
file: "/var/log/recodex/monitor.log"
level: "debug"
max-size: 1048576 # 1 MB
rotations: 3
...
```
## Cleaner
The default location for cleaner configuration file is
`/etc/recodex/cleaner/config.yml`.
### Configuration items
- **cache-dir** -- directory which cleaner manages
- **file-age** -- file age in seconds which is considered as outdated and will
be marked for deletion
### Example configuration
```{.yml}
cache-dir: "/tmp"
file-age: "3600" # in seconds
```
## REST API
The API can be configured in `config.neon` and `config.local.neon` files in
`app/config` directory of the API project source tree. The first file is
predefined by authors and should not be modified. The second one is not present
and could be created by copying `config.local.neon.example` template in the
config directory. Local configuration have higher precedence, so it will
override default values from `config.neon`.
### Configurable items
Description of configurable items. All timeouts are in milliseconds if not
stated otherwise.
- accessManager -- configuration of access token in [JWT
standard](https://www.rfc-editor.org/rfc/rfc7519.txt). Do **not** modify
unless you really know what are you doing.
- fileServer -- connection to fileserver
- address -- URI of fileserver
- auth -- _username_ and _password_ for HTTP basic authentication
- timeouts -- _connection_ timeout for establishing new connection and
_request_ timeout for completing one request
- broker -- connection to broker
- address -- URI of broker
- auth -- _username_ and _password_ for broker callback authentication back
to API
- timeouts -- _ack_ timeout for first response that broker receives the
message, _send_ timeout how long try to send new job to the broker and
_result_ timeout how long to wait for confirmation if job can be processed
or not
- monitor -- connection to monitor
- address -- URI of monitor
- CAS -- CAS external authentication
- serviceId -- visible identifier of this service
- ldapConnection -- parameters for connecting to LDAP, _hostname_,
_base_dn_, _port_, _security_ and _bindName_
- fields -- names of LDAP keys for informations as _email_, _firstName_ and
_lastName_
- emails -- common configuration for sending email (addresses and template
variables)
- apiUrl -- base URL of API server including port (for referencing pictures
in messages)
- footerUrl -- link in the message footer
- siteName -- name of frontend (ReCodEx, or KSP for unique instance for KSP
course)
- githubUrl -- URL to GitHub repository of this project
- from -- sending email address
- failures -- admin messages on errors
- emails -- additional info for sending mails, _to_ is admin mail address,
_from_ is source address, _subjectPrefix_ is prefix of mail subject
- forgottenPassword -- user messages for changing passwords
- redirectUrl -- URL of web application where the password can be changed
- tokenExpiration -- expiration timeout of temporary token (in seconds)
- emails -- additional info for sending mails, _from_ is source address and
_subjectPrefix_ is prefix of mail subject
- mail -- configuration of sending mails
- smtp -- using SMTP server, have to be "true"
- host -- address of the server
- port -- sending port (common values are 25, 465, 587)
- username -- login to the server
- password -- password to the server
- secure -- security, values are empty for no security, "ssl" or "tls"
- context -- additional parameters, depending on used mail engine. For
examle self-signed certificates can be allowed as _verify_peer_ and
_verify_peer_name_ to false and _allow_self_signed_ to true under _ssl_
key (see example).
Outside the parameters section of configuration is configuration for Doctrine.
It is ORM framework which maps PHP objects (entities) into database tables and
rows. The configuration is simple, required items are only _user_, _password_
and _host_ with _dbname_, i.e. address of database computer (mostly localhost)
with name of ReCodEx database.
### Example local configuration file
```{.yml}
parameters:
accessManager:
leeway: 60
issuer: https://recodex.projekty.ms.mff.cuni.cz
audience: https://recodex.projekty.ms.mff.cuni.cz
expiration: 86400 # 24 hours in seconds
usedAlgorithm: HS256
allowedAlgorithms:
- HS256
verificationKey: "recodex-123"
fileServer:
address: http://127.0.0.1:9999
auth:
username: "user"
password: "pass"
timeouts:
connection: 500
broker:
address: tcp://127.0.0.1:9658
auth:
username: "user"
password: "pass"
timeouts:
ack: 100
send: 5000
result: 1000
monitor:
address: wss://recodex.projekty.ms.mff.cuni.cz:4443/ws
CAS:
serviceId: "cas-uk"
ldapConnection:
hostname: "ldap.cuni.cz"
base_dn: "ou=people,dc=cuni,dc=cz"
port: 389
security: SSL
bindName: "cunipersonalid"
fields:
email: "mail"
firstName: "givenName"
lastName: "sn"
emails:
apiUrl: https://recodex.projekty.ms.mff.cuni.cz:4000
footerUrl: https://recodex.projekty.ms.mff.cuni.cz
siteName: "ReCodEx"
githubUrl: https://github.com/ReCodEx
from: "ReCodEx <noreply@example.com>"
failures:
emails:
to: "Admin Name <admin@example.com>"
from: %emails.from%
subjectPrefix: "ReCodEx Failure Report - "
forgottenPassword:
redirectUrl: "https://recodex.projekty.ms.mff.cuni.cz/
forgotten-password/change"
tokenExpiration: 600 # 10 minues
emails:
from: %emails.from%
subjectPrefix: "ReCodEx Forgotten Password Request - "
mail:
smtp: true
host: "smtp.ps.stdin.cz"
port: 587
username: "user"
password: "pass"
secure: "tls"
context:
ssl:
verify_peer: false
verify_peer_name: false
allow_self_signed: true
doctrine:
user: "user"
password: "pass"
host: localhost
dbname: "recodex-api"
```
## Web application
The location for configuration of the web application is in root of the project
source tree. The name have to be `.env` and can be created by copying template
`.env-example` file.
### Configurable items
Description of configurable options. Bold are required values, optional ones are
in italics.
- **NODE_ENV** -- mode of the server
- **API_BASE** -- base address of API server, including port and API version
- **PORT** -- port where the app is listening
- _WEBPACK_DEV_SERVER_PORT_ -- port for webpack dev server when running in
development mode. Default one is 8081, this option might be useful when this
port is necessary for some other service.
### Example configuration file
```{.ini}
NODE_ENV=production
API_BASE=https://recodex.projekty.ms.mff.cuni.cz:4000/v1
PORT=8080
```
<!---
// vim: set formatoptions=tqn flp+=\\\|^\\*\\s* textwidth=80 colorcolumn=+1:
-->
Reference in New Issue View Git Blame Copy Permalink