recodex-wiki/Configuration-and-usage.md

Configuration and usage

This page describes how to set up and run broker and worker programs. It's supposed to have required binaries installed. For instructions see system configuration page. Also, using systemd is recommended for best user experience, but it's not required. Almost all modern Linux distribuions are using systemd now.

Worker

Installation of worker program does following step to your computer:

• create config file /etc/recodex/worker/config-1.yml
• create systemd unit file /etc/systemd/system/recodex-worker@.service
• put main binary to /usr/bin/recodex-worker
• put judges binaries to /usr/bin/recodex-judge-normal, /usr/bin/recodex-judge-shuffle and /usr/bin/recodex-judge-filter
• create system user and group recodex with /sbin/nologin shell (if not existing)
• create log directory /var/log/recodex
• set ownership of config (/etc/recodex) and log (/var/log/recodex) directories to recodex user and group

Default worker configuration

Worker should have some default configuration which is applied to worker itself or may be used in given jobs (implicitly if something is missing, or explicitly with special variables). This configuration should be hardcoded and can be rewritten by explicitly declared configuration file. Format of this configuration is yaml like in the job config.

worker-id: 1
broker-uri: tcp://localhost:9657
headers:
    env:
        - c
        - python
    threads: 2
hwgroup: "group1"
working-directory: /tmp/isoeval
file-managers:
    - hostname: "http://localhost:9999"  # port is optional
      username: ""  # can be ignored in specific modules
      password: ""  # can be ignored in specific modules
file-cache:  # only in case that there is cache module
    cache-dir: "/tmp/isoeval/cache"
logger:
    file: "/var/log/recodex/worker"  # w/o suffix - actual names will be worker.log, worker.1.log, ...
    level: "debug"  # level of logging
    max-size: 1048576  # 1 MB; max size of file before log rotation
    rotations: 3  # number of rotations kept
limits:
    time: 5  # in secs
    wall-time: 6  # seconds
    extra-time: 2  # seconds
    stack-size: 0  # normal in KB, but 0 means no special limit
    memory: 50000  # in KB
    parallel: 1  # time and memory limits are merged
    disk-size: 50
    disk-files: 5
    environ-variable:
        ISOLATE_BOX: "/box"
        ISOLATE_TMP: "/tmp"
    bound-directories:
        - src: /tmp/isoeval/eval_5
          dst: /evaluate
          mode: RW,NOEXEC

Running worker

For easy and comfortable managing worker is included systemd unit file. It integrates worker nicely into your Linux system and allow you to run worker automaticaly after system startup for example. It's supposed to have more than one worker on every server, so provided unit file is templated. Each instance of worker have unique string identifier, which is used for managing that instance through systemd. By default, only one worker instance is ready to use after installation. It's ID is "1".

• Starting worker with id "1" can be done this way:

# systemctl start recodex-worker@1.service

Check with

# systemctl status recodex-worker@1.service

if the worker is running. You should see "active (running)" message.

• Worker can be stopped or restarted accordigly using systemctl stop and systemctl restart commands.
• If you want to run worker after system startup, run:

# systemctl enable recodex-worker@1.service

For further information about using systemd please refer to systemd documentation.

Adding new worker

To add a new worker you need to do a few steps:

• Get unique string ID. Think of one.
• Copy default configuration file /etc/recodex/worker/config-1.yml to the same directory and name it config-<your_unique_ID>.yml
• Edit that config file to fit your needs. Note that you must at least change worker-id and logger file values to be unique.
• Run new instance using

# systemctl start recodex-worker@<your_unique_ID>.service

Broker

Installation of broker program does following step to your computer:

• create config file /etc/recodex/brokerr/config.yml
• create systemd unit file /etc/systemd/system/recodex-broker.service
• put main binary to /usr/bin/recodex-broker
• create system user and group recodex with nologin shell (if not existing)
• create log directory /var/log/recodex
• set ownership of config (/etc/recodex) and log (/var/log/recodex) directories to recodex user and group

Default broker configuration

# Address and port for clients (frontend)
clients:
    address: "*"
    port: 9658
# Address and port for workers
workers:
    address: "*"
    port: 9657
logger:
    file: "/var/log/recodex/broker"  # w/o suffix - actual names will be broker.log, broker.1.log, ...
    level: "debug"  # level of logging
    max-size: 1048576  # 1 MB; max size of file before log rotation
    rotations: 3  # number of rotations kept

Running broker

Running broker is very similar to the worker setup. There is only one broker per whole ReCodEx solution, so there is no need for systemd templates. So running broker is just:

# systemctl start recodex-broker.service

For more info please refer to worker part of this page or systemd documentation.