master
Martin Polanka 8 years ago
parent 423924b3d8
commit 0ca188ea50

@ -16,7 +16,7 @@ divided into two parts:
These parts run in separate threads of the same process and communicate through a ZeroMQ in-process
socket. This design allows the worker to keep sending `ping` messages even when
it's processing a job.
it is processing a job.
After receiving an evaluation request, worker has to:
@ -36,7 +36,7 @@ These properties help the broker decide which worker is suitable for processing
a request.
The hardware group is a string identifier used to group worker machines with
similar hardware configuration, for example "i7-4560-quad-ssd". It's
similar hardware configuration, for example "i7-4560-quad-ssd". It is
important for assignments where running times are compared to those of reference
solutions (we have to make sure that both programs run on simmilar hardware).
@ -89,7 +89,7 @@ $ git submodule update --init
```
### Install worker on Linux
It's supposed that your current working directory is that one with clonned worker source codes.
It is supposed that your current working directory is that one with clonned worker source codes.
- Prepare environment running `mkdir build && cd build`
- Build sources by `cmake ..` following by `make -j#` where '#' symbol refers to number of your CPU threads.
@ -107,7 +107,7 @@ The worker installation process is composed of following steps:
- create log directory `/var/log/recodex`
- set ownership of config (`/etc/recodex`) and log (`/var/log/recodex`) directories to `recodex` user and group
_Note:_ If you don't want to generate binary packages, you can just install the project with `make install` (as root). But installation through your distribution's package manager is preferred way to keep your system clean and manageable in long term horizon.
_Note:_ If you do not want to generate binary packages, you can just install the project with `make install` (as root). But installation through your distribution's package manager is preferred way to keep your system clean and manageable in long term horizon.
### Install worker on Windows
From beginning we are determined to support Windows operating system on which some of the workers may run (especially for projects in C# programming language). Support for Windows is quite hard and time consuming and there were several problems during the development. To ensure capability of compilation on Windows we set up CI for Windows named [Appveyor](http://www.appveyor.com/). However installation should be easy due to provided installation script.
@ -118,7 +118,7 @@ Mentioned script is placed in *install* directory alongside supportive scripts f
Building and installing of worker is then quite simple, script has command line parameters which can be used to specify what will be done:
- *-build* -- It's the default options if none specified. Builds worker and its tests, all is saved in *build* folder and subfolders.
- *-build* -- It is the default options if none specified. Builds worker and its tests, all is saved in *build* folder and subfolders.
- *-clean* -- Cleanup of downloaded NuGet packages and built application/libraries.
- *-test* -- Build worker and run tests on compiled test cases.
- *-package* -- Generation of clickable installation using cpack and [NSIS](http://nsis.sourceforge.net/) (has to be installed on machine to get this to work).
@ -141,7 +141,7 @@ created, it can be found in *build* folder under name
## Configuration and usage
Following text describes how to set up and run **worker** program. It's supposed to have required binaries installed. Also, using systemd is recommended for best user experience, but it's not required. Almost all modern Linux distributions are using systemd nowadays.
Following text describes how to set up and run **worker** program. It is supposed to have required binaries installed. Also, using systemd is recommended for best user experience, but it is not required. Almost all modern Linux distributions are using systemd nowadays.
### Default worker configuration
@ -158,7 +158,7 @@ Mandatory items are bold, optional italic.
- _headers_ - headers specifies worker's capabilities
- _env_ - map of enviromental variables
- _threads_ - information about available threads for this worker
- **hwgroup** - hardware group of this worker. Hardware group must specify worker hardware and software capabilities and it's main item for broker routing decisions.
- **hwgroup** - hardware group of this worker. Hardware group must specify worker hardware and software capabilities and it is main item for broker routing decisions.
- _working-directory_ - where will be stored all needed files. Can be the same for multiple workers on one server.
- **file-managers** - addresses and credentials to all file managers used (eq. all different frontends using this worker)
- **hostname** - URI of file manager
@ -221,7 +221,7 @@ limits:
A systemd unit file is distributed with the worker to simplify its launch. It
integrates worker nicely into your Linux system and allows you to run it
automatically on system startup. It's possible to have more than one worker on
automatically on system startup. It is possible to have more than one worker on
every server, so the provided unit file is templated. Each instance of the
worker unit has a unique string identifier, which is used for managing that
instance through systemd. By default, only one worker instance is ready to use
@ -271,7 +271,7 @@ Chapter by itself is filesystem handling. Isolate uses mount kernel namespace to
#### Limit isolate boxes to particular cpu or memory node
New feature in version 1.3 is possibility of limit Isolate box to one or more cpu or memory node. This functionality is provided by _cpusets_ kernel mechanism and is now integrated in isolate. It's allowed to set only `cpuset.cpus` and `cpuset.mems` which should be just fine for sandbox purposes. As kernel functionality further description can be found in manual page of _cpuset_ or in Linux documentation in section `linux/Documentation/cgroups/cpusets.txt`. As previously stated this settings can be applied for particular isolate boxes and has to be written in isolate configuration. Standard configuration path should be `/usr/local/etc/isolate` but it may depend on your installation process. Configuration of _cpuset_ in there is really simple and is described in example below.
New feature in version 1.3 is possibility of limit Isolate box to one or more cpu or memory node. This functionality is provided by _cpusets_ kernel mechanism and is now integrated in isolate. It is allowed to set only `cpuset.cpus` and `cpuset.mems` which should be just fine for sandbox purposes. As kernel functionality further description can be found in manual page of _cpuset_ or in Linux documentation in section `linux/Documentation/cgroups/cpusets.txt`. As previously stated this settings can be applied for particular isolate boxes and has to be written in isolate configuration. Standard configuration path should be `/usr/local/etc/isolate` but it may depend on your installation process. Configuration of _cpuset_ in there is really simple and is described in example below.
```
box0.cpus = 0 # assign processor with ID 0 to isolate box with ID 0
@ -288,7 +288,7 @@ box3.cpus = 1,2,3 # assign list of processors to isolate box 3
### WrapSharp
WrapSharp is sandbox for programs in C# written also in C#. We have written it as a proof of concept sandbox for using in Windows environment. However, it's not properly tested and integrated to the worker yet. Security audit should be done before using in production. After that, with just a little bit of effort integrating into worker there can be a running sandbox for C# programs on Windows system.
WrapSharp is sandbox for programs in C# written also in C#. We have written it as a proof of concept sandbox for using in Windows environment. However, it is not properly tested and integrated to the worker yet. Security audit should be done before using in production. After that, with just a little bit of effort integrating into worker there can be a running sandbox for C# programs on Windows system.
## Cleaner
@ -297,15 +297,15 @@ WrapSharp is sandbox for programs in C# written also in C#. We have written it a
Cleaner is integral part of worker which manages its cache folder, mainly deletes outdated files. Every cleaner instance maintains one cache folder, which can be used by multiple workers. This means on one server there can be numerous instances of workers with the same cache folder, but there should be only one cleaner.
Cleaner is written in Python programming language and is used as simple script which just does its job and ends, so has to be cronned. For proper function of cleaner some suitable cronning interval has to be used. It's recommended to use 24 hour interval which should be sufficient enough.
Cleaner is written in Python programming language and is used as simple script which just does its job and ends, so has to be cronned. For proper function of cleaner some suitable cronning interval has to be used. It is recommended to use 24 hour interval which should be sufficient enough.
#### Last access timestamp
There is a bit of catch with cleaner service, to work properly, server filesystem has to have enabled last access timestamp. Cleaner checks these stamps and based on them it decides if file will be deleted or not, simple write timestamp or created at timestamp are not enough to reflect real usage and need of particular file. Last access timestamp feature is a bit controversial (more on this subject can be found [here](https://en.wikipedia.org/wiki/Stat_%28system_call%29#Criticism_of_atime)) and it's not by default enabled on conventional filesystems. In linux this can be solved by adding `strictatime` option to `fstab` file. On Windows following command has to be executed (as administrator) `fsutil behavior set disablelastaccess 0`.
There is a bit of catch with cleaner service, to work properly, server filesystem has to have enabled last access timestamp. Cleaner checks these stamps and based on them it decides if file will be deleted or not, simple write timestamp or created at timestamp are not enough to reflect real usage and need of particular file. Last access timestamp feature is a bit controversial (more on this subject can be found [here](https://en.wikipedia.org/wiki/Stat_%28system_call%29#Criticism_of_atime)) and it is not by default enabled on conventional filesystems. In linux this can be solved by adding `strictatime` option to `fstab` file. On Windows following command has to be executed (as administrator) `fsutil behavior set disablelastaccess 0`.
### Installation
To install and use the cleaner, it's necessary to have Python3 with package manager `pip` installed.
To install and use the cleaner, it is necessary to have Python3 with package manager `pip` installed.
- Dependencies of cleaner has to be installed:
```

Loading…
Cancel
Save