@ -285,49 +285,61 @@ box3.cpus = 1,2,3 # assign list of processors to isolate box 3
### WrapSharp
### WrapSharp
WrapSharp is sandbox for programs in C# written also in C#. We have written it as a proof of concept sandbox for using in Windows environment. However, it's not properly tested and integrated to the worker yet. With just a little bit of effort there can be a running sandbox for C# programs on Windows system.
WrapSharp is sandbox for programs in C# written also in C#. We have written it as a proof of concept sandbox for using in Windows environment. However, it's not properly tested and integrated to the worker yet. Security audit should be done before using in production. After that, with just a little bit of effort integrating into worker there can be a running sandbox for C# programs on Windows system.
## Cleaner
## Cleaner
### Description
### Description
Cleaner is integral part of **worker** which manages its cache folder, mainly deletes outdated files. Every cleaner maintains its one and only cache folder, which can be used by multiple workers. This means on one server there can be numerous instances of workers with the same cache folder, but there can be (and should be) only one cleaner.
Cleaner is written in **Python** and is used as simple script which just does its job and ends and therefore has to be cronned. For proper function of cleaner some suitable cronning interval has to be used. Its recommended to use 24 hour interval which should be sufficient enough.
Cleaner is integral part of worker which manages its cache folder, mainly deletes outdated files. Every cleaner instance maintains one cache folder, which can be used by multiple workers. This means on one server there can be numerous instances of workers with the same cache folder, but there should be only one cleaner.
Cleaner is written in Python programming language and is used as simple script which just does its job and ends, so has to be cronned. For proper function of cleaner some suitable cronning interval has to be used. Its recommended to use 24 hour interval which should be sufficient enough.
#### Last access timestamp
#### Last access timestamp
There is a bit of catch with cleaner service, to work properly, server filesystem has to have enabled last access timestamp. Cleaner checks these stamps and based on them it decides if file will be deleted or not, simple write timestamp or created at timestamp are not enough to reflect real usage and need of particular file. Last access timestamp feature is a bit controversial (more on this subject can be found [here](https://en.wikipedia.org/wiki/Stat_%28system_call%29#Criticism_of_atime)) and its not by default enabled on conventional filesystems. In linux this can be solved by adding `strictatime` option to `fstab` file. On Windows following command has to be executed (as administrator) `fsutil behavior set disablelastaccess 0`.
There is a bit of catch with cleaner service, to work properly, server filesystem has to have enabled last access timestamp. Cleaner checks these stamps and based on them it decides if file will be deleted or not, simple write timestamp or created at timestamp are not enough to reflect real usage and need of particular file. Last access timestamp feature is a bit controversial (more on this subject can be found [here](https://en.wikipedia.org/wiki/Stat_%28system_call%29#Criticism_of_atime)) and its not by default enabled on conventional filesystems. In linux this can be solved by adding `strictatime` option to `fstab` file. On Windows following command has to be executed (as administrator) `fsutil behavior set disablelastaccess 0`.
### Installation
### Installation
To install and use the cleaner, it's necessary to have **Python** and **Pip** in version 3 installed.
* Firstly dependencies of cleaner has to be installed:
To install and use the cleaner, it's necessary to have Python3 with package manager `pip` installed.
- Dependencies of cleaner has to be installed:
```
```
pip install -r requirements.txt
$ pip install -r requirements.txt
```
```
- RPM distributions can make and install binary package. This can be done like this:
#### Fedora (and other RPM distributions)
```
- run `python setup.py bdist_rpm --post-install ./cleaner/install/postinst` to generate binary `.rpm` package
- decide in which folder cleaner should be installed, `C:\Program Files\ReCodEx\cleaner` is assumed
- start `cmd` with administrator permissions
- run installation with `python setup.py install --install-scripts "C:\Program Files\ReCodEx\cleaner"` where path specified with `--install-scripts` can be changed
- run installation with
- copy configuration file alongside with installed executable using `copy install\config.yml "C:\Program Files\ReCodEx\cleaner\config.yml"`
```
> python setup.py install --install-scripts \
"C:\Program Files\ReCodEx\cleaner"
```
where path specified with `--install-scripts` can be changed
- copy configuration file alongside with installed executable using
```
> copy install\config.yml \
"C:\Program Files\ReCodEx\cleaner\config.yml"
```
### Configuration and usage
### Configuration and usage
#### Configuration items
#### Configuration items
- **cache-dir** - directory which cleaner manages
- **cache-dir** -- directory which cleaner manages
- **file-age** - file age in seconds which are considered outdated and will be deleted
- **file-age** -- file age in seconds which are considered outdated and will be deleted
#### Example configuration
#### Example configuration
```
```{.yml}
cache-dir: "/tmp"
cache-dir: "/tmp"
file-age: "3600" # in seconds
file-age: "3600" # in seconds
```
```
@ -335,7 +347,22 @@ file-age: "3600" # in seconds
#### Usage
#### Usage
As stated before cleaner should be cronned, on linux systems this can be done by built in `cron` service or if there is `systemd` present cleaner itself provides `*.timer` file which can be used for cronning from `systemd`. On Windows systems internal scheduler should be used.
As stated before cleaner should be cronned, on linux systems this can be done by built in `cron` service or if there is `systemd` present cleaner itself provides `*.timer` file which can be used for cronning from `systemd`. On Windows systems internal scheduler should be used.
- Running cleaner from command line is fairly simple: `recodex-cleaner -c /etc/recodex/cleaner`
- Running cleaner from command line is fairly simple:
- Enable cleaner service using systemd: `systemctl start recodex-cleaner.timer`
```
- Add cleaner to linux cron service using following configuration line: `0 0 * * * /usr/bin/recodex-cleaner -c /etc/recodex/cleaner/config.yml`
$ recodex-cleaner -c /etc/recodex/cleaner
- Add cleaner to Windows cheduler service with following command: `schtasks /create /sc daily /tn "ReCodEx Cleaner" /tr "\"C:\Program Files\ReCodEx\cleaner\recodex-cleaner.exe\" -c \"C:\Program Files\ReCodEx\cleaner\config.yml\""`
```
- Enable cleaner service using systemd:
```
$ systemctl start recodex-cleaner.timer
```
- Add cleaner to linux cron service using following configuration line:
Petr Stefan
8 years ago