LEdoian
/
recodex-wiki
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Updated Installation (markdown)

Browse Source
master
Martin Kruliš 4 years ago
parent e3c13ad35e
commit 46afcd3466
1 changed files with 175 additions and 22 deletions
Show Stats Download Patch File Download Diff File

197
Installation.md
Unescape Escape View File

@ -377,6 +377,10 @@ path (`dst`) needs to be added to `environ-variable` list. Be careful when
editing `PATH` or `LD_LIBRARY_PATH` environmental variables as they may apply
to multiple runtimes.
After you set all runtime environments successfully (will be explained below),
it is recommended to set `cleanup-submission` to `true`, so that worker removes
old data from its working directory when each evaluation concludes.
### Isolate configuration
@ -406,21 +410,35 @@ planning the purchase of your hardware).
### Cleaner configuration
TODO
The cleaner module removes old records from worker cache. It is a separate module
since there may be multiple workers using the same cache.
After installing, edit the `/etc/recodex/cleaner/config.yml`. The most important
parameter is `cache-dir` which holds the path to worker cache directory
(i.e., `/var/recodex-worker-cache` if you followed the previous instructions).
Additionally, you may set the expiration time and logging properties.
Although `recodex-cleaner` is also registered as systemd service, you need to
enable and start the `recodex-cleaner.timer` service, which runs the cleaner
periodically (once a day). An alternative is to add cleaner to cron, but the
systemd timer is the recommended way.
### Runtime environments
Some runtimes require access to `/etc` directory since their compilers or
interprets have their configuration there (e.g., freepascal or PHP). It is not
a good idea to map entire `/etc` in sandbox. One possibility is to prepare
separated config directory (e.g., `/usr/etc`) and place configs visible to sandbox
there (and add it to `bound-directories` list).
interprets have their configuration there (e.g., freepascal or PHP). It could be
a security risk to map entire `/etc` into sandbox (although, some environments
are having `/etc` mapped anyways, but only for the compilation step). Furthermore,
it might be helpful to have separate configuration files for the system and for
the sandbox. One possible solution is to prepare separated config directory
(e.g., `/usr/etc`) and place configs for the sandbox there (and add it to
`bound-directories` list).
Runtimes `bash` and `data-linux` should work out of the box. Other runtimes will
require additional installations and configurations:
#### C and C++ (`c-gcc-linux` and `cxx-gcc-linux`)
#### C and C++
Simply install the GCC compiler.
```
@ -429,14 +447,29 @@ dnf -y install gcc gcc-c++
#### C# and .NET Core
TODO! Currently migrating from Mono to .NET Core.
**We are currently migrating configuration from Mono to .NET Core. This
configuration applies to `cs-dotnet-core` environment, `mono` is considered
deprecated.**
Install .NET core SDK:
```
# dnf -y install dotnet-sdk-3.1
```
Download the following files:
```
https://raw.githubusercontent.com/ReCodEx/utils/master/runners/cs/Reader.cs
https://raw.githubusercontent.com/ReCodEx/utils/master/runners/cs/Wrapper.cs
https://raw.githubusercontent.com/ReCodEx/utils/master/runners/cs/recodex.csproj
```
#### Free Pascal (`freepascal-linux`)
Open your running ReCodEx instance in web browser and log in as administrator.
Open `Pipeline` page, find the *C# .NET Core Compilation* pipeline, and click on
the `Edit` button on the right. Scroll down to *Supplementary files* box
and upload all three downloaded files here.
#### Free Pascal
First you need to install Free Pascal compiler:
```
@ -455,13 +488,36 @@ Simply install the Go language package.
# dnf -y install golang
```
#### Groovy
TODO
This environment requires Java runtime to be installed as well (please do before
venturing forth).
Download latest stable binary of Apache Groovy from `https://groovy.apache.org/download.html`
(e.g., `https://dl.bintray.com/groovy/maven/apache-groovy-binary-3.0.5.zip`)
And unzip the content into `/opt/groovy`.
Add environmental variable in worker configuration (`environ-variable` list)
`GROOVY_HOME` with value `/opt/groovy` and enable access to this directory by
adding it into `bound-directories`.
Create a symlink `/opt/groovy/groovy.jar` referring to
`/opt/groovy/libs/groovy-3.0.5.jar` (where `3.0.5` is the actual version of
downloaded Groovy).
Create the following symlinks in `/usr/bin`, each referring to the binary of the
same name in `/opt/groovy/bin`: `groovy`, `groovyc`, `groovyConsole`,
`groovydoc`, `groovysh`.
#### Haskell
TODO
Simply install the Haskell compiler.
```
# dnf -y install ghc
```
#### Java
@ -485,7 +541,18 @@ and upload the compiled `javarun.class`.
#### Kotlin
TODO
This environment requires Java runtime to be installed as well (please do before
venturing forth).
Download latest Kotlin compiler release from [GitHub](https://github.com/JetBrains/kotlin)
and unzip it to `/opt/kotlin`.
Create a symlink `/usr/bin/kotlinc` referring to `/opt/kotlin/bin/kotlinc` and
make sure `kotlinc` have executable flag set.
Make the `/opt/kotlin` directory accessible by the sandbox (add it to
`bound-directories` list).
#### Node.js (JavaScript)
@ -521,19 +588,74 @@ Optionally, you might want to consider installing User Operations for Zend
(`uopz`) PECL package for PHP. This package could help you creating hooks or
mock existing functions, which is helpful when testing PHP assignments. If you
enable this module, it is important to re-allow the source code control over the
exit opcode as follows:
Create a PHP script in a folder accessible by sandbox (e.g., `/usr/etc`):
exit opcode by adding
```
<?php
uopz_allow_exit(true);
uopz.exit = 1
```
Make sure this script is executed before any other (tested) code by setting
its path to `auto_prepend_file` in `/usr/etc/php.ini`.
Into `/usr/etc/php.ini`.
**If you share PHP installation between core API and worker, make sure the `uopz`
extension is either disabled or allows the exit code override for the API module!!!**
#### Prolog
Unfortunately, SWI Prolog has to be compiled manually. Do not forget to have
EPEL 8 and PowerTools repositories enabled.
Install dependencies:
```
# dnf -y install \
gcc \
gcc-c++ \
cmake3 \
ninja-build \
libunwind \
freetype-devel \
gmp-devel \
java-1.8.0-openjdk-devel \
jpackage-utils \
libICE-devel \
libjpeg-turbo-devel \
libSM-devel \
libX11-devel \
libXaw-devel \
libXext-devel \
libXft-devel \
libXinerama-devel \
libXmu-devel \
libXpm-devel \
libXrender-devel \
libXt-devel \
ncurses-devel \
openssl-devel \
pkgconfig \
readline-devel \
libedit-devel \
unixODBC-devel \
zlib-devel \
uuid-devel \
libarchive-devel \
libyaml-devel
```
Download SWI-Prolog 8.0.1. and compile it (it is recommended to do this in
dedicated directory as regular user):
```
$ wget http://www.swi-prolog.org/download/stable/src/swipl-8.0.1.tar.gz
$ tar xvf ./swipl-8.0.1.tar.gz
$ cd ./swipl-8.0.1
$ mkdir build
$ cd ./build
$ cmake3 ..
$ make
$ sudo make install
```
For technical reasons, the ReCodEx requires that the `swipl` is present in
`/usr/bin`. The simplest way is to create symlink `/usr/bin/swipl` that refers
to `/usr/local/lib/swipl/bin/x86_64-linux/swipl`.
Download `recodex-init.pl`, `recodex-swipl-wrapper.sh`, and `recodex-wrapper.pl`
from
```
@ -544,9 +666,7 @@ Open `Pipeline` page, find the *Prolog Compilation* pipeline, and click on
the `Edit` button on the right. Scroll down to *Supplementary files* box
and upload all three downloaded files here.
TODO
#### Python (`python3`)
#### Python
Python is executed in a wrapper script that handles exceptions and translates
them in exit codes (which is necessary for error reporting). Download
@ -575,7 +695,6 @@ VIRTUAL_ENV: /var/recodex-worker-python-venv
And register directory mapping in `bound-directories` list:
```
- src: "/var/recodex-worker-python-venv"
dst: "/var/recodex-worker-python-venv"
```
#### Rust
@ -601,3 +720,37 @@ configure the system to use latest java as default:
```
## Are we done?
Not quite yet. There are at least two things you should consider.
### Backup
The data are stored at two places -- in the database and in the file server.
The database can be easily dumped thusly:
```
mysqldump --default-character-set=utf8mb4 -uroot -p recodex > /path/to/backup/file.sql
```
Where `recodex` is the database name.
Furthermore, you need to backup the fileserver main directory
(`/var/recodex-fileserver` by default), preferably using tools like `rsync`.
In addition, it is a good idea to backup the `/opt/recodex-code/uploaded_data`
directory. Important uploaded files will be eventually all transferred to
fileserver, but at present, some of them are still kept here.
It might be a good idea to perform a backup every night and to keep several last
copies. For instance, in our setup, we keep last 7 daily backups, all backups made
on the 1st of every month for the last year, and all backups made on January 1st
of every year.
### Monitoring
For mission-critical systems, some form of monitoring is a must. Depending on your
needs, the monitoring could be as simple as setting up a script that uses ping
or wget to verify the ReCodEx is running, or you can use more sophisticated tools.
For our main instance, we use [Prometheus](https://prometheus.io/) with
`node_exporter` and `mysqld_exporter` to gather performance statistics and
[Grafana](https://grafana.com/) to visualize them.

Write Preview
Loading…
Cancel
Save