@ -763,52 +763,83 @@ Only remaining part is evaluation of results. This is provided on demand when us
## Installation
## Installation
### Ansible installer
Installation of whole ReCodEx solution is a very complex process. It's recommended to have good unix skills with basic knowledge of project architecture.
### Hints to manuall install
There are a lot of different GNU/Linux distributions with different package management, naming convenction and version release policies. So it's impossible to cover all of the possible variants. We picked one distribution, which is fully supported by automatic installation script, for others there are brief informations about installation in every project component's own chapter.
This page contains steps how to set up a computer on which some parts of **ReCodEx** may run. Most steps are listed in two variants, for Red Hat based distributions (like RHEL, Centos or Fedora) and Debian based distibutions. Before starting, make sure you have completed basic OS installation and set up, including users and logins, SSH, Git, firewall, etc.
Distribution of our choice is CentOS, currently in version 7. It's a well known server distribution, derived from enterpreise distrubution from Red Hat, so it's very stable and widely used system with long term support. There are [EPEL](https://fedoraproject.org/wiki/EPEL) additional repositories from Fedora project, which adds newer version of some packages into CentOS, which allows us to use current environment. Also, _rpm_ packages are much easier to build (for example from Python sources) and maintain.
#### Repositories
The big rival of CentOS in server distributions field is Debian. We're running one instance of ReCodEx on Debian too. You need to use _testing_ repositories to use some decent package versions. It's easy to mess your system easily, so create file `/etc/apt/apt.conf` with content of `APT::Default-Release "stable";`. After you add testing repos to `/etc/apt/sources.list`, you can install packages from there like `$ sudo apt-get -t testing install gcc`.
**Add _testing_ repositories to Debian OS**
### Ansible installer
- Create file `/etc/apt/apt.conf` with content
```
For automatic installation is used set of Ansible scripts. Ansible is one of the best known and used tools for automatic server management. It's required only to have SSH access to the server and ansible installed on the client machine. For further reading is supposed basic Ansible knowledge. For more info check their [documentation](http://docs.ansible.com/ansible/intro.html).
APT::Default-Release "stable";
```
- Add testing repos to `/etc/apt/sources.list`
```
deb http://ftp.cz.debian.org/debian/ testing main contrib non-free
- Install packages with `-t testing` option. For example
```
$ sudo apt-get -t testing install gcc
```
**Add EPEL repository to RHEL**
- See [this](https://fedoraproject.org/wiki/EPEL) for instructions.
All Ansible scripts are located in _utils_ repository, _installation_ [directory](https://github.com/ReCodEx/utils/tree/master/installation). Before installation itself it's required to edit two files -- set addresses of hosts and values of some variables.
#### Installation of dependencies
#### Hosts configuration
Install as new version of each package as possible, so mostly Debian packages are from testing repositories and RHEL packages are the newest ones from EPEL repositories.
First, it's needed to set ip addresses of your computers. Common practise is to have multiple files with definitions, one for development, another for production for example. Example configuration is in _development_ file. Each component of ReCodEx project can be installed on different server. Hosts can be specified as hostnames or ip addresses, optionally with port of SSH after colon.
**Basic development tools**
Shorten example of hosts config:
- `g++`
- `cmake`
- `make`
- `git`
**Install ZeroMQ** in version at least 4.0
```
- Debian package is `libzmq3-dev`.
[workers]
- RedHat packages are `zeromq` and `zeromq-devel`.
127.0.0.1:235
[broker]
127.0.0.1:235
**Install YAML-CPP library**
[all:children]
- Debian packages: `libyaml-cpp0.5v5` and `libyaml-cpp-dev`.
workers
- RedHat packages are `yaml-cpp` and `yaml-cpp-devel`.
broker
```
**Install libcurl library**
#### Variables
- Debian package is `libcurl4-gnutls-dev`.
- RedHat package is `libcurl-devel`.
Configurable variables are saved in _group_vars/all.yml_ file. Syntax is basic key-value pair per line, separated by colon. Values with brief description:
- _source_dir_ -- Directory, where to store all sources from GitHub. Defaults `/opt/recodex`.
- _mysql_root_password_ -- Password of root user of MySQL database. Will be set after installation and saved to `/root/.my.cnf` file.
- _mysql_recodex_username_ -- MySQL username for ReCodEx API access.
- _mysql_recodex_password_ -- Password for the user above.
- _admin_email_: Email of administrator. Used when configuring Apache webserver.
- _recodex_hostname: Hostname where the API and web app will be accessible. For example "recodex.projekty.ms.mff.cuni.cz".
- _webapp_node_addr_ -- IP address of NodeJS server running web app. Defaults to "127.0.0.1" and should not be chnaged.
- _webapp_node_port_ -- Port to above.
- _webapp_public_addr_ -- Public address, where web server for web app will listen. Defaults to "*".
- _webapp_public_port_ -- Port to above.
- _webapp_firewall_ -- Open port for web app in firewall, values "yes" or "no".
- _webapi_public_endpoint_ -- Public URL when the API will be running, for example "https://recodex.projekty.ms.mff.cuni.cz:4000/v1".
- _webapi_public_addr_ -- Public address, where web server for API will listen. Defaults to "*".
- _webapi_public_port_ -- Port to above.
- _webapi_firewall_ - Open port for API in firewall, values "yes" or "no".
- _database_firewall_ - Open port for database in firewall, values "yes" or "no".
- _broker_to_webapi_addr_ -- Address, where API can reach broker. Private one is recommended.
- _broker_to_webapi_port_ -- Port to above.
- _broker_firewall_api_ -- Open above port in firewall, "yes" or "no".
- _broker_to_workers_addr_ -- Address, where workers can reach broker. Private one is recommended.
- _broker_to_workers_port_ -- Port to above.
- _broker_firewall_workers_ -- Open above port in firewall, "yes" or "no".
- _broker_notifier_address_ -- URL (on API), where broker will send notifications, for example "https://recodex.projekty.ms.mff.cuni.cz/v1/broker-reports".
- _broker_notifier_port_ -- Port to above, should be the same as for API itself (_webapi_public_port_)
- _broker_notifier_username_ -- Username for HTTP Authentication for reports
- _broker_notifier_password_ -- Password for HTTP Authentication for reporst
- _monitor_websocket_addr_ -- Address, where websocket connection from monitor will be available
- _monitor_websocket_port_ -- Port to above.
- _monitor_firewall_websocket_ -- Open above port in firewall, "yes" or "no".
- _monitor_zeromq_addr_ -- Address, where monitor will be available on ZeroMQ socket for broker to receive reports.
- _monitor_zeromq_port_ -- Port to above.
- _monitor_firewall_zeromq_ -- Open above port in firewall, "yes" or "no".
- _fileserver_addr_ -- Address, where fileserver will serve files.
- _fileserver_port_ -- Port to above.
- _fileserver_firewall_ -- Open above port in firewall, "yes" or "no".
- _fileserver_username_ -- Username for HTTP Authentication for access the fileserver.
- _fileserver_password_ -- Password for HTTP Authentication for access the fileserver.
- _worker_cache_dir_ -- File cache storage for workers. Defaults to "/tmp/recodex/cache".
- _worker_cache_age_ -- How long hold fetched files in worker cache, in seconds.
- _isolate_version_ -- Git tag of Isolate version worker depends on.
@ -53,11 +53,28 @@ Picture below is internal architecture of worker which shows its defined classes
## Installation
## Installation
### Dependencies
### Dependencies
Worker specific requirements are written in this section. Some parts of this guide may be different for each type of worker, for example Compiler principles worker is not fully covered here.
Worker specific requirements are written in this section. Some parts of this guide may be different for each type of worker, for example Compiler principles worker is not fully covered here.
**Install ZeroMQ** in version at least 4.0
- Debian package is `libzmq3-dev`.
- RedHat packages are `zeromq` and `zeromq-devel`.
**Install YAML-CPP library**
- Debian packages: `libyaml-cpp0.5v5` and `libyaml-cpp-dev`.
- RedHat packages are `yaml-cpp` and `yaml-cpp-devel`.
**Install libcurl library**
- Debian package is `libcurl4-gnutls-dev`.
- RedHat package is `libcurl-devel`.
**Install libarchive library** (optional)
**Install libarchive library** (optional)
Installing this package will only speed up build process, otherwise libarchive is built from source.
Installing this package will only speed up build process, otherwise libarchive is built from source.
- Debian package is `libarchive-dev`.
- Debian package is `libarchive-dev`.
- RedHat packages are `libarchive` and `libarchive-devel`. These are probably not available for RHEL.
- RedHat packages are `libarchive` and `libarchive-devel`. These are probably not available for RHEL.
Petr Stefan
8 years ago