LEdoian
/
recodex-wiki
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
You cannot select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
fixes
master
Branches Tags
${ item.name }
Create tag ${ searchTerm }
Create branch ${ searchTerm }
from 'b3dfd6b3b2'
${ noResults }
recodex-wiki/Installation.md

14 KiB
Raw Blame History

This tutorial describes a complete installation and configuration of ReCodEx on CentOS 8 system as it was conducted in August 2020. Details of this description may vary for other systems (RPM packages are released for CentOS and Fedora only) and possibly also change in the future.

We are trying to make this tutorial for linux noobs, but some admin skills are still required.

For more details about the individual modules, please see their readme pages.

Prerequisites

Before we get started, make sure that you are using a file system that supports ACLs. If you are doing fresh install of modern distro, it should not be a problem. Filesystems like xfs and zfs use ACLs always. Older filesystems like ext4 use ACLs unless you explicitly disable them. However, if you are using more obscure FS, make sure ACLs are in place (ReCodEx will work without ACLs, but all recodex-core CLI commands would have to be executed under apache user).

After minimal installation of CentOS 8 (with enabled EPEL and Power Tools repos) install the following:

  • Apache 2.4 (httpd service), configure it, install SSL certificates, and open firewall for HTTPS
  • MariaDB (10.3 or newer); it is also recommended to secure the DB properly (set root password etc.)
  • PHP 7.3 or newer
  • Node.js 12.x or newer (14.x recommended)

A few tips for installing the database:

# dnf install httpd mariadb-server
# mysql_secure_installation

You can do this by running mysql -uroot -p and then executing these SQLs:

CREATE DATABASE `recodex`;
CREATE USER 'recodex'@'localhost' IDENTIFIED BY 'someSecretPasswordYouNeedToSetYourself';
GRANT ALL PRIVILEGES ON `recodex`.* TO 'recodex'@'localhost';

Configuring the PHP repository:

# dnf install dnf-utils http://rpms.remirepo.net/enterprise/remi-release-8.rpm
# dnf module enable php:remi-7.3

Installing Node.js:

# dnf -y install curl
# curl -sL https://rpm.nodesource.com/setup_14.x | bash -
# dnf -y install nodejs

You may check the right version is installed by typing node -v, which should report version of Node.js (starting with 14).

Install ReCodEx RPM Packages

Although individual components may run on different servers, typical deployments would install everything on a single server or split only the worker(s) to a separate server (e.g., when ReCodEx is deployed in VM and workers need to run on bare metal so they can provide more accurate measurements).

The main part of ReCodEx is installed as follows:

# dnf copr enable semai/ReCodEx
# dnf install recodex-core recodex-web recodex-broker recodex-monitor recodex-fileserver

The worker (and its utility cleaner) is installed thusly:

# dnf install recodex-worker recodex-cleaner

Please note, that if you install worker on another server, it is strongly recommended to secure the connection between these two servers (by VPN or IPSec tunnel).

If successful, the following systemd services are now available:

  • recodex-web
  • recodex-broker
  • recodex-monitor
  • recodex-fileserver

The core API runs under web server (does not need a custom service) and workers will be covered separately later.

All these services should not be running right after installation. They need to be configured first and then you can start and enable them:

systemctl start <service-name>
systemctl enable <service-name>
systemctl status <service-name>

The last command should show status of the service, which should be running.

Configure The Installation

Fileserver

The fileserver runs under mod_wsgi in Apache, so its configuration is in /etc/httpd/conf.d/010-fileserver.conf. There should be no need to edit this config file. You need only to se the HTTP authentication credentials and match them with credentials in core module config (fileServer > auth structure).

The credentials are in /etc/httpd/recodex_htpasswd. You may set them using htpasswd, an Apache CLI tool for generating auth config files. Do not forget to restart your web server after you are done:

#> systemctl restart httpd

Broker

Broker configuration is in /etc/recodex/broker/config.yml. The most important thing you need to set up is the communication route to API (see notifier structure). The address must point to API URL (as configured on HTTP server) suffixed with /v1/broker-reports. The username and password must match credentials in broker > auth structure in core module configuration.

By default, broker listens on all interfaces (represented by * in address fields) both for clients and for workers. Based on your deployment, it might be advisable to restrict the listening for specific addresses, namely for 127.0.0.1 if the core/workers run on the same machine. If you choose to change the ports, do not forget to change them in core and workers configurations as well.

Monitor

Monitor configuration is in /etc/recodex/broker/config.yml. Make sure the zeromq_uri (address and port) matches configuration of broker (monitor structure).

The websocket_uri sets listening address and port where the web socket server runs. Make sure the web server redirects all WebSocket requests here. For Apache with proxy module, add the following lines to the site configuration (assuming you did not change the configuration of the monitor):

ProxyPass /ws/ ws://127.0.0.1:4567/
ProxyPass /ws ws://127.0.0.1:4567/

Core (API)

Before getting started, make sure the core directory is duly configured in your web server and the server is capable of executing PHP in this directory.

The core module has configuration stored in /etc/recodex/core-api/config.local.neon. This is perhaps the most important config, so let us go through it step by step. Furthermore, it is necessary to invoke /opt/recodex-core/cleaner script after every modification of the config file. The cleaner will purge internal Nette caches with pre-generated PHP files (so they can be regenerated automatically again).

You may refer to already set parameters by using references. E.g., %webapp.address% used in a string will actually insert the webapp > address parameter value (which you need to setup in the first step of the following list).

  1. Set URL of web application and the API (webapp > address and api > address). The API URL must correspond to outside address as perceived by clients (i.e., the web application), especially when you use proxies or mod_rewrite in your setup.

  2. Setup access manager. The issuer and audience should match your web app domain. Furthermore, the verificationKey should be set to a long (possibly random) secret string. This string is used to sign and verify security tokens. Btw. if you need to invalidate all ReCodEx security tokens at once, just modify this string (that will effectively sing everybody off, so everyone will need to go through login process again).

  3. Configure fileServer connection. Under normal circumstances, you just need to fill in the credentials you have stored in /etc/httpd/recodex_htpasswd.

  4. In the broker structure, auth must hold credentials that match those set in broker configuration (notifier structure) and the address must provide URL with TCP protocol pointing to clients interface of the broker.

  5. Monitor address must be set to the external address where the monitor is listening for web sockets. If you used proxy pass as suggested in Monitor configuration, the address should be something like wss://your.recodex.domain:443/ws. The 443 port makes sure the initial handshake is done in HTTPS manner by Apache.

  6. Setup generated URLs in notification emails. The footerUrl should be the base URL of the web application. The from parameter configures the From: field set in all notification mails. The defaultAdminTo should be a string or an array of strings with email addresses where the error notifications will be sent. Error emails may contain sensitive information so it is highly recommended to send them to actual administrators of ReCodEx only. On the other hand, it is a good idea to have more than one administrator to reduce the chance of overlooking these failures.

  7. Set your SMTP configuration in the mail structure. SMTP is necessary so the API can send notification emails. You may temporary use ReCodEx without emails (setting emails > debugMode to true), but emails are required for key features like resetting forgotten password.

  8. Although this is the last step, it is perhaps the most important one. Fill in your database credentials of the recodex user (which you were supposed to create at the very beginning) into doctrine configuration (Doctrine framework is responsible for database interface in the core module).

There are many more configuration parameters. For inspiration, you may take a look in config.neon file, but always remember to edit the config.local.neon which works as an override of config.neon. The config.neon file may be updated in the future releases. However, the list

Finally, you need to set up a database. Switch to /opt/recodex-core directory (that is important) and execute

# su -c 'php www/index.php migrations:migrate' recodex

You may see warnings that some migrations did not execute any SQL statements, which is all right since there are no data in the DB yet. However, the whole migration process must not end with an error.

Note that the migration should be also executed after every recodex-core package upgrade!

Web application frontend

Web application has configuration in /etc/recodex/web-app/env.json. The most important thing to configure is API_BASE, the external URL of the API (as configured in Apache and in previous step). Also the PERSISTENT_TOKENS_KEY_PREFIX might be important, if you are running multiple installations of ReCodEx on single domain (this prefix is used for local storage and cookies, so the data of different instances are prefixed with different keys).

If you want to enable public registration to ReCodEx (use with caution), set ALLOW_NORMAL_REGISTRATION to true and do not forget to enable this feature in API (localRegistration > enabled).

The web application runs locally on Node.js server. The port is also configured in env.json. If you use Apache as your http frontend, you may need to set up a proxy for your web application:

ProxyPass / http://127.0.0.1:8080/
ProxyPassReverse / http://127.0.0.1:8080/

Finalization

When all components are working together, consider switching the logger > level from debug to info for broker and monitor (and do not forget to restart the services).

It is also recommended that you fill in the initial data into the database:

# su -c 'php www/index.php db:fill init' recodex

After executing the fill command the database will contain:

  • Instance with administrator registered as local account with credentials user name: admin@admin.com, password: admin
  • Runtime environments which ReCodEx can handle
  • Default single hardware group which might be used for workers
  • Pipelines for runtime environments which can be used when building exercises

To modify the data further, you might want to set up some database administration tool. We are shipping the Adminer along with core module, so it should be directly available under your-api-url/adminer. If you do not want to disable it, configure your HTTP server to deny access to www/adminer folder. You may use phpMyAdmin as an alternative.

Finally, there are several commands that should be executed periodically. All commands are executed similarly as db commands we used earlier:

php /opt/recodex-core/www/index.php command:name

The important commands are

  • notifications:assignment-deadlines will send emails to students (who actually allowed this in their configurations) about approaching deadlines. This is the most important command as it is directly related to ReCodEx operations. It is recommended to run this command every night.
  • db:cleanup:uploads will remove old uploaded files
  • db:cleanup:localized-texts will remove old texts of groups and exercises
  • db:cleanup:exercise-configs will remove old exercise configs
  • db:cleanup:pipeline-configs will remove old pipeline configs
  • users:remove-inactive will soft-delete and anonymize users who are deemed inactive (i.e., they have not verified their credentials for a period of time that is set in core module config)
  • notifications:general-stats will send an email to all administrators with brief statistics about ReCodEx usage. It is recommended to run this command once a week (e.g., during the night between Saturday and Monday).

The frequency of cleanup commands depend on your system utilization. In intensively utilized instances, it might be prudent to call cleanups at least once a week. In case of mostly idle instances, cleanup once per month or even once per year may be sufficient.

One option is to create a script (or multiple scripts) and schedule their execution in crontab. Do not forget to run these commands as recodex user. For example, adding the following line in /etc/crontab will execute your cleanup script every day at 3 AM under recodex user.

0 3 * * *	recodex /path/to/your/cleanup/script.sh

Set-up Workers and Environments

To be documented yet...