|
|
|
|
# Fileserver
|
|
|
|
|
|
|
|
|
|
The fileserver is a simple frontend to a disk storage space that contains
|
|
|
|
|
auxiliary files for assignments, files submitted by users and evaluation
|
|
|
|
|
results. For a description of the communication protocol used by the frontend
|
|
|
|
|
and workers, see the [Communication](#communication) chapter
|
|
|
|
|
|
|
|
|
|
## Description
|
|
|
|
|
|
|
|
|
|
The storage is implemented in Python, using the Flask web framework. This
|
|
|
|
|
particular implementation evolved from a simple mock fileserver we used in early
|
|
|
|
|
stages of development.
|
|
|
|
|
|
|
|
|
|
### Internal storage structure
|
|
|
|
|
|
|
|
|
|
Fileserver stores its data in a configurable filesystem folder. This folder has
|
|
|
|
|
the following subfolders:
|
|
|
|
|
|
|
|
|
|
- `./submissions/<id>` - folders that contain files submitted by users
|
|
|
|
|
(student's solutions to assignments). `<id>` is an identifier received from
|
|
|
|
|
the ReCodEx API.
|
|
|
|
|
- `./submission_archives/<id>.zip` - ZIP archives of all submissions. These are
|
|
|
|
|
created automatically when a submission is uploaded. `<id>` is an identifier
|
|
|
|
|
of the corresponding submission.
|
|
|
|
|
- `./tasks/<subkey>/<key>` - supplementary task files (e.g. test inputs and
|
|
|
|
|
outputs). `<key>` is a hash of the file content (sha-1 is used) and `<subkey>`
|
|
|
|
|
is its first letter (this is an attempt to prevent creating a flat directory
|
|
|
|
|
structure).
|
|
|
|
|
|
|
|
|
|
## Installation
|
|
|
|
|
|
|
|
|
|
To install and use the fileserver, it's necessary to have Python and Pip
|
|
|
|
|
installed.
|
|
|
|
|
|
|
|
|
|
1. Clone this repository
|
|
|
|
|
2. Install dependencies using `pip`
|
|
|
|
|
|
|
|
|
|
```
|
|
|
|
|
pip install -r requirements.txt
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
## Configuration and usage
|
|
|
|
|
|
|
|
|
|
There are several ways of running the ReCodEx fileserver. We'll cover two
|
|
|
|
|
typical use cases.
|
|
|
|
|
|
|
|
|
|
### Running in development mode
|
|
|
|
|
|
|
|
|
|
For simple development usage, it's possible to run the fileserver in the command
|
|
|
|
|
line:
|
|
|
|
|
|
|
|
|
|
~~~~~~~~~~~~~~~
|
|
|
|
|
python3 fileserver.py runserver
|
|
|
|
|
~~~~~~~~~~~~~~~
|
|
|
|
|
|
|
|
|
|
When run like this, the fileserver creates a temporary directory where it stores
|
|
|
|
|
all the files and which is deleted when it exits. It's possible to configure it
|
|
|
|
|
to use any other directory. Use the `--help` flag for a complete list of options
|
|
|
|
|
(such as the port where the server listens).
|
|
|
|
|
|
|
|
|
|
### Running with uWSGI and a web server
|
|
|
|
|
|
|
|
|
|
If you need features such as HTTP authentication or efficient serving of static
|
|
|
|
|
files, it is recommended to run the app in a full-fledged web server (such as
|
|
|
|
|
Apache or Nginx) using WSGI. Following examples are simplified, use the `--help`
|
|
|
|
|
flag of the `mkconfig.py` script to see all available options.
|
|
|
|
|
|
|
|
|
|
### Running using uWSGI
|
|
|
|
|
|
|
|
|
|
1. (Optional) Create a user for running the fileserver
|
|
|
|
|
2. Make sure that your user can access your clone of the repository
|
|
|
|
|
3. Run `python mkconfig/mkconfig.py uwsgi --port PORT --user YOUR_USER
|
|
|
|
|
--working-directory DIR`
|
|
|
|
|
4. Save the configuration file generated by the script and run it with uWSGI,
|
|
|
|
|
either directly or using systemd. This depends heavily on your distribution
|
|
|
|
|
5. To integrate this with another web server, see the [uWSGI
|
|
|
|
|
documentation](http://uwsgi-docs.readthedocs.io/en/latest/WebServers.html)
|
|
|
|
|
|
|
|
|
|
Note that the ways distributions package uWSGI can vary wildly. In Debian 8 it's
|
|
|
|
|
necessary to convert the configuration file to XML and make sure that the
|
|
|
|
|
python3 plugin is loaded instead of python. This plugin also uses Python 3.4,
|
|
|
|
|
even though the rest of the system uses Python 3.5 - make sure to install
|
|
|
|
|
dependencies for the correct version.
|
