From bcefaf1f669807d683afc74a557e920716eafae4 Mon Sep 17 00:00:00 2001 From: Teyras Date: Sat, 8 Oct 2016 11:54:04 +0200 Subject: [PATCH] add fileserver doc --- Fileserver.md | 78 +++++++++++++++++++++++++++++++++++++++++++++++++-- 1 file changed, 76 insertions(+), 2 deletions(-) diff --git a/Fileserver.md b/Fileserver.md index f754327..80b1383 100644 --- a/Fileserver.md +++ b/Fileserver.md @@ -1,9 +1,83 @@ # Fileserver +The fileserver is a simple frontend to a disk storage space that contains +auxiliary files for assignments, files submitted by users and evaluation +results. For a description of the communication protocol used by the frontend +and workers, see the [Communication](#communication) chapter + ## Description -## Architecture +The storage is implemented in Python, using the Flask web framework. This +particular implementation evolved from a simple mock fileserver we used in early +stages of development. + +### Internal storage structure + +Fileserver stores its data in a configurable filesystem folder. This folder has +the following subfolders: + +- `./submissions/` - folders that contain files submitted by users + (student's solutions to assignments). `` is an identifier received from + the ReCodEx API. +- `./submission_archives/.zip` - ZIP archives of all submissions. These are + created automatically when a submission is uploaded. `` is an identifier + of the corresponding submission. +- `./tasks//` - supplementary task files (e.g. test inputs and + outputs). `` is a hash of the file content (sha-1 is used) and `` + is its first letter (this is an attempt to prevent creating a flat directory + structure). ## Installation -## Configuration and usage \ No newline at end of file +To install and use the fileserver, it's necessary to have Python and Pip +installed. + +1. Clone this repository +2. Install dependencies using `pip` + +``` +pip install -r requirements.txt +``` + +## Configuration and usage + +There are several ways of running the ReCodEx fileserver. We'll cover two +typical use cases. + +### Running in development mode + +For simple development usage, it's possible to run the fileserver in the command +line: + +~~~~~~~~~~~~~~~ +python3 fileserver.py runserver +~~~~~~~~~~~~~~~ + +When run like this, the fileserver creates a temporary directory where it stores +all the files and which is deleted when it exits. It's possible to configure it +to use any other directory. Use the `--help` flag for a complete list of options +(such as the port where the server listens). + +### Running with uWSGI and a web server + +If you need features such as HTTP authentication or efficient serving of static +files, it is recommended to run the app in a full-fledged web server (such as +Apache or Nginx) using WSGI. Following examples are simplified, use the `--help` +flag of the `mkconfig.py` script to see all available options. + +### Running using uWSGI + +1. (Optional) Create a user for running the fileserver +2. Make sure that your user can access your clone of the repository +3. Run `python mkconfig/mkconfig.py uwsgi --port PORT --user YOUR_USER + --working-directory DIR` +4. Save the configuration file generated by the script and run it with uWSGI, + either directly or using systemd. This depends heavily on your distribution +5. To integrate this with another web server, see the [uWSGI + documentation](http://uwsgi-docs.readthedocs.io/en/latest/WebServers.html) + +Note that the ways distributions package uWSGI can vary wildly. In Debian 8 it's +necessary to convert the configuration file to XML and make sure that the +python3 plugin is loaded instead of python. This plugin also uses Python 3.4, +even though the rest of the system uses Python 3.5 - make sure to install +dependencies for the correct version.