You cannot select more than 25 topics
Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
36 lines
2.1 KiB
Markdown
36 lines
2.1 KiB
Markdown
# Fileserver
|
|
|
|
The fileserver is a simple frontend to a disk storage space that contains auxiliary files for assignments, archives with job configuration and files submitted by users and evaluation results. These files are the only ones required for backend to run, so dedicated fileserver gives the possibility of testing backend separately. Also, one fileserver instance could be shared among multiple API instances (with the same broker), so common files does not need to be duplicated in each API instance.
|
|
|
|
One exception is that important files with character of database entry (but not stored in database due to size) are stored directly in filesystem of API server. But this fact does not devaluate benefit of separate fileserver. From security point of view, fileserver should be completely isolated from public internet to keep the data safe while API server must be public from its nature.
|
|
|
|
For a description of the communication protocol used by the frontend
|
|
and workers, see the [Communication](#communication) chapter.
|
|
|
|
|
|
## Description
|
|
|
|
The storage is implemented in Python, using the Flask web framework. This
|
|
particular implementation evolved from a simple mock fileserver we used in early
|
|
stages of development. It prooved to be very reliable, so we decided to keep fileserver
|
|
as separate component instead of integrating this functionality into main API.
|
|
|
|
### Internal storage structure
|
|
|
|
Fileserver stores its data in a configurable filesystem folder. This folder has
|
|
the following subfolders:
|
|
|
|
- `./submissions/<id>` -- folders that contain files submitted by users
|
|
(student's solutions to assignments). `<id>` is an identifier received from
|
|
the ReCodEx API.
|
|
- `./submission_archives/<id>.zip` -- ZIP archives of all submissions. These are
|
|
created automatically when a submission is uploaded. `<id>` is an identifier
|
|
of the corresponding submission.
|
|
- `./tasks/<subkey>/<key>` -- supplementary task files (e.g. test inputs and
|
|
outputs). `<key>` is a hash of the file content (sha-1 is used) and `<subkey>`
|
|
is its first letter (this is an attempt to prevent creating a flat directory
|
|
structure).
|
|
|
|
|
|
|