![File Server Infrastructure](https://github.com/ReCodEx/GlobalWiki/blob/master/images/File_Server.png)
![File Server Infrastructure](https://github.com/ReCodEx/GlobalWiki/blob/master/images/File_Server.png)
**File Server** stores data, which is better keep out of **WebApp's** database. It should meet following requirements:
**File Server** stores data, that should be kept outside of **WebApp's**
database (both because storing files in a database is inefficient and because
the workers need to access the files in the simplest possible way). It should
meet following requirements:
- store files without duplicates
- store files without duplicates
- keep consistent state with main database
- keep consistent state with main database
- serve files to workers on demand
- serve files to workers on demand
@ -76,8 +79,14 @@ To meet these requirements, **Storage** and **Database** must be set as bellow.
└── temp
└── temp
└── 1795184136b8bdddabe50453cc2cc2d46f0f7c5e
└── 1795184136b8bdddabe50453cc2cc2d46f0f7c5e
```
```
- **submits** keep information about all files submited by users to ReCodEx. There are subdirectoris _user_id_ and _advanced_dot_net_1_ which divides submits by users and class they took part of. This structure is easy to maintain for new and deleted users.
- **submits** keep information about all files submited by users to ReCodEx.
- **tasks** contains all files for all existing task in ReCodEx. To avoid too much files in one directory, files are separated to subfolders by first character of their name.
There are subdirectories _user_id_ and _advanced_dot_net_1_ which groups
submits by users and courses the submits are for. This structure is easy to
maintain for new and deleted users.
- **tasks** contains all files for all existing task in ReCodEx. To avoid too
many files in one directory, files are separated to subfolders by first
character of their name.
- **temp** directory is dedicated to temporary storing outputs of programs on teachers' demand. This directory will be erased by cron job on daily basis.
- **temp** directory is dedicated to temporary storing outputs of programs on teachers' demand. This directory will be erased by cron job on daily basis.
### Database
### Database
@ -87,7 +96,13 @@ For user friendly access and modifying tasks following information should be sto
- for every hash name one human readable filename
- for every hash name one human readable filename
### Conclusion
### Conclusion
Files are internally stored by their `sha1sum` hashes, so it's easy to implement versioning and get rid of files with duplicity content. **Worker** also uses files by their hashes, which is great for local caching without worries about actual version number of given file. On the other hand, **Database** keeps info about human readable names, so to users (teachers) in **WebApp** are files presented in a friendly way.
Files are internally stored by their `sha1sum` hashes, so it's easy to implement
versioning and get rid of files with duplicate content (multiple files can have
the same content, which is only stored once). **Worker** also uses files by
their hashes, which is great for local caching without worries about actual
version number of given file. On the other hand, **Database** stores information
about human readable names, so that the files are presented in a friendly way to
users (teachers) in **WebApp**.
## Frontend - broker communication
## Frontend - broker communication
@ -108,6 +123,11 @@ If the broker is capable of routing the request to a worker, it responds with
`ack`. Otherwise (for example when the requirements specified by the headers
`ack`. Otherwise (for example when the requirements specified by the headers
cannot be met), it responds with `nack`.
cannot be met), it responds with `nack`.
Note that we will need to store the job ID and the assignment configuration
somewhere close to the submitted files so it's possible to check how a
submission was evaluated. The job ID will likely be a part of the submission's
path. The configuration could be linked there under some well-known name.
### Notifying the frontend about evaluation progress
### Notifying the frontend about evaluation progress
The script that requested the evaluation will have exited by the time a worker
The script that requested the evaluation will have exited by the time a worker