recodex-wiki/Architecture.md

# Overall Architecture
![Overall Architecture](https://github.com/ReCodEx/GlobalWiki/blob/master/images/Overall_Architecture.png)

**ReCodEx** is designed to be very modular. **WebApp** is considered to be frontend part of whole system while all other parts/services are considered to be backend part. **WebApp** contains almost all logic of the app including _user management and authentication_, _storing and versioning files_ (through connection with **Fileserver**), _counting and assigning points_ to users etc. **WebApp** is connected with **Broker**, **Fileserver** and partially with **Monitor**. **Broker** is essential part of whole architecture and can be marked as single point of failure.

Almost whole communication goes through **Broker** and `ZeroMQ` messaging middleware. When **WebApp** wants to execute submission then all datas are handed over to **Worker** through **Broker**, similar situation is with progress state which start in **Worker** goes through **Broker** then pass **Monitor** and end up in **WebApp**. Only part of communication, which does not include **Broker**, is communication with **Fileserver** which is realized through `HTTP` commmunication. This communication can be initiated by **WebApp** or by **Worker**, other services have no access to **Fileserver**. Summarize of above stated can be found in overall architecture image (green arrows = `HTTP` communication, red arrows = communication through `ZeroMQ`).

## WebApp

- TODO

## Broker

- TODO

## Worker
![Worker Architecture](https://github.com/ReCodEx/GlobalWiki/blob/master/images/Worker_Architecture.png)

**Worker's** main role is securely execute given submission and possibly _evaluate_ results against model solutions provided by submitter. **Worker** is logicaly divided into two parts:
- **Listener** - listens and communicates with **Broker** through [ZeroMQ](http://zeromq.org/). It receives new jobs, communicates with **Evaluator** part and sends back results or progress.
- **Evaluator** - gets jobs to evaluate from **Listener** part, evaluate them (possibly in sandbox) and get to know to other part that evaluation ended. This part also communicates with **Fileserver**, downloads needed files and uploads detailed results.

**Worker** after getting evaluation request has to:

- Download the archive containing submitted source files and configuration file
- Download any supplementary files based on the configuration file, such as test 
  inputs or helper programs (This is done on demand, using a `fetch` command
  in the assignment configuration)
- Evaluate the submission accordingly to job configuration
- During evaluation progress states can be sent back to **Broker**
- Upload the results of the evaluation to the **Fileserver**
- Notify **Broker** that the evaluation finished

### Internal Worker architecture
Picture below is overall internal architecture of worker which shows its defined classes with private variables and public functions. Vector version of this picture is available [here](https://github.com/ReCodEx/GlobalWiki/raw/master/images/Worker_Internal_Architecture.pdf).
![Internal Worker architecture](https://github.com/ReCodEx/GlobalWiki/blob/master/images/Worker_Internal_Architecture.png)

## File Server
![File Server Infrastructure](https://github.com/ReCodEx/GlobalWiki/blob/master/images/File_Server.png)

**File Server** stores data, that should be kept outside of **WebApp's** 
database (both because storing files in a database is inefficient and because 
the workers need to access the files in the simplest possible way). It should 
meet following requirements:
- store files without duplicates
- keep consistent state with main database
- serve files to workers on demand
- allow versioning of tasks with revert back feature

To meet these requirements, **Storage** and **Database** must be set as bellow.

### Storage
**Storage** is meant as disc space with some commonly used filesystem. We'll use `ext4`, but the other ones should work too. **Storage** file structure is:
```
.
├── submits
│   └── user_id
│       └── advanced_dot_net_1
│           └── submit_id
│               ├── eval.yml
│               └── source.cs
├── submit_archives
│   └── submit_id.tar.gz
├── tasks
│   ├── a
│   │   ├── a014ed2abb56371bfaf2b4298a85d5dfb56509ed
│   │   └── a5edbd8b12e670ed1e3110d6c0524000cd4c3c7a
│   └── b
│       └── b1696358b8540923eb79b68f95c0f94c13a83fa7
└── temp
    └── 1795184136b8bdddabe50453cc2cc2d46f0f7c5e
```
- **submits** keep information about all files submited by users to ReCodEx. 
  There are subdirectories _user_id_ and _advanced_dot_net_1_ which groups
  submits by users and courses the submits are for. This structure is easy to 
  maintain for new and deleted users.
- **submit_archives** contains the student submissions in compressed archives so 
  that they can be easily downloaded by workers.
- **tasks** contains supplementary files (such as test inputs or helper 
  programs) for all existing task in ReCodEx. To avoid too many files in one 
  directory, files are separated to subfolders by first character of their name.
- **temp** directory is dedicated to temporary storing outputs of programs on teachers' demand. This directory will be erased by cron job on daily basis.

### Database
For user friendly access and modifying tasks following information should be stored in database:
- list of tasks with their newest version number
- for every task and version list of used files (their hashed names)
- for every hash name one human readable filename

### Conclusion
Files are internally stored by their `sha1sum` hashes, so it's easy to implement 
versioning and get rid of files with duplicate content (multiple files can have 
the same content, which is only stored once). **Worker** also uses files by 
their hashes, which is great for local caching without worries about actual 
version number of given file. On the other hand, **Database** stores information 
about human readable names, so that the files are presented in a friendly way to 
users (teachers) in **WebApp**.

## Monitor

- TODO