Every project should have some consistent coding style in which all contributors write. Bellow you can find our conventions on which we agreed on and which we try to keep.
**ReCodEx** is designed to be very modular. In the following picture main components are arranged into one possible configuration. Note, that connections between components are not fully accurate.
**NOTE, that C++ projects have set code linter (`cmake-format`) with custom format. To reformat code run `make format` inside `build` directory of the project (probably not working on Windows).** For quick introduction into our format, see following paragraphs.
**Web app** is main part of whole project for users. It provides nice user interface and is the only part, that interacts with outside world directly. **Web API** contains almost all logic of the app including _user management and authentication_, _storing and versioning files_ (with help of **File server**), _counting and assigning points_ to users etc. **Broker** is essential part of whole architecture and can be marked as single point of failure. It maintains list of available **Workers**, receives submissions from the **Web API** and routes them further and reports progress of evaluations back to the **Web app**. **Worker** securely runs each received job and evaluate it's results. **Monitor** resends evaluation progress messages to the **Web app** in order to be presented to users.
In C++ is written worker and queue manager. Generally its used underscore style with all small letters. Inspired by [Google C++ style guide](https://google.github.io/styleguide/cppguide.html). If something is not defined than naming/formatting can be arbitrary, but should be similar to bellow-defined behaviour.
Almost whole communication goes through **Broker** and ZeroMQ messaging middleware. When **Web app** wants to execute submission then all datas are handed over to **Worker** through **Broker**, similar situation is with progress state which start in **Worker** goes through **Broker** then pass **Monitor** and end up in **Web app** (as WebSockets). Only part of communication, which does not include **Broker**, is communication with **File server** which is realized through HTTP commmunication. This communication can be initiated by **Web API** or by **Worker**, other services have no access to **File server**. Detailed view into communication is on separate page [[Communication]].
#### Naming convention
* For source codes use all lower case with underscores not dashes. Header files should end with `.h` and C++ files with `.cpp`.
* Typenames are all in lower case with underscores between words. This is applicable to classes, structs, typedefs, enums and type template parameters.
* Variable names can be divided on local variables and class members. Local variables are all lower case with underscores between words. Class members have in addition trailing underscore on the end (struct data members do not have underscore on the end).
* Constants are just like any other variables and do not have any specifics.
* All function names are again all lower case with underscores between words.
* Namespaces if there are ones they should have lower case and underscores.
* Macros are classical and should have all capitals and underscores.
* Comments can be two types documentational and ordinery ones in code. Documentation should start with `/**` and end with `*/`, convention inside them is javadoc documentation format. Classical comments in code are one liners which starts with `//` and end with the end of the line.
## Worker
#### Formatting convention
* Line length is not explicitly defined, but should be reasonable.
* All files should use UTF-8 character set.
* For code indentation tabs (`\t`) are used.
* Function declaration/definition: return type should be on the same line as the rest of the declaration, if line is too long, than particular parameters are placed on new line. Opening parenthesis of function should be placed on new line bellow declaration. Its possible to write small function which can be on only one line. Between parameter and comma should be one space.
```
int run(int id, string msg);
**Worker's** main role is securely execute given submission and possibly _evaluate_ results against model solutions provided by submitter. **Worker** is logicaly divided into two parts:
- **Listener** - listens and communicates with **Broker** through [ZeroMQ](http://zeromq.org/). It receives new jobs, communicates with **Evaluator** part and sends back results or progress.
- **Evaluator** - gets jobs to evaluate from **Listener** part, evaluate them (possibly in sandbox) and get to know to other part that evaluation ended. This part also communicates with **Fileserver**, downloads needed files and uploads detailed results.
void print_hello_world()
{
std::cout << "Hello world" <<std::endl;
return;
}
**Worker** after getting evaluation request has to:
int get_five() { return 5; }
```
* Lambda expressions: same formatting as classical functions
```
auto hello = [](int x) { std::cout << "hello_" <<x<<std::endl;}
```
* Function calls: basically same as function header definition.
* Condition: after if, or else there always have to be one space in front of opening bracket and again one space after closing condition bracket (and in front of opening parenthesis). If and else always should be on separate lines. Inside condition there should not be any pointless spaces.
```
if (x == 5) {
std::cout << "Exactly five!" <<std::endl;
} else if (x <5&&y> 5) {
std::cout << "Whoa, that is weird format!" <<std::endl;
} else {
std::cout << "I dont know what is this!" <<std::endl;
}
```
* For and while cycles: basically same rules as for if condition.
* Try-catch blocks: again same rules as for if conditions. Closing parentheses of try block should be on the same line as catch block.
```
try {
int a = 5 / 0;
} catch (...) {
std::cout << "Division by zero" <<std::endl;
}
```
* Switch: again basics are the same as for if condition. Case statements should not be indented and case body should be intended with 1 tab.
```
switch (switched) {
case 0: // no tab indent
... // 1 tab indent
break;
case 1:
...
break;
default:
exit(1);
}
```
* Pointers and references: no spaces between period or arrow in accessing type member. No spaces after asterisk or ampersand. In declaration of pointer or reference format should be that asterisk or ampersand is adjacent to name of the variable not type.
```
number = *ptr;
ptr = &val;
number = ptr->number;
number = val_ref.number;
- Download the archive containing submitted source files and configuration file
- Download any supplementary files based on the configuration file, such as test
inputs or helper programs (This is done on demand, using a `fetch` command
in the assignment configuration)
- Evaluate the submission accordingly to job configuration
- During evaluation progress states can be sent back to **Broker**
- Upload the results of the evaluation to the **Fileserver**
- Notify **Broker** that the evaluation finished
int *i;
int &j;
### Internal Worker architecture
Picture below is overall internal architecture of worker which shows its defined classes with private variables and public functions. Vector version of this picture is available [here](https://github.com/ReCodEx/GlobalWiki/raw/master/images/Worker_Internal_Architecture.pdf).
* Boolean expression: long boolean expression should be divided into more lines. The division point should always be after logical operators.
```
if (i > 10 &&
j <10&&
k > 20) {
std::cout << "Were here!" <<std::endl;
}
```
* Return values should not be generally wrapped with parentheses, only if needed.
* Preprocessor directives start with `#` and always should start at the beginning of the line.
* Classes: sections aka. public, protected, private should have same indentation as the class start itself. Opening parenthesis of class should be on the same line as class name.
```
class my_class {
public:
void class_function();
private:
int class_member_;
};
```
* Operators: around all binary operators there always should be spaces.
```
int x = 5;
x = x * 5 / 5;
x = x + 5 * (10 - 5);
```
## File Server
### Python
**File Server** stores data, that should be kept outside of **WebApp's**
database (both because storing files in a database is inefficient and because
the workers need to access the files in the simplest possible way). It should
meet following requirements:
- store files without duplicates
- keep consistent state with main database
- serve files to workers on demand
- allow versioning of tasks with revert back feature
Python code should correspond to [PEP 8](https://www.python.org/dev/peps/pep-0008/) style.
To meet these requirements, **Storage** and **Database** must be set as bellow.
### PHP
TODO:
### Storage
**Storage** is meant as disc space with some commonly used filesystem. We'll use `ext4`, but the other ones should work too. **Storage** file structure is:
```
.
├── submits
│ └── user_id
│ └── advanced_dot_net_1
│ └── submit_id
│ ├── eval.yml
│ └── source.cs
├── submit_archives
│ └── submit_id.tar.gz
├── tasks
│ ├── a
│ │ ├── a014ed2abb56371bfaf2b4298a85d5dfb56509ed
│ │ └── a5edbd8b12e670ed1e3110d6c0524000cd4c3c7a
│ └── b
│ └── b1696358b8540923eb79b68f95c0f94c13a83fa7
└── temp
└── 1795184136b8bdddabe50453cc2cc2d46f0f7c5e
```
- **submits** keep information about all files submited by users to ReCodEx.
There are subdirectories _user_id_ and _advanced_dot_net_1_ which groups
submits by users and courses the submits are for. This structure is easy to
maintain for new and deleted users.
- **submit_archives** contains the student submissions in compressed archives so
that they can be easily downloaded by workers.
- **tasks** contains supplementary files (such as test inputs or helper
programs) for all existing task in ReCodEx. To avoid too many files in one
directory, files are separated to subfolders by first character of their name.
- **temp** directory is dedicated to temporary storing outputs of programs on teachers' demand. This directory will be erased by cron job on daily basis.
### Database
For user friendly access and modifying tasks following information should be stored in database:
- list of tasks with their newest version number
- for every task and version list of used files (their hashed names)
- for every hash name one human readable filename
### Conclusion
Files are internally stored by their `sha1sum` hashes, so it's easy to implement
versioning and get rid of files with duplicate content (multiple files can have
the same content, which is only stored once). **Worker** also uses files by
their hashes, which is great for local caching without worries about actual
version number of given file. On the other hand, **Database** stores information
about human readable names, so that the files are presented in a friendly way to
Martin Polanka
8 years ago