|
|
@ -972,9 +972,45 @@ HTTP(S).
|
|
|
|
|
|
|
|
|
|
|
|
![Communication schema](https://github.com/ReCodEx/wiki/raw/master/images/Backend_Connections.png)
|
|
|
|
![Communication schema](https://github.com/ReCodEx/wiki/raw/master/images/Backend_Connections.png)
|
|
|
|
|
|
|
|
|
|
|
|
### Job Configuration
|
|
|
|
### Job Configuration File
|
|
|
|
|
|
|
|
|
|
|
|
@todo: link to execution unit executed by recodex and lets give this chapter a bit of petite description
|
|
|
|
As discussed previously in 'Evaluation Unit Executed by ReCodEx' evaluation unit
|
|
|
|
|
|
|
|
will have form of job which will contain small tasks representing one piece of
|
|
|
|
|
|
|
|
work executed by worker. This implies jobs have to be somehow given from
|
|
|
|
|
|
|
|
frontend to backend. The best option for this is to use some kind of
|
|
|
|
|
|
|
|
configuration file which will represent particular jobs. Mentioned configuration
|
|
|
|
|
|
|
|
file should be specified in frontend and in backend, namely worker, will be
|
|
|
|
|
|
|
|
parsed and executed.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
There are many formats which can be used for configuration representation. The
|
|
|
|
|
|
|
|
ones which make sense are:
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
- *XML* -- is broadly used general markup language which is flavoured with DTD
|
|
|
|
|
|
|
|
definition which can express and check XML file structure, so it does not have
|
|
|
|
|
|
|
|
to be checked within application. But XML with its tags can be sometimes quite
|
|
|
|
|
|
|
|
'chatty' and extensive which does not have to be desirable. And overally XML
|
|
|
|
|
|
|
|
with all its features and properties can be a bit heavy-weight.
|
|
|
|
|
|
|
|
- *JSON* -- is notation which was developed to represent javascript objects. As
|
|
|
|
|
|
|
|
such it is quite simple, there can be expressed only: key-value structures,
|
|
|
|
|
|
|
|
arrays and primitive values. Structure and hierarchy of data is solved by
|
|
|
|
|
|
|
|
braces and brackets.
|
|
|
|
|
|
|
|
- *INI* -- is very simple configuration format which is able to represents only
|
|
|
|
|
|
|
|
key-value structures which can be grouped into sections. Which is not enough
|
|
|
|
|
|
|
|
to represent job and its tasks hierarchy.
|
|
|
|
|
|
|
|
- *YAML* -- format which is very similar to JSON with its capabilities. But with
|
|
|
|
|
|
|
|
small difference in structure and hirarchy of configuration which is solved
|
|
|
|
|
|
|
|
not with braces but with indentation. This means that YAML is easily readable
|
|
|
|
|
|
|
|
by both human and machine.
|
|
|
|
|
|
|
|
- *specific format* -- newly created format used just for job configuration.
|
|
|
|
|
|
|
|
Obvious drawback is non-existing parsers which would have to be written from
|
|
|
|
|
|
|
|
scratch.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Given previous list of different formats we decided to use YAML. There are
|
|
|
|
|
|
|
|
existing parsers for most of the programming languages and it is easy enough to
|
|
|
|
|
|
|
|
learn and understand. Another choice which make sense is JSON but at the end
|
|
|
|
|
|
|
|
YAML seemed to be better.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
#### Configuration File Content
|
|
|
|
|
|
|
|
|
|
|
|
@todo: discuss what should be in configuration: limits, dependencies, priorities... whatever
|
|
|
|
@todo: discuss what should be in configuration: limits, dependencies, priorities... whatever
|
|
|
|
|
|
|
|
|
|
|
@ -1021,9 +1057,6 @@ This format was used because of special dollar sign character which cannot be
|
|
|
|
used within paths of regular filesystems. Braces are there only to border
|
|
|
|
used within paths of regular filesystems. Braces are there only to border
|
|
|
|
textual description of variable.
|
|
|
|
textual description of variable.
|
|
|
|
|
|
|
|
|
|
|
|
Variables as such will solve problem with specific worker environment and most
|
|
|
|
|
|
|
|
notably specific hierarchy of directories.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
### Broker
|
|
|
|
### Broker
|
|
|
|
|
|
|
|
|
|
|
|
The broker is responsible for keeping track of available workers and
|
|
|
|
The broker is responsible for keeping track of available workers and
|
|
|
|