job configuration file - the beginning

master
Martin Polanka 8 years ago
parent 94e47f1704
commit f3e5804e76

@ -972,9 +972,45 @@ HTTP(S).
![Communication schema](https://github.com/ReCodEx/wiki/raw/master/images/Backend_Connections.png)
### Job Configuration
@todo: link to execution unit executed by recodex and lets give this chapter a bit of petite description
### Job Configuration File
As discussed previously in 'Evaluation Unit Executed by ReCodEx' evaluation unit
will have form of job which will contain small tasks representing one piece of
work executed by worker. This implies jobs have to be somehow given from
frontend to backend. The best option for this is to use some kind of
configuration file which will represent particular jobs. Mentioned configuration
file should be specified in frontend and in backend, namely worker, will be
parsed and executed.
There are many formats which can be used for configuration representation. The
ones which make sense are:
- *XML* -- is broadly used general markup language which is flavoured with DTD
definition which can express and check XML file structure, so it does not have
to be checked within application. But XML with its tags can be sometimes quite
'chatty' and extensive which does not have to be desirable. And overally XML
with all its features and properties can be a bit heavy-weight.
- *JSON* -- is notation which was developed to represent javascript objects. As
such it is quite simple, there can be expressed only: key-value structures,
arrays and primitive values. Structure and hierarchy of data is solved by
braces and brackets.
- *INI* -- is very simple configuration format which is able to represents only
key-value structures which can be grouped into sections. Which is not enough
to represent job and its tasks hierarchy.
- *YAML* -- format which is very similar to JSON with its capabilities. But with
small difference in structure and hirarchy of configuration which is solved
not with braces but with indentation. This means that YAML is easily readable
by both human and machine.
- *specific format* -- newly created format used just for job configuration.
Obvious drawback is non-existing parsers which would have to be written from
scratch.
Given previous list of different formats we decided to use YAML. There are
existing parsers for most of the programming languages and it is easy enough to
learn and understand. Another choice which make sense is JSON but at the end
YAML seemed to be better.
#### Configuration File Content
@todo: discuss what should be in configuration: limits, dependencies, priorities... whatever
@ -1021,9 +1057,6 @@ This format was used because of special dollar sign character which cannot be
used within paths of regular filesystems. Braces are there only to border
textual description of variable.
Variables as such will solve problem with specific worker environment and most
notably specific hierarchy of directories.
### Broker
The broker is responsible for keeping track of available workers and

Loading…
Cancel
Save