|
|
@ -2770,23 +2770,40 @@ directory is configurable and can be the same for multiple worker instances.
|
|
|
|
|
|
|
|
|
|
|
|
### Judges
|
|
|
|
### Judges
|
|
|
|
|
|
|
|
|
|
|
|
For future extensibility is critical that judges have some shared interface of
|
|
|
|
ReCodEx provides a few initial judges programs. They are mostly adopted from
|
|
|
|
calling and return values.
|
|
|
|
CodEx system and installed with worker component. Judging programs have to meet
|
|
|
|
|
|
|
|
some requirements. Basic ones are inspired by standard `diff` application -- two
|
|
|
|
|
|
|
|
mandatory positional parameters which have to be the files for comparison and
|
|
|
|
|
|
|
|
exit code reflecting if the results is correct (0) of wrong (1).
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
This interface lacks support for returning additional data by the judges, for
|
|
|
|
|
|
|
|
example similarity of the two files calculated as Levenshtein's edit distance.
|
|
|
|
|
|
|
|
To allow passing these additional values an extended judge interface can be
|
|
|
|
|
|
|
|
implemented:
|
|
|
|
|
|
|
|
|
|
|
|
- Parameters: There are two mandatory positional parameters which has to be
|
|
|
|
- Parameters: There are two mandatory positional parameters which has to be
|
|
|
|
files for comparision
|
|
|
|
files for comparision
|
|
|
|
- Results:
|
|
|
|
- Results:
|
|
|
|
- _comparison OK_
|
|
|
|
- _comparison OK_
|
|
|
|
- exitcode: 0
|
|
|
|
- exitcode: 0
|
|
|
|
- stdout: there is one line with double value which should be set to 1.0
|
|
|
|
- stdout: there is one line with a double value which should be set to
|
|
|
|
|
|
|
|
1.0
|
|
|
|
- _comparison BAD_
|
|
|
|
- _comparison BAD_
|
|
|
|
- exitcode: 1
|
|
|
|
- exitcode: 1
|
|
|
|
- stdout: there is one line with double value which should be percentage
|
|
|
|
- stdout: there is one line with a double value which should be
|
|
|
|
of correctness of quality of two given files
|
|
|
|
quality percentage of the two given files
|
|
|
|
- _error during execution_
|
|
|
|
- _error during execution_
|
|
|
|
- exitcode: 2
|
|
|
|
- exitcode: 2
|
|
|
|
- stderr: there should be description of error
|
|
|
|
- stderr: there should be description of error
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
The additional double value is saved to the results file and can be used for
|
|
|
|
|
|
|
|
score calculation in the frontend. If just the basic judge is used, the values
|
|
|
|
|
|
|
|
are 1.0 for exit code 0 and 0.0 for exit code 1.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
If more values are needed for score computation, multiple judges can be used in
|
|
|
|
|
|
|
|
sequence and the values used together. However, extended judge interface should
|
|
|
|
|
|
|
|
comply most of possible use cases.
|
|
|
|
|
|
|
|
|
|
|
|
### Additional libraries
|
|
|
|
### Additional libraries
|
|
|
|
|
|
|
|
|
|
|
|
@todo: libcurl, spdlog, boost, yaml-cpp, libarchive, cppzmq
|
|
|
|
@todo: libcurl, spdlog, boost, yaml-cpp, libarchive, cppzmq
|
|
|
|