From 492bbd2ab28c9a7ec1523d3210be6ec1921bb9be Mon Sep 17 00:00:00 2001 From: Petr Stefan Date: Tue, 24 Jan 2017 22:33:16 +0100 Subject: [PATCH] Judge implementation improvement --- Rewritten-docs.md | 27 ++++++++++++++++++++++----- 1 file changed, 22 insertions(+), 5 deletions(-) diff --git a/Rewritten-docs.md b/Rewritten-docs.md index cd5d775..ea8dd13 100644 --- a/Rewritten-docs.md +++ b/Rewritten-docs.md @@ -2770,23 +2770,40 @@ directory is configurable and can be the same for multiple worker instances. ### Judges -For future extensibility is critical that judges have some shared interface of -calling and return values. +ReCodEx provides a few initial judges programs. They are mostly adopted from +CodEx system and installed with worker component. Judging programs have to meet +some requirements. Basic ones are inspired by standard `diff` application -- two +mandatory positional parameters which have to be the files for comparison and +exit code reflecting if the results is correct (0) of wrong (1). + +This interface lacks support for returning additional data by the judges, for +example similarity of the two files calculated as Levenshtein's edit distance. +To allow passing these additional values an extended judge interface can be +implemented: - Parameters: There are two mandatory positional parameters which has to be files for comparision - Results: - _comparison OK_ - exitcode: 0 - - stdout: there is one line with double value which should be set to 1.0 + - stdout: there is one line with a double value which should be set to + 1.0 - _comparison BAD_ - exitcode: 1 - - stdout: there is one line with double value which should be percentage - of correctness of quality of two given files + - stdout: there is one line with a double value which should be + quality percentage of the two given files - _error during execution_ - exitcode: 2 - stderr: there should be description of error +The additional double value is saved to the results file and can be used for +score calculation in the frontend. If just the basic judge is used, the values +are 1.0 for exit code 0 and 0.0 for exit code 1. + +If more values are needed for score computation, multiple judges can be used in +sequence and the values used together. However, extended judge interface should +comply most of possible use cases. + ### Additional libraries @todo: libcurl, spdlog, boost, yaml-cpp, libarchive, cppzmq