|
|
@ -672,16 +672,24 @@ arbitrary number of tasks with other types.
|
|
|
|
|
|
|
|
|
|
|
|
### Evaluation progress state
|
|
|
|
### Evaluation progress state
|
|
|
|
|
|
|
|
|
|
|
|
Users surely want to know a progress state of their submitted solution. The very
|
|
|
|
Users want to know the state of their submitted solution (whether it's waiting
|
|
|
|
first idea would be to report a state based on done messages from compilation,
|
|
|
|
in a queue, compiling, etc.). The very first idea would be to report a state
|
|
|
|
execution and evaluation as a lot of evaluation systems are already providing.
|
|
|
|
based on "done" messages from compilation, execution and evaluation like many
|
|
|
|
However the ReCodEx have more advanced execution pipeline where there can be
|
|
|
|
evaluation systems are already providing. However ReCodEx has a more complicated
|
|
|
|
more compilations or more executions per test and also other technical tasks
|
|
|
|
execution pipeline where there can be more compilation or execution tasks per
|
|
|
|
controlling the job execution flow. The users do not know about these technical
|
|
|
|
test and also other internal tasks that control the job execution flow.
|
|
|
|
details and data from this tasks may confuse them. A solution is to show users
|
|
|
|
|
|
|
|
only percentual completion of the job without additional information about task
|
|
|
|
The users do not know the technical details of the evaluation and data about
|
|
|
|
|
|
|
|
completion of tasks may confuse them. A solution is to show users only
|
|
|
|
|
|
|
|
percentual completion of the job without any additional information about task
|
|
|
|
types. This solution works well for all of the jobs and is very user friendly.
|
|
|
|
types. This solution works well for all of the jobs and is very user friendly.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
It is possible to expand upon this by adding a special "send progress message"
|
|
|
|
|
|
|
|
task to the job configuration that would mark the completion of a specific part
|
|
|
|
|
|
|
|
of the evaluation. However, the benefits of this feature are not worth the
|
|
|
|
|
|
|
|
effort of implementing it and unnecessarily complicating the job configuration
|
|
|
|
|
|
|
|
files.
|
|
|
|
|
|
|
|
|
|
|
|
### Results of evaluation
|
|
|
|
### Results of evaluation
|
|
|
|
|
|
|
|
|
|
|
|
The evaluation data have to be processed and then presented in human readable
|
|
|
|
The evaluation data have to be processed and then presented in human readable
|
|
|
|