|
|
@ -666,47 +666,18 @@ arbitrary number of tasks with other types.
|
|
|
|
|
|
|
|
|
|
|
|
### Evaluation progress state
|
|
|
|
### Evaluation progress state
|
|
|
|
|
|
|
|
|
|
|
|
Users surely want to know progress state of their submitted solution this kind
|
|
|
|
Users surely want to know a progress state of their submitted solution. The very
|
|
|
|
of functionality comes particularly handy in long duration exercises. Because of
|
|
|
|
first idea would be to report state based on done messages from compilation,
|
|
|
|
reporting progress users have immediate knowledge if anything goes wrong, not
|
|
|
|
execution and evaluation as a lot of evaluation systems are already providing.
|
|
|
|
mention psychological effect that whole system and its parts are working and
|
|
|
|
However the ReCodEx have more advanced execution pipeline where there can be
|
|
|
|
doing something. That is why this feature was considered from beginning but
|
|
|
|
more compilations or more executions per test and also other technical tasks
|
|
|
|
there are multiple ways how to look at it in particular.
|
|
|
|
controlling the job execution flow. The users do not know about these technical
|
|
|
|
|
|
|
|
details and data from this tasks may confuse them. A solution is to show users
|
|
|
|
The very first idea would be to provide progress state based on done messages
|
|
|
|
only percentual completion of the job as a plain progress bar without additional
|
|
|
|
from compilation, execution and evaluation. Which is something what a lot of
|
|
|
|
information about task types. This solution works well for all of the jobs and
|
|
|
|
evaluation systems are providing. These information are high level enough for
|
|
|
|
is very user friendly. To make the output more interesting, there is a database
|
|
|
|
users and they probably know what is going on and executing right now. If
|
|
|
|
of random kind-of-funny statements and a random new one is displayed every time
|
|
|
|
compilation fails users know that their solution is not compilable, if execution
|
|
|
|
a task is completed.
|
|
|
|
fails there were some problems with their program. The clarity of this kind of
|
|
|
|
|
|
|
|
progress state is nice and understandable. But as we learnt ReCodEx has to have
|
|
|
|
|
|
|
|
more advanced execution pipeline there can be more compilations or more
|
|
|
|
|
|
|
|
executions. And in addition parts of the system which ensure execution of users
|
|
|
|
|
|
|
|
solutions do not have to precisely know what they are executing at the moment.
|
|
|
|
|
|
|
|
This kind of information may be meaningless for them.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
That is why another solution of progress state was considered. As we know right
|
|
|
|
|
|
|
|
now one of the best ways how to ensure generality is to have jobs with
|
|
|
|
|
|
|
|
single-purpose tasks. These tasks can be anything, some internal operation or
|
|
|
|
|
|
|
|
execution of external and sandboxed program. Based on this there is one very
|
|
|
|
|
|
|
|
simple solution how to provide general progress state which should be
|
|
|
|
|
|
|
|
independent on task types. We know that job has some number of tasks which has
|
|
|
|
|
|
|
|
to be executed so we can send state info after execution of every task. And that
|
|
|
|
|
|
|
|
is how we get percentual completion of an execution. Yes, it is kind of boring
|
|
|
|
|
|
|
|
and standard way but on top of that there can be built something else and more
|
|
|
|
|
|
|
|
appealing to users.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
So displaying progress to users can be done numerous ways. We have percentual
|
|
|
|
|
|
|
|
completion which is of course begging for simple solution which is displaying
|
|
|
|
|
|
|
|
only the percentage or some kind of standard graphical progress bar. But that is
|
|
|
|
|
|
|
|
too mainstream lets try something else. Very good idea is to have some kind of
|
|
|
|
|
|
|
|
puzzled image or images which will be composed together according to progress.
|
|
|
|
|
|
|
|
Nice way but kind of challenging if we do not have designer around. Another
|
|
|
|
|
|
|
|
original solution is to have database of random kind-of-funny statements which
|
|
|
|
|
|
|
|
will be displayed every time task is completed. It is easy enough for
|
|
|
|
|
|
|
|
implementation and even for making up these messages and it is quite new and
|
|
|
|
|
|
|
|
original. That is why this last solution was chosen for displaying progress
|
|
|
|
|
|
|
|
state.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
### Results of evaluation
|
|
|
|
### Results of evaluation
|
|
|
|
|
|
|
|
|
|
|
|