@ -609,13 +609,47 @@ arbitrary number of tasks with other types.
### Evaluation progress state
Users surely want to know progress state of their submitted solution this kind of functionality comes particularly handy in long duration exercises. Because of reporting progress users have immediate knowledge if anything goes wrong, not mention psychological effect that whole system and its parts are working and doing something. That is why this feature was considered from beginning but there are multiple ways how to look at it in particular.
The very first idea would be to provide progress state based on done messages from compilation, execution and evaluation. Which is something what a lot of evaluation systems are providing. These information are high level enough for users and they probably know what is going on and executing right now. If compilation fails users know that their solution is not compilable, if execution fails there were some problems with their program. The clarity of this kind of progress state is nice and understandable. But as we learnt ReCodEx has to have more advanced execution pipeline there can be more compilations or more executions. And in addition parts of the system which ensure execution of users solutions do not have to precisely know what they are executing at the moment. This kind of information may be meaningless for them.
That is why another solution of progress state was considered. As we know right now one of the best ways how to ensure generality is to have jobs with single-purpose tasks. These tasks can be anything, some internal operation or execution of external and sandboxed program. Based on this there is one very simple solution how to provide general progress state which should be independent on task types. We know that job has some number of tasks which has to be executed so we can send state info after execution of every task. And that is how we get percentual completion of an execution. Yes, it is kind of boring and standard way but on top of that there can be built something else and more appealing to users.
So displaying progress to users can be done numerous ways. We have percentual completion which is of course begging for simple solution which is displaying only the percentage or some kind of standard graphical progress bar. But that is too mainstream lets try something else. Very good idea is to have some kind of puzzled image or images which will be composed together according to progress. Nice way but kind of challenging if we do not have designer around. Another original solution is to have database of random kind-of-funny statements which will be displayed every time task is completed. It is easy enough for implementation and even for making up these messages and it is quite new and original. That is why this last solution was chosen for displaying progress state.
Users surely want to know progress state of their submitted solution this kind
of functionality comes particularly handy in long duration exercises. Because of
reporting progress users have immediate knowledge if anything goes wrong, not
mention psychological effect that whole system and its parts are working and
doing something. That is why this feature was considered from beginning but
there are multiple ways how to look at it in particular.
The very first idea would be to provide progress state based on done messages
from compilation, execution and evaluation. Which is something what a lot of
evaluation systems are providing. These information are high level enough for
users and they probably know what is going on and executing right now. If
compilation fails users know that their solution is not compilable, if execution
fails there were some problems with their program. The clarity of this kind of
progress state is nice and understandable. But as we learnt ReCodEx has to have
more advanced execution pipeline there can be more compilations or more
executions. And in addition parts of the system which ensure execution of users
solutions do not have to precisely know what they are executing at the moment.
This kind of information may be meaningless for them.
That is why another solution of progress state was considered. As we know right
now one of the best ways how to ensure generality is to have jobs with
single-purpose tasks. These tasks can be anything, some internal operation or
execution of external and sandboxed program. Based on this there is one very
simple solution how to provide general progress state which should be
independent on task types. We know that job has some number of tasks which has
to be executed so we can send state info after execution of every task. And that
is how we get percentual completion of an execution. Yes, it is kind of boring
and standard way but on top of that there can be built something else and more
appealing to users.
So displaying progress to users can be done numerous ways. We have percentual
completion which is of course begging for simple solution which is displaying
only the percentage or some kind of standard graphical progress bar. But that is
too mainstream lets try something else. Very good idea is to have some kind of
puzzled image or images which will be composed together according to progress.
Nice way but kind of challenging if we do not have designer around. Another
original solution is to have database of random kind-of-funny statements which
will be displayed every time task is completed. It is easy enough for
implementation and even for making up these messages and it is quite new and
original. That is why this last solution was chosen for displaying progress
state.
### Results of evaluation
@ -631,6 +665,7 @@ So displaying progress to users can be done numerous ways. We have percentual co
@todo: where is kept the state (MariaDB)
@todo: and many many more general concepts which can be discussed and solved... please append more of them if something comes to your mind... thanks
Martin Polanka
8 years ago