|
|
@ -663,7 +663,35 @@ state.
|
|
|
|
|
|
|
|
|
|
|
|
### Results of evaluation
|
|
|
|
### Results of evaluation
|
|
|
|
|
|
|
|
|
|
|
|
@todo: how to display generally all outputs of executed programs to user (supervisor, student), what students can or cannot see and why
|
|
|
|
There are lot of things which deserves discussion concerning results of
|
|
|
|
|
|
|
|
evaluation, how they should be displayed, what should be visible or not and also
|
|
|
|
|
|
|
|
some what kind of reward for users solutions should be chosen.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
At first lets focus on all kinds of outputs from executed programs within job.
|
|
|
|
|
|
|
|
Out of discussion is that supervisors should be able to view almost all outputs
|
|
|
|
|
|
|
|
from solutions if they choose them to be visible and recorded. This feature is
|
|
|
|
|
|
|
|
critical in debugging either whole exercises or users solutions. But should it
|
|
|
|
|
|
|
|
be default behaviour to record every output? Absolutely not, supervisor should
|
|
|
|
|
|
|
|
have choice to turn it on but defaults has to be not record it. Even without
|
|
|
|
|
|
|
|
this functionality can be file base around whole ReCodEx system quite large and
|
|
|
|
|
|
|
|
on top of that outputs from executed programs can be sometimes very extensive.
|
|
|
|
|
|
|
|
Storing this amount of data is purely nonsense to every solution. But if
|
|
|
|
|
|
|
|
requested by supervisor then this feature should be available.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Question what should regular users see from execution of their solution is more
|
|
|
|
|
|
|
|
interesting. Simple answer is of course that they should not see anything which
|
|
|
|
|
|
|
|
is partly true. Outputs from their programs can be anything and users can
|
|
|
|
|
|
|
|
somehow analyse inputs or even redirect them to output. So outputs from
|
|
|
|
|
|
|
|
execution should not be visible at all or under very special circumstances. But
|
|
|
|
|
|
|
|
what about compilation or other kinds of initiations. Well, this is another
|
|
|
|
|
|
|
|
story, it really depends on the particular case. But generally it is quite
|
|
|
|
|
|
|
|
harmless to display user some kind of compilation error which can help a lot
|
|
|
|
|
|
|
|
during troubleshooting. Of course again this kind of functionality should be
|
|
|
|
|
|
|
|
configurable by supervisors and disabled by default. There is also the last kind
|
|
|
|
|
|
|
|
of tasks which can output some information which is evaluation tasks. Output of
|
|
|
|
|
|
|
|
these tasks is somehow important to whole system and again can contain some
|
|
|
|
|
|
|
|
information about inputs or reference outputs. So outputs of evaluation tasks
|
|
|
|
|
|
|
|
should not also be visible to regular users.
|
|
|
|
|
|
|
|
|
|
|
|
@todo: discuss points assigned to solution, why are there bonus points, explain minimal point threshold
|
|
|
|
@todo: discuss points assigned to solution, why are there bonus points, explain minimal point threshold
|
|
|
|
|
|
|
|
|
|
|
|