diff --git a/Rewritten-docs.md b/Rewritten-docs.md index 4c4d1af..0778df7 100644 --- a/Rewritten-docs.md +++ b/Rewritten-docs.md @@ -663,7 +663,35 @@ state. ### Results of evaluation -@todo: how to display generally all outputs of executed programs to user (supervisor, student), what students can or cannot see and why +There are lot of things which deserves discussion concerning results of +evaluation, how they should be displayed, what should be visible or not and also +some what kind of reward for users solutions should be chosen. + +At first lets focus on all kinds of outputs from executed programs within job. +Out of discussion is that supervisors should be able to view almost all outputs +from solutions if they choose them to be visible and recorded. This feature is +critical in debugging either whole exercises or users solutions. But should it +be default behaviour to record every output? Absolutely not, supervisor should +have choice to turn it on but defaults has to be not record it. Even without +this functionality can be file base around whole ReCodEx system quite large and +on top of that outputs from executed programs can be sometimes very extensive. +Storing this amount of data is purely nonsense to every solution. But if +requested by supervisor then this feature should be available. + +Question what should regular users see from execution of their solution is more +interesting. Simple answer is of course that they should not see anything which +is partly true. Outputs from their programs can be anything and users can +somehow analyse inputs or even redirect them to output. So outputs from +execution should not be visible at all or under very special circumstances. But +what about compilation or other kinds of initiations. Well, this is another +story, it really depends on the particular case. But generally it is quite +harmless to display user some kind of compilation error which can help a lot +during troubleshooting. Of course again this kind of functionality should be +configurable by supervisors and disabled by default. There is also the last kind +of tasks which can output some information which is evaluation tasks. Output of +these tasks is somehow important to whole system and again can contain some +information about inputs or reference outputs. So outputs of evaluation tasks +should not also be visible to regular users. @todo: discuss points assigned to solution, why are there bonus points, explain minimal point threshold