From ee4a07836e12f325556c6cf16407d49282bbf700 Mon Sep 17 00:00:00 2001 From: Martin Polanka Date: Mon, 9 Jan 2017 19:00:55 +0100 Subject: [PATCH] analysis - outputs concept --- Rewritten-docs.md | 30 +++++++++++++++++++++++++++++- 1 file changed, 29 insertions(+), 1 deletion(-) diff --git a/Rewritten-docs.md b/Rewritten-docs.md index fa0fa21..1b57d50 100644 --- a/Rewritten-docs.md +++ b/Rewritten-docs.md @@ -663,7 +663,35 @@ state. ### Results of evaluation -@todo: how to display generally all outputs of executed programs to user (supervisor, student), what students can or cannot see and why +There are lot of things which deserves discussion concerning results of +evaluation, how they should be displayed, what should be visible or not and also +some what kind of reward for users solutions should be chosen. + +At first lets focus on all kinds of outputs from executed programs within job. +Out of discussion is that supervisors should be able to view almost all outputs +from solutions if they choose them to be visible and recorded. This feature is +critical in debugging either whole exercises or users solutions. But should it +be default behaviour to record every output? Absolutely not, supervisor should +have choice to turn it on but defaults has to be not record it. Even without +this functionality can be file base around whole ReCodEx system quite large and +on top of that outputs from executed programs can be sometimes very extensive. +Storing this amount of data is purely nonsense to every solution. But if +requested by supervisor then this feature should be available. + +Question what should regular users see from execution of their solution is more +interesting. Simple answer is of course that they should not see anything which +is partly true. Outputs from their programs can be anything and users can +somehow analyse inputs or even redirect them to output. So outputs from +execution should not be visible at all or under very special circumstances. But +what about compilation or other kinds of initiations. Well, this is another +story, it really depends on the particular case. But generally it is quite +harmless to display user some kind of compilation error which can help a lot +during troubleshooting. Of course again this kind of functionality should be +configurable by supervisors and disabled by default. There is also the last kind +of tasks which can output some information which is evaluation tasks. Output of +these tasks is somehow important to whole system and again can contain some +information about inputs or reference outputs. So outputs of evaluation tasks +should not also be visible to regular users. @todo: judges, discuss what they possibly can do and what it can be used for (returning for instance 2 numbers instead of 1 and why we return just one)