From 998c1518754f7ff160da854d7c03b8aafe6c404e Mon Sep 17 00:00:00 2001 From: Petr Stefan Date: Wed, 18 Jan 2017 20:37:28 +0100 Subject: [PATCH] Progress state shortened --- Rewritten-docs.md | 53 +++++++++++------------------------------------ 1 file changed, 12 insertions(+), 41 deletions(-) diff --git a/Rewritten-docs.md b/Rewritten-docs.md index ef05ace..1d9802e 100644 --- a/Rewritten-docs.md +++ b/Rewritten-docs.md @@ -666,47 +666,18 @@ arbitrary number of tasks with other types. ### Evaluation progress state -Users surely want to know progress state of their submitted solution this kind -of functionality comes particularly handy in long duration exercises. Because of -reporting progress users have immediate knowledge if anything goes wrong, not -mention psychological effect that whole system and its parts are working and -doing something. That is why this feature was considered from beginning but -there are multiple ways how to look at it in particular. - -The very first idea would be to provide progress state based on done messages -from compilation, execution and evaluation. Which is something what a lot of -evaluation systems are providing. These information are high level enough for -users and they probably know what is going on and executing right now. If -compilation fails users know that their solution is not compilable, if execution -fails there were some problems with their program. The clarity of this kind of -progress state is nice and understandable. But as we learnt ReCodEx has to have -more advanced execution pipeline there can be more compilations or more -executions. And in addition parts of the system which ensure execution of users -solutions do not have to precisely know what they are executing at the moment. -This kind of information may be meaningless for them. - -That is why another solution of progress state was considered. As we know right -now one of the best ways how to ensure generality is to have jobs with -single-purpose tasks. These tasks can be anything, some internal operation or -execution of external and sandboxed program. Based on this there is one very -simple solution how to provide general progress state which should be -independent on task types. We know that job has some number of tasks which has -to be executed so we can send state info after execution of every task. And that -is how we get percentual completion of an execution. Yes, it is kind of boring -and standard way but on top of that there can be built something else and more -appealing to users. - -So displaying progress to users can be done numerous ways. We have percentual -completion which is of course begging for simple solution which is displaying -only the percentage or some kind of standard graphical progress bar. But that is -too mainstream lets try something else. Very good idea is to have some kind of -puzzled image or images which will be composed together according to progress. -Nice way but kind of challenging if we do not have designer around. Another -original solution is to have database of random kind-of-funny statements which -will be displayed every time task is completed. It is easy enough for -implementation and even for making up these messages and it is quite new and -original. That is why this last solution was chosen for displaying progress -state. +Users surely want to know a progress state of their submitted solution. The very +first idea would be to report state based on done messages from compilation, +execution and evaluation as a lot of evaluation systems are already providing. +However the ReCodEx have more advanced execution pipeline where there can be +more compilations or more executions per test and also other technical tasks +controlling the job execution flow. The users do not know about these technical +details and data from this tasks may confuse them. A solution is to show users +only percentual completion of the job as a plain progress bar without additional +information about task types. This solution works well for all of the jobs and +is very user friendly. To make the output more interesting, there is a database +of random kind-of-funny statements and a random new one is displayed every time +a task is completed. ### Results of evaluation