diff --git a/Rewritten-docs.md b/Rewritten-docs.md index e1791c0..ede64e9 100644 --- a/Rewritten-docs.md +++ b/Rewritten-docs.md @@ -487,14 +487,51 @@ described as well. ### Evaluation unit executed on backend -@todo: describe possibilities of "piece of work" which can backend execute, how they can look like, describe our job and its tasks - -@todo: why is there division to internal and external tasks and why it is needed +One of the bigger requests for the new system is to support complex +configuration of execution pipeline. The idea comes from lecturers of Compiler +principles class who want to migrate their semi-manual evaluation process to +CodEx. Unfortunately, CodEx is not capable of such compilicated exercise setup. +None of evaluation systems we found is capable of such task, so design from +scratch is needed. + +There are two main approaches to design a complex execution configuration. It +can be composed of small amount of relatively big components or much more small +tasks. Big components are easy to write and whole configuration is reasonably +small. The components are designed for current problems, so it is not scalable +enough for pleasant future usage. This can be solved by introducing small set of +single-purposed tasks which can be composed together. The whole configuration is +then quite bigger, but with great adaptation ability for new conditions and also +less amount of work programming them. For better user experience, configuration +generators for some common cases can be introduced. + +ReCodEx target is to be continuously developed and used for many years, so the +smaller tasks are the right choice. Observation of CodEx system shows that +only a few tasks are needed. In extreme case, only one task is enough -- execute +a binary. However, for better portability of configurations along different +systems it is better to implement reasonable subset of operations directly +without calling system provided binaries. These operations are copy file, create +new directory, extract archive and so on, altogether called internal tasks. +Another benefit from custom implementation of these tasks is guarantied safety, +so no sandbox needs to be used as in exernal tasks case. + +For a job evaluation, the tasks needs to be executed sequentialy in a specified +order. The idea of running independent tasks in parallel is bad because exact +time measurement needs controled environment on target computer with +minimalization of interrupts by other processes. It seems that connecting tasks +into directed acyclic graph (DAG) can handle all possible problem cases. None of +the authors, supervisors and involved faculty staff can think of a problem that +cannot be decomposed into tasks connected in a DAG. The goal of evaluation is +to satisfy as many tasks as possible. During execution there are sometimes +multiple choices of next task. To control that, each task can have a priority, +which is used as a secondary ordering criterium. For better understanding, here +is a small example. + +![Task serialization](https://github.com/ReCodEx/wiki/raw/master/images/Assignment_overview.png) + +@todo: why we need priority and exact order of tasks? @todo: division to initiation, execution and evaluation why and what for -@todo: in what order should tasks be executed, how to sort them - @todo: how to solve problem with specific worker environment, mention internal job variables