master
Petr Stefan 8 years ago
parent ad29c09bb6
commit 1a1859d9a4

@ -487,14 +487,51 @@ described as well.
### Evaluation unit executed on backend ### Evaluation unit executed on backend
@todo: describe possibilities of "piece of work" which can backend execute, how they can look like, describe our job and its tasks One of the bigger requests for the new system is to support complex
configuration of execution pipeline. The idea comes from lecturers of Compiler
@todo: why is there division to internal and external tasks and why it is needed principles class who want to migrate their semi-manual evaluation process to
CodEx. Unfortunately, CodEx is not capable of such compilicated exercise setup.
None of evaluation systems we found is capable of such task, so design from
scratch is needed.
There are two main approaches to design a complex execution configuration. It
can be composed of small amount of relatively big components or much more small
tasks. Big components are easy to write and whole configuration is reasonably
small. The components are designed for current problems, so it is not scalable
enough for pleasant future usage. This can be solved by introducing small set of
single-purposed tasks which can be composed together. The whole configuration is
then quite bigger, but with great adaptation ability for new conditions and also
less amount of work programming them. For better user experience, configuration
generators for some common cases can be introduced.
ReCodEx target is to be continuously developed and used for many years, so the
smaller tasks are the right choice. Observation of CodEx system shows that
only a few tasks are needed. In extreme case, only one task is enough -- execute
a binary. However, for better portability of configurations along different
systems it is better to implement reasonable subset of operations directly
without calling system provided binaries. These operations are copy file, create
new directory, extract archive and so on, altogether called internal tasks.
Another benefit from custom implementation of these tasks is guarantied safety,
so no sandbox needs to be used as in exernal tasks case.
For a job evaluation, the tasks needs to be executed sequentialy in a specified
order. The idea of running independent tasks in parallel is bad because exact
time measurement needs controled environment on target computer with
minimalization of interrupts by other processes. It seems that connecting tasks
into directed acyclic graph (DAG) can handle all possible problem cases. None of
the authors, supervisors and involved faculty staff can think of a problem that
cannot be decomposed into tasks connected in a DAG. The goal of evaluation is
to satisfy as many tasks as possible. During execution there are sometimes
multiple choices of next task. To control that, each task can have a priority,
which is used as a secondary ordering criterium. For better understanding, here
is a small example.
![Task serialization](https://github.com/ReCodEx/wiki/raw/master/images/Assignment_overview.png)
@todo: why we need priority and exact order of tasks?
@todo: division to initiation, execution and evaluation why and what for @todo: division to initiation, execution and evaluation why and what for
@todo: in what order should tasks be executed, how to sort them
@todo: how to solve problem with specific worker environment, mention internal job variables @todo: how to solve problem with specific worker environment, mention internal job variables

Loading…
Cancel
Save