From 6e6bfaba8d8befce4c6b725b1e8293b496f8f703 Mon Sep 17 00:00:00 2001 From: Martin Polanka Date: Sun, 25 Dec 2016 19:18:35 +0100 Subject: [PATCH] Backend implementation analysis todos --- Rewritten-docs.md | 54 +++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 54 insertions(+) diff --git a/Rewritten-docs.md b/Rewritten-docs.md index 01d39fd..e46827f 100644 --- a/Rewritten-docs.md +++ b/Rewritten-docs.md @@ -231,6 +231,60 @@ Czech). Most notable features are following: - evaluation procedure configured in YAML file, compound of small tasks connected into arbitrary oriented acyclic graph +### Implementation analysis + +@todo: GENERALLY... what problems were solved, how they can be solved and what was the final solution + +@todo: backend: what should be the structure, how many components and which one, mention scalability possibilities of current solution + +@todo: describe possibilities of "piece of work" which can backend execute, how they can look like, describe our job and its tasks + +@todo: why is there division to internal and external tasks and why it is needed + +@todo: in what order should tasks be executed, how to sort them + +@todo: how to solve problem with specific worker environment, mention internal job variables + +@todo: what type of communication within backend could be used, mention some frameworks, queue managers, protocols, which was considered + +@todo: fileserver and why is separated + +@todo: mention hashing on fileserver and why this approach was chosen + +@todo: what can be stored on fileserver + +@todo: how can jobs be stored on fileserver, mainly mention that it is nonsence to store inputs and outputs within job archive + +@todo: assigning of jobs to workers, which are possible algorithms, queues, which one was chosen + +@todo: how can jobs be sent over zeromq, mainly mention that files can be transported, but it is not feasible + +@todo: making action and reaction over zeromq more general and easily extensible, mention reactor and why is needed and what it solves + +@todo: worker and its internal structure, why there are two threads and what they can do, mention also multiplatform approach during development + +@todo: execution of job on worker, how it is done, what steps are necessary and general for all jobs + +@todo: how can inputs and outputs (and supplementary files) be handled (they can be downloaded on start of execution, or during...) + +@todo: caching of supplementary files (link to hashing above), describe cleaner and why it is a separate component + +@todo: describe a bit more cleaner functionality and that it is safe and there are no unrecoverable races + +@todo: sandboxing, what possibilites are out there (linux, Windows), what are general and really needed features, mention isolate, what are isolate features + +@todo: how can progress status can be sent, why is there separate component of system (monitor) and why is this feature only optional + +@todo: monitor and what is done there, mention caching and why it is needed + +@todo: communication between backend and frontend + +@todo: why is frontend divided into server and client part, mention possibilities of separated api + +@todo: what apis can be used on server frontend side, why rest in particular + +@todo: what technologies can be used on client frontend side, why react was used + Structure of the project ------------------------