diff --git a/Rewritten-docs.md b/Rewritten-docs.md index 63fb8c7..e1791c0 100644 --- a/Rewritten-docs.md +++ b/Rewritten-docs.md @@ -482,6 +482,7 @@ The communication protocol between these two logical parts will be described as well. @todo: move "General backend implementation" here + @todo: move "General frontend implementation" here ### Evaluation unit executed on backend @@ -631,13 +632,61 @@ Previous description implies that there is gap between detection of last access #### Sandboxing -@todo: sandboxing, what possibilites are out there (linux, Windows), what are general and really needed features, mention isolate, what are isolate features +@todo: sandboxing, what possibilites are out there (Linux, Windows), what are general and really needed features, mention isolate, what are isolate features ### Monitor -@todo: how progress status can be sent, why is there separate component of system (monitor) and why is this feature only optional +Users want to view real time evaluation progress of their solution. It can be +easily done with established double-sided connection stream, but it is hard to +achive with web technologies. HTTP protocol works differently on separate +requests basis with no longterm connection. However, there is widely used +technology to solve this problem, WebSocket protocol. + +Working with WebSocket protocol from the backend is possible, but not ideal from +design point of view. Backend should be hidden from public internet to minimize +surface for possible attacks. With this in mind, there are two possible options: + +- send progress messages through API +- make separate component for progress messages + +Each of the two possibilities has some pros and cons. The first one is good +beacuse there is no additional component and API is already publicly visible. On +the other side, working with WebSocket protocol from PHP is not much pleasant +(but it is possible) and embedding this functionality into API is not +extendable. The second approach is better for future changing the protocol or +implementing extensions like caching of messages. Also, the progress feature is +considered only optional, because there may be clients for which this feature is +useless. Major drawback of separate component is another part, which needs to +be publicly exposed. + +We decided to make a separate component, mainly because it is smaller component +with only one role, better maintainability and optional demands for progress +callback. + +There are several possibilities how to write the component. Notably, considered +options were already used languages C++, PHP, JavaScript and Python. At the end, +the Python language was chosen for its simplicity, great support for all used +technologies and also there are free Python developers in out team. Then, +responsibility of this component is determined. Concept of message flow is on +following picture. + +![Message flow inside montior](https://raw.githubusercontent.com/ReCodEx/wiki/master/images/Monitor_arch.png) + +The message channel inputing the monitor uses ZeroMQ as main message framework +used by backend. This decission keeps rest of backend avare of used +communication protocol and related libraries. Output channel is WebSocket as a +protocol for sending messages to web browsers. In Python, there are several +WebSocket libraries. The most popular one is `websockets` in cooperation with +`asyncio`. This combination is easy to use and well documented, so it is used in +monitor component too. For ZeroMQ, there is `zmq` library with binding to +framework core in C++. + +Incomming messages are cached for short period of time. Early testing shows, +that backend can start sending progress messages sooner than client connects to +the monitor. To solve this, messages for each job are hold 5 minutes after +reception of last message. The client gets all already received messages at time +of connection with no message loss. -@todo: monitor and what is done there, mention caching and why it is needed ### General frontend implementation