Monitor analysis

master
Petr Stefan 8 years ago
parent 20e5faca1f
commit ad29c09bb6

@ -482,6 +482,7 @@ The communication protocol between these two logical parts will be
described as well. described as well.
@todo: move "General backend implementation" here @todo: move "General backend implementation" here
@todo: move "General frontend implementation" here @todo: move "General frontend implementation" here
### Evaluation unit executed on backend ### Evaluation unit executed on backend
@ -631,13 +632,61 @@ Previous description implies that there is gap between detection of last access
#### Sandboxing #### Sandboxing
@todo: sandboxing, what possibilites are out there (linux, Windows), what are general and really needed features, mention isolate, what are isolate features @todo: sandboxing, what possibilites are out there (Linux, Windows), what are general and really needed features, mention isolate, what are isolate features
### Monitor ### Monitor
@todo: how progress status can be sent, why is there separate component of system (monitor) and why is this feature only optional Users want to view real time evaluation progress of their solution. It can be
easily done with established double-sided connection stream, but it is hard to
achive with web technologies. HTTP protocol works differently on separate
requests basis with no longterm connection. However, there is widely used
technology to solve this problem, WebSocket protocol.
Working with WebSocket protocol from the backend is possible, but not ideal from
design point of view. Backend should be hidden from public internet to minimize
surface for possible attacks. With this in mind, there are two possible options:
- send progress messages through API
- make separate component for progress messages
Each of the two possibilities has some pros and cons. The first one is good
beacuse there is no additional component and API is already publicly visible. On
the other side, working with WebSocket protocol from PHP is not much pleasant
(but it is possible) and embedding this functionality into API is not
extendable. The second approach is better for future changing the protocol or
implementing extensions like caching of messages. Also, the progress feature is
considered only optional, because there may be clients for which this feature is
useless. Major drawback of separate component is another part, which needs to
be publicly exposed.
We decided to make a separate component, mainly because it is smaller component
with only one role, better maintainability and optional demands for progress
callback.
There are several possibilities how to write the component. Notably, considered
options were already used languages C++, PHP, JavaScript and Python. At the end,
the Python language was chosen for its simplicity, great support for all used
technologies and also there are free Python developers in out team. Then,
responsibility of this component is determined. Concept of message flow is on
following picture.
![Message flow inside montior](https://raw.githubusercontent.com/ReCodEx/wiki/master/images/Monitor_arch.png)
The message channel inputing the monitor uses ZeroMQ as main message framework
used by backend. This decission keeps rest of backend avare of used
communication protocol and related libraries. Output channel is WebSocket as a
protocol for sending messages to web browsers. In Python, there are several
WebSocket libraries. The most popular one is `websockets` in cooperation with
`asyncio`. This combination is easy to use and well documented, so it is used in
monitor component too. For ZeroMQ, there is `zmq` library with binding to
framework core in C++.
Incomming messages are cached for short period of time. Early testing shows,
that backend can start sending progress messages sooner than client connects to
the monitor. To solve this, messages for each job are hold 5 minutes after
reception of last message. The client gets all already received messages at time
of connection with no message loss.
@todo: monitor and what is done there, mention caching and why it is needed
### General frontend implementation ### General frontend implementation

Loading…
Cancel
Save