diff --git a/Rewritten-docs.md b/Rewritten-docs.md index ac952bc..7a8611f 100644 --- a/Rewritten-docs.md +++ b/Rewritten-docs.md @@ -2671,16 +2671,28 @@ The responsibilites of broker are: either via monitor or REST API - notifying the frontend of errors in the backend -### Architecture +### Internal Structure The broker uses our ZeroMQ _reactor_ to bind events on sockets to handler -classes. There are currently two handlers -- one that handles the main +classes. There are currently two handlers -- one that handles the main functionality and another one that sends status reports to the REST API asynchronously so that the broker does not have to wait for HTTP requests which can take a lot of time, especially when some kind of error happens on the server. -#### Worker registry +Main handler takes care of requests from workers and API servers: + +- *init* -- initial connection from worker to broker +- *done* -- currently processed job on worker was executed and is done +- *ping* -- worker prooving that it is still alive +- *progress* -- job progress state from worker which is immediatelly forwarded + to monitor +- *eval* -- request from API server to execute given job + +Second handler is asynchronous status notifier which is able to execute HTTP +requests. This notifier is used on error reporting from backend to frontend API. + +#### Worker Registry The `worker_registry` class is used to store information about workers, their status and the jobs in their queue. It can look up a worker using the headers @@ -2699,7 +2711,14 @@ end, which makes it less likely to receive another job soon. When a worker is assigned a job, it will not be assigned another one until we receive a `done` message from it. -#### Error handling +#### Error Reporting + +Broker is the only backend component which is able to report errors to frontend +API. For this purpose HTTP protocol is used through *libcurl* library. To +address security concerns there is *HTTP Basic Auth* configured on particular +endpoints which is simply enough to use within *libcurl*. + +Following types of failures are distinguished: **Job failure** -- we recognize two ways a job can fail -- an internally and externally. An internal failure is the fault of worker -- for example when it @@ -2725,10 +2744,6 @@ External failures are reported to the frontend immediately. internally cannot be reassigned, because the "new" broker does not know their headers -- they are reported as failed immediately. -@todo: what to mention: - - API notification using curl, authentication using HTTP Basic Auth - - asynchronous resending progress messages - ### Additional Libraries Broker implementation depends on several open-source C and C++ libraries.