master
Martin Polanka 8 years ago
parent 6c0aa45c07
commit 423924b3d8

@ -22,7 +22,7 @@ The broker's responsibilites are:
The broker uses our ZeroMQ _reactor_ to bind events on sockets to handler classes. The broker uses our ZeroMQ _reactor_ to bind events on sockets to handler classes.
There are currently two handlers - one that handles the main functionality and There are currently two handlers - one that handles the main functionality and
another one that sends status reports to the REST API asynchronously so that the another one that sends status reports to the REST API asynchronously so that the
broker doesn't have to wait for HTTP requests which can take a lot of time, broker does not have to wait for HTTP requests which can take a lot of time,
especially when some kind of error happens on the server. especially when some kind of error happens on the server.
### Worker registry ### Worker registry
@ -30,18 +30,18 @@ especially when some kind of error happens on the server.
The `worker_registry` class is used to store information about workers, their The `worker_registry` class is used to store information about workers, their
status and the jobs in their queue. It can look up a worker using the headers status and the jobs in their queue. It can look up a worker using the headers
received with a request. It also uses a basic load balancing algorithm - the received with a request. It also uses a basic load balancing algorithm - the
workers are contained in a queue and whenever one of them receives a job, it's workers are contained in a queue and whenever one of them receives a job, it is
moved to the back, which makes it less likely to receive another job soon. moved to the back, which makes it less likely to receive another job soon.
When a worker is assigned a job, it won't be assigned another one until we When a worker is assigned a job, it will not be assigned another one until we
receive a `done` message from it. receive a `done` message from it.
### Error handling ### Error handling
**Job failure** - we recognize two ways a job can fail - an internally and **Job failure** - we recognize two ways a job can fail - an internally and
externally. An internal failure is the worker's fault - for example when it externally. An internal failure is the worker's fault - for example when it
can't download a file needed for the evaluation for some reason. An external cannot download a file needed for the evaluation for some reason. An external
error is for example when the job configuration is malformed. Note that we don't error is for example when the job configuration is malformed. Note that we do not
consider a student entering an incorrect solution a job failure. consider a student entering an incorrect solution a job failure.
Jobs that failed internally are reassigned until a limit on the amount of Jobs that failed internally are reassigned until a limit on the amount of
@ -59,7 +59,7 @@ failed to the frontend via REST API.
will reconnect automatically. However, all jobs in their queues are lost. If a will reconnect automatically. However, all jobs in their queues are lost. If a
worker manages to finish a job and notifies the "new" broker, the report is worker manages to finish a job and notifies the "new" broker, the report is
forwarded to the frontend. The same goes for external failures. Jobs that fail forwarded to the frontend. The same goes for external failures. Jobs that fail
internally cannot be reassigned, because the "new" broker doesn't know their internally cannot be reassigned, because the "new" broker does not know their
headers - they are reported as failed immediately. headers - they are reported as failed immediately.
## Installation ## Installation
@ -79,7 +79,7 @@ $ git submodule update --init
``` ```
### Install broker ### Install broker
It's supposed that your current working directory is that one with clonned worker source codes. It is supposed that your current working directory is that one with clonned worker source codes.
- Prepare environment running `mkdir build && cd build` - Prepare environment running `mkdir build && cd build`
- Build sources by `cmake ..` following by `make -j#` where `#` symbol refers to number of your CPU threads. - Build sources by `cmake ..` following by `make -j#` where `#` symbol refers to number of your CPU threads.
@ -87,11 +87,11 @@ It's supposed that your current working directory is that one with clonned worke
Note that `rpm` and `deb` packages are build in the same time. You may need to have `rpmbuild` command (usually as `rpmbuild` or `rpm` package) or edit CPACK_GENERATOR variable _CMakeLists.txt_ file in root of source code tree. Note that `rpm` and `deb` packages are build in the same time. You may need to have `rpmbuild` command (usually as `rpmbuild` or `rpm` package) or edit CPACK_GENERATOR variable _CMakeLists.txt_ file in root of source code tree.
- Install generated package through your package manager (`yum`, `dnf`, `dpkg`). - Install generated package through your package manager (`yum`, `dnf`, `dpkg`).
_Note:_ If you don't want to generate binary packages, you can just install the project with `make install` (as root). But installation through your distribution's package manager is preferred way to keep your system clean and manageable in long term horizon. _Note:_ If you do not want to generate binary packages, you can just install the project with `make install` (as root). But installation through your distribution's package manager is preferred way to keep your system clean and manageable in long term horizon.
## Configuration and usage ## Configuration and usage
Following text describes how to set up and run broker program. It's supposed to have required binaries installed. Also, using systemd is recommended for best user experience, but it's not required. Almost all modern Linux distributions are using systemd now. Following text describes how to set up and run broker program. It is supposed to have required binaries installed. Also, using systemd is recommended for best user experience, but it is not required. Almost all modern Linux distributions are using systemd now.
Installation of broker program does following step to your computer: Installation of broker program does following step to your computer:

Loading…
Cancel
Save