LEdoian
/
recodex-wiki
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Typos

Browse Source
master
Petr Stefan 8 years ago
parent 2cc1147f6d
commit d12fde67dc
1 changed files with 172 additions and 158 deletions
Show Stats Download Patch File Download Diff File

330
Rewritten-docs.md
Unescape Escape View File

@ -443,7 +443,7 @@ duties related with lecturing of labs are already associated with supervisors,
so this role does not seem useful. In addition, no one requested more than three so this role does not seem useful. In addition, no one requested more than three
level privilege scheme. level privilege scheme.
School labs are lessons for some students lead by supervisors. All students ina School labs are lessons for some students lead by supervisors. All students in a
lab have the same homework and supervisors evaluate their solutions. This lab have the same homework and supervisors evaluate their solutions. This
organization has to be carried over into the new system. Virtual groups are a organization has to be carried over into the new system. Virtual groups are a
counterpart to real-life labs. This concept was already discussed in the counterpart to real-life labs. This concept was already discussed in the
@ -537,8 +537,8 @@ For a job evaluation, the tasks need to be executed sequentially in a specified
order. Running independent tasks is possible, but there are complications -- order. Running independent tasks is possible, but there are complications --
exact time measurement requires a controlled environment with as few exact time measurement requires a controlled environment with as few
interruptions as possible from other processes. It would be possible to run interruptions as possible from other processes. It would be possible to run
tasks that do not need exact time measuremet in parallel, but in this case a tasks that do not need exact time measurement in parallel, but in this case a
synchronization mechanism has to be developed to exclude paralellism for synchronization mechanism has to be developed to exclude parallelism for
measured tasks. Usually, there are about four times more unmeasured tasks than measured tasks. Usually, there are about four times more unmeasured tasks than
tasks with time measurement, but measured tasks tend to be much longer. With tasks with time measurement, but measured tasks tend to be much longer. With
[Amdahl's law](https://en.wikipedia.org/wiki/Amdahl's_law) in mind, the [Amdahl's law](https://en.wikipedia.org/wiki/Amdahl's_law) in mind, the
@ -546,7 +546,7 @@ parallelism does not seem to provide a notable benefit in overall execution
speed and brings trouble with synchronization. Moreover, most of the internal speed and brings trouble with synchronization. Moreover, most of the internal
tasks are also limited by IO speed (most notably copying and downloading files tasks are also limited by IO speed (most notably copying and downloading files
and reading archives). However, if there are performance issues, this approach and reading archives). However, if there are performance issues, this approach
could be reconsiderred, along with using a ram disk for storing supplementary could be reconsidered, along with using a ram disk for storing supplementary
files. files.
It seems that connecting tasks into directed acyclic graph (DAG) can handle all It seems that connecting tasks into directed acyclic graph (DAG) can handle all
@ -662,7 +662,7 @@ To avoid assigning points for insufficient solutions (like only printing "File
error" which is the valid answer in two tests), a minimal point threshold can be error" which is the valid answer in two tests), a minimal point threshold can be
specified. If the solution is to get less points than specified, it will get specified. If the solution is to get less points than specified, it will get
zero points instead. This functionality can be embedded into grading computation zero points instead. This functionality can be embedded into grading computation
algoritm itself, but it would have to be present in each implementation algorithm itself, but it would have to be present in each implementation
separately, which is not maintainable. Because of this the threshold feature is separately, which is not maintainable. Because of this the threshold feature is
separated from score computation. separated from score computation.
@ -869,7 +869,7 @@ for this approach than RPC. Moreover, we rarely need to receive replies to our
requests immediately. requests immediately.
RabbitMQ seems well suited for many use cases, but implementing a job routing RabbitMQ seems well suited for many use cases, but implementing a job routing
mechanism between heterogenous workers would be complicated -- we would probably mechanism between heterogeneous workers would be complicated -- we would probably
have to create a separate load balancing service, which cancels the advantage of have to create a separate load balancing service, which cancels the advantage of
a message broker already being provided by the framework. It is also written in a message broker already being provided by the framework. It is also written in
Erlang, which nobody from our team understands. Erlang, which nobody from our team understands.
@ -988,8 +988,8 @@ considered ones are:
- *XML* -- broadly used general markup language which is flavoured with document - *XML* -- broadly used general markup language which is flavoured with document
type definition (DTD) which can express and check XML file structure, so it type definition (DTD) which can express and check XML file structure, so it
does not have to be checked within application. But XML with its tags can be does not have to be checked within application. But XML with its tags can be
sometimes quite 'chatty' and extensive which is not desirable. And overally sometimes quite 'chatty' and extensive which is not desirable. And overly XML
XML with all its features and properties can be a bit heavy-weight. with all its features and properties can be a bit heavy-weight.
- *JSON* -- a notation which was developed to represent javascript objects. As - *JSON* -- a notation which was developed to represent javascript objects. As
such it is quite simple, there can be expressed only: key-value structures, such it is quite simple, there can be expressed only: key-value structures,
arrays and primitive values. Structure and hierarchy of data is solved by arrays and primitive values. Structure and hierarchy of data is solved by
@ -998,7 +998,7 @@ considered ones are:
key-value structures which can be grouped into sections. This is not enough key-value structures which can be grouped into sections. This is not enough
to represent a job and its tasks hierarchy. to represent a job and its tasks hierarchy.
- *YAML* -- format which is very similar to JSON with its capabilities. But with - *YAML* -- format which is very similar to JSON with its capabilities. But with
small difference in structure and hirarchy of configuration which is solved small difference in structure and hierarchy of configuration which is solved
not with braces but with indentation. This means that YAML is easily readable not with braces but with indentation. This means that YAML is easily readable
by both human and machine. by both human and machine.
- *specific format* -- newly created format used just for job configuration. - *specific format* -- newly created format used just for job configuration.
@ -1230,7 +1230,7 @@ one. Implementation of first one is quite straightforward and clear. So let us
discuss what should be happening in execution subsystem. discuss what should be happening in execution subsystem.
After successful arrival of the job from broker to the listening thread, the job After successful arrival of the job from broker to the listening thread, the job
is immediatelly redirected to execution thread. In there worker has to prepare is immediately redirected to execution thread. In there worker has to prepare
new execution environment, solution archive has to be downloaded from fileserver new execution environment, solution archive has to be downloaded from fileserver
and extracted. Job configuration is located within these files and loaded into and extracted. Job configuration is located within these files and loaded into
internal structures and executed. After that, results are uploaded back to internal structures and executed. After that, results are uploaded back to
@ -1589,9 +1589,9 @@ To speed up the development process of the PHP server application we decided to
use a web framework. After evaluating and trying several frameworks, such as use a web framework. After evaluating and trying several frameworks, such as
Lumen, Laravel, and Symfony, we ended up using Nette. Lumen, Laravel, and Symfony, we ended up using Nette.
- **Lumen** and **Laravel** seemed promissing but the default ORM framework - **Lumen** and **Laravel** seemed promising but the default ORM framework
Eloquent is an implementation of ActiveRecord which we wanted to avoid. It Eloquent is an implementation of ActiveRecord which we wanted to avoid. It
was also suprisingly complicated to implement custom middleware for validation was also surprisingly complicated to implement custom middleware for validation
of access tokens in the headers of incoming HTTP requests. of access tokens in the headers of incoming HTTP requests.
- **Symfony** is a very good framework and has Doctrine "built-in". The reason - **Symfony** is a very good framework and has Doctrine "built-in". The reason
why we did not use Symfony in the end was our lack of experience with this why we did not use Symfony in the end was our lack of experience with this
@ -1625,7 +1625,7 @@ both the URL and the HTTP method of the request.
#### Authentication #### Authentication
To make certain data and actions acessible only for some specific users, there To make certain data and actions accessible only for some specific users, there
must be a way how these users can prove their identity. We decided to avoid PHP must be a way how these users can prove their identity. We decided to avoid PHP
sessions to make the server stateless (session ID is stored in the cookies of sessions to make the server stateless (session ID is stored in the cookies of
the HTTP requests and responses). The server issues a specific token for the the HTTP requests and responses). The server issues a specific token for the
@ -1697,7 +1697,7 @@ several advantages:
- no installation or setup is required on the device of the user - no installation or setup is required on the device of the user
- works on all platforms including mobile devices - works on all platforms including mobile devices
- when a new version is released, all the clients will use this version without - when a new version is released, all the clients will use this version without
any need for manual instalation of the update any need for manual installation of the update
One of the downsides is the large number of different web browsers (including One of the downsides is the large number of different web browsers (including
the older versions of a specific browser) and their different interpretation the older versions of a specific browser) and their different interpretation
@ -1754,7 +1754,7 @@ worth considering:
- **Angular 2** - it is a new framework which was developed by Google. This - **Angular 2** - it is a new framework which was developed by Google. This
framework is very complex and provides the developer with many tools which framework is very complex and provides the developer with many tools which
make creating a website very straigtforward. The code can be written in pure make creating a website very straightforward. The code can be written in pure
JavaScript (ES5) or using the TypeScript language which is then transpiled JavaScript (ES5) or using the TypeScript language which is then transpiled
into JavaScript. Creating a web application in Angular 2 is based on creating into JavaScript. Creating a web application in Angular 2 is based on creating
and composing components. The previous version of Angular is not compatible and composing components. The previous version of Angular is not compatible
@ -1772,7 +1772,7 @@ worth considering:
We decided to use React and Redux over Angular 2 for several reasons: We decided to use React and Redux over Angular 2 for several reasons:
- There is a large community arround these libraries and there is a large number - There is a large community around these libraries and there is a large number
of tutorials, libraries, and other resources available online. of tutorials, libraries, and other resources available online.
- Many of the web frontend developers are familiar with React and Redux and - Many of the web frontend developers are familiar with React and Redux and
contributing to the project should be easy for them. contributing to the project should be easy for them.
@ -1939,7 +1939,7 @@ description of dashboard will be provided later on with according roles.
## Student ## Student
Student is a default role for every newly registered user. This role has quite Student is a default role for every newly registered user. This role has quite
limited capabilites in ReCodEx. Generally, a student can only submit solutions limited capabilities in ReCodEx. Generally, a student can only submit solutions
of exercises in some particular groups. These groups should correspond to of exercises in some particular groups. These groups should correspond to
courses he/she attends. courses he/she attends.
@ -2201,7 +2201,7 @@ updated. To remove assigned bonus points, submit just the zero number. The bonus
points are not additive, newer value overrides older values. points are not additive, newer value overrides older values.
It is useful to give a feedback about the solution back to the user. For this It is useful to give a feedback about the solution back to the user. For this
you can use the "Commens and notes" box. Make sure that the messages are not you can use the "Comments and notes" box. Make sure that the messages are not
private, so that the student can see them. More detailed description of this box private, so that the student can see them. More detailed description of this box
can be nicely used the "Comments and notes" box. Make sure that the messages are can be nicely used the "Comments and notes" box. Make sure that the messages are
not private, so the student can see them. More detailed description of this box not private, so the student can see them. More detailed description of this box
@ -2660,7 +2660,7 @@ communication. It was designed to maintain a heavy load of messages by making
only small actions in the main communication thread and asynchronous execution only small actions in the main communication thread and asynchronous execution
of other actions. of other actions.
The responsibilites of broker are: The responsibilities of broker are:
- allowing workers to register themselves and keep track of their capabilities - allowing workers to register themselves and keep track of their capabilities
- tracking status of each worker and handle cases when they crash - tracking status of each worker and handle cases when they crash
@ -2672,19 +2672,19 @@ The responsibilites of broker are:
### Internal Structure ### Internal Structure
The main work of the broker is to handle incomming messages. For that a The main work of the broker is to handle incoming messages. For that a _reactor_
_reactor_ subcomponent is written to bind events on sockets to handler classes. subcomponent is written to bind events on sockets to handler classes. There are
There are currently two handlers -- one that handles the main functionality and currently two handlers -- one that handles the main functionality and the other
the other that sends status reports to the REST API asynchronously. This that sends status reports to the REST API asynchronously. This prevents broker
prevents broker freezes when synchronously waiting for responses of HTTP freezes when synchronously waiting for responses of HTTP requests, especially
requests, especially when some kind of error happens on the server. when some kind of error happens on the server.
Main handler takes care of requests from workers and API servers: Main handler takes care of requests from workers and API servers:
- *init* -- initial connection from worker to broker - *init* -- initial connection from worker to broker
- *done* -- currently processed job on worker was executed and is done - *done* -- currently processed job on worker was executed and is done
- *ping* -- worker prooving that it is still alive - *ping* -- worker proving that it is still alive
- *progress* -- job progress state from worker which is immediatelly forwarded - *progress* -- job progress state from worker which is immediately forwarded
to monitor to monitor
- *eval* -- request from API server to execute given job - *eval* -- request from API server to execute given job
@ -2727,7 +2727,7 @@ Following types of failures are distinguished:
student solution is not considered as a job failure. student solution is not considered as a job failure.
Jobs that failed internally are reassigned until a limit on the amount of Jobs that failed internally are reassigned until a limit on the amount of
reassingments (configurable with the `max_request_failures` option) is reached. reassignments (configurable with the `max_request_failures` option) is reached.
External failures are reported to the frontend immediately. External failures are reported to the frontend immediately.
**Worker failure** -- when a worker crash is detected, an attempt to reassign **Worker failure** -- when a worker crash is detected, an attempt to reassign
@ -2757,7 +2757,7 @@ Broker implementation depends on several open-source C and C++ libraries.
- **spdlog** -- Spdlog is small, fast and modern logging library used for system - **spdlog** -- Spdlog is small, fast and modern logging library used for system
logging. It is highly customizable and configurable from the logging. It is highly customizable and configurable from the
configuration of the broker. configuration of the broker.
- **yaml-cpp** -- Yaml-cpp is used for parsing borker configuration text file in - **yaml-cpp** -- Yaml-cpp is used for parsing broker configuration text file in
YAML format. YAML format.
- **boost-filesystem** -- Boost filesystem is used for managing logging - **boost-filesystem** -- Boost filesystem is used for managing logging
directory (create if necessary) and parsing filesystem paths from strings as directory (create if necessary) and parsing filesystem paths from strings as
@ -2773,7 +2773,7 @@ Broker implementation depends on several open-source C and C++ libraries.
## Fileserver ## Fileserver
The fileserver component provides a shared file storage between the frontend and The fileserver component provides a shared file storage between the frontend and
the backend. It is writtend in Python 3 using Flask web framework. Fileserver the backend. It is written in Python 3 using Flask web framework. Fileserver
stores files in configurable filesystem directory, provides file deduplication stores files in configurable filesystem directory, provides file deduplication
and HTTP access. To keep the stored data safe, the fileserver should not be and HTTP access. To keep the stored data safe, the fileserver should not be
visible from public internet. Instead, it should be accessed indirectly through visible from public internet. Instead, it should be accessed indirectly through
@ -2832,7 +2832,7 @@ evaluation request, worker has to do following:
### Internal Structure ### Internal Structure
Worker is logicaly divided into two parts: Worker is logically divided into two parts:
- **Listener** -- communicates with broker through ZeroMQ. On startup, it - **Listener** -- communicates with broker through ZeroMQ. On startup, it
introduces itself to the broker. Then it receives new jobs, passes them to introduces itself to the broker. Then it receives new jobs, passes them to
@ -2879,7 +2879,7 @@ administrator.
Student submissions are executed in a sandbox environment to prevent them from Student submissions are executed in a sandbox environment to prevent them from
damaging the host system and also to restrict the amount of used resources. damaging the host system and also to restrict the amount of used resources.
Currently, only the Isolate sandbox support is implemented, but it is possible Currently, only the Isolate sandbox support is implemented, but it is possible
to add support for another sandox. to add support for another sandbox.
Every sandbox, regardless of the concrete implementation, has to be a command Every sandbox, regardless of the concrete implementation, has to be a command
line application taking parameters with arguments, standard input or file. line application taking parameters with arguments, standard input or file.
@ -2948,7 +2948,7 @@ To allow passing these additional values an extended judge interface can be
implemented: implemented:
- Parameters: There are two mandatory positional parameters which have to be - Parameters: There are two mandatory positional parameters which have to be
files for comparision files for comparison
- Results: - Results:
- _comparison OK_ - _comparison OK_
- exitcode: 0 - exitcode: 0
@ -2978,7 +2978,7 @@ them are multi-platform, so both Linux and Windows builds are possible.
- **libcurl** -- Libcurl is used for all HTTP communication, that is downloading - **libcurl** -- Libcurl is used for all HTTP communication, that is downloading
and uploading files. Due to lack of documentation of all C++ bindings the and uploading files. Due to lack of documentation of all C++ bindings the
plain C API is used. plain C API is used.
- **libarchive** -- Libarchive is ised for compressing and extracting archives. - **libarchive** -- Libarchive is used for compressing and extracting archives.
Actual supported formats depends on installed packages on target system, but Actual supported formats depends on installed packages on target system, but
at least ZIP and TAR.GZ should be available. at least ZIP and TAR.GZ should be available.
- **cppzmq** -- Cppzmq is a simple C++ wrapper for core ZeroMQ C API. It - **cppzmq** -- Cppzmq is a simple C++ wrapper for core ZeroMQ C API. It
@ -3023,7 +3023,7 @@ unencrypted connection and will not show the progress to the users.
Monitor runs in 2 threads. _Thread 1_ is the main thread, which initializes all Monitor runs in 2 threads. _Thread 1_ is the main thread, which initializes all
components (logger for example), starts the other thread and runs the ZeroMQ components (logger for example), starts the other thread and runs the ZeroMQ
part of the application. This thread receives and parses incomming messages from part of the application. This thread receives and parses incoming messages from
broker and forwards them to _thread 2_ sending logic. broker and forwards them to _thread 2_ sending logic.
_Thread 2_ is responsible for managing all of WebSocket connections _Thread 2_ is responsible for managing all of WebSocket connections
@ -3033,12 +3033,12 @@ all events from other threads (actually only `send_message` method invocation)
must be called within the event loop (via `asyncio.loop.call_soon_threadsafe` must be called within the event loop (via `asyncio.loop.call_soon_threadsafe`
function). Please note, that most of the Python interpreters use [Global function). Please note, that most of the Python interpreters use [Global
Interpreter Lock](https://wiki.python.org/moin/GlobalInterpreterLock), so there Interpreter Lock](https://wiki.python.org/moin/GlobalInterpreterLock), so there
is actualy no parallelism in the performance point of view, but proper is actually no parallelism in the performance point of view, but proper
synchronization is still required. synchronization is still required.
### Handling of Incomming Messages ### Handling of Incoming Messages
Incomming ZeroMQ progress message is received and parsed to JSON format (same as Incoming ZeroMQ progress message is received and parsed to JSON format (same as
our WebSocket communication format). JSON string is then passed to _thread 2_ our WebSocket communication format). JSON string is then passed to _thread 2_
for asynchronous sending. Each message has an identifier of channel where to for asynchronous sending. Each message has an identifier of channel where to
send it to. send it to.
@ -3141,7 +3141,7 @@ its own attribute (database column). Current supported options are `darkTheme`,
Every user has a role in the system. The basic ones are student, supervisor and Every user has a role in the system. The basic ones are student, supervisor and
administrator, but new roles can be created by adding `Role` entities. Roles can administrator, but new roles can be created by adding `Role` entities. Roles can
have permissions associated with them. These associations are represented by have permissions associated with them. These associations are represented by
`Permission` entitites. Each permission consists of a role, resource, action and `Permission` entities. Each permission consists of a role, resource, action and
an `isAllowed` flag. If the `isAllowed` flag is set to true, the permission is an `isAllowed` flag. If the `isAllowed` flag is set to true, the permission is
positive (lets the role access the resource), and if it is false, it denies positive (lets the role access the resource), and if it is false, it denies
access. The `Resource` entity contains just a string identifier of a resource access. The `Resource` entity contains just a string identifier of a resource
@ -3386,7 +3386,7 @@ does not have to worry about it.
### Job Configuration Parsing and Modifying ### Job Configuration Parsing and Modifying
Even in the API the job configuration file can be loaded in the corresponding Even in the API the job configuration file can be loaded in the corresponding
internal stuctures. This is necessary because there has to be possibility to internal structures. This is necessary because there has to be possibility to
modify particular job details, such as the job identification or the fileserver modify particular job details, such as the job identification or the fileserver
address, during the submission. address, during the submission.
@ -3403,7 +3403,7 @@ should have *get* counterparts. Job configuration is serialized through
`__toString()` methods. `__toString()` methods.
For loading of the job configuration there is separate `Storage` class which can For loading of the job configuration there is separate `Storage` class which can
be used for loading, saving or archivating of job configuration. For parsing the be used for loading, saving or archiving of job configuration. For parsing the
storage uses the `Loader` class which does all the checks and loads the data storage uses the `Loader` class which does all the checks and loads the data
from given strings in the appropriate structures. In case of parser error from given strings in the appropriate structures. In case of parser error
`App\Exceptions\JobConfigLoadingException` is thrown. `App\Exceptions\JobConfigLoadingException` is thrown.
@ -3458,41 +3458,43 @@ production.
### State Management ### State Management
Web application is a SPA (Single Page Application. When the user accesses the page, Web application is a SPA (Single Page Application. When the user accesses the
the source codes are downloaded and are interpreted by the web browser. The communication page, the source codes are downloaded and are interpreted by the web browser.
between the browser and the server then runs in the background without reloading the page. The communication between the browser and the server then runs in the background
without reloading the page.
The application keeps its internal state which can be altered by the actions of the user The application keeps its internal state which can be altered by the actions of
(e.g., clicking on links and buttons, filling input fields of forms) and by the outcomes the user (e.g., clicking on links and buttons, filling input fields of forms)
of HTTP requests to the API server. This internal state is kept in memory of the web and by the outcomes of HTTP requests to the API server. This internal state is
browser and is not persisted in any way -- when the page is refreshed, the internal state kept in memory of the web browser and is not persisted in any way -- when the
is deleted and a new one is created from scratch (i.e., all of the data is fetched from the page is refreshed, the internal state is deleted and a new one is created from
API server again). scratch (i.e., all of the data is fetched from the API server again).
The only part of the state which is persisted is the token of the logged in user. This token The only part of the state which is persisted is the token of the logged in
is kept in cookies and in the local storage. Keeping the token in the cookies is necessary user. This token is kept in cookies and in the local storage. Keeping the token
for server-side rendering. in the cookies is necessary for server-side rendering.
#### Redux #### Redux
The in-memory state is handled by the *redux* library. This library is storngly insipred The in-memory state is handled by the *redux* library. This library is strongly
by the [Flux](https://facebook.github.io/flux/) architecture but it has some specifics. inspired by the [Flux](https://facebook.github.io/flux/) architecture but it has
The whole state is in a single serializable tree structure called the *store*. This store some specifics. The whole state is in a single serializable tree structure
can be modified only by dispatching *actions* which are Plain Old JavaScript Oblects (POJO) called the *store*. This store can be modified only by dispatching *actions*
which are processed by *reducers*. A reducer is a pure function which takes the state which are Plain Old JavaScript Oblects (POJO) which are processed by *reducers*.
object and the action object and it creates a new state. This process is very easy to A reducer is a pure function which takes the state object and the action object
reason about and is also very easy to test using unit tests. Please read the and it creates a new state. This process is very easy to reason about and is
[redux documentation](http://redux.js.org/) for detailed information about the library. also very easy to test using unit tests. Please read the [redux
documentation](http://redux.js.org/) for detailed information about the library.
![Redux state handling schema](https://github.com/ReCodEx/wiki/raw/master/images/redux.png) ![Redux state handling schema](https://github.com/ReCodEx/wiki/raw/master/images/redux.png)
##### Redux Middleware ##### Redux Middleware
A middleware in redux is a function which can process actions before they are passed A middleware in redux is a function which can process actions before they are
to the reducers to update the state. passed to the reducers to update the state.
The middleware used by the ReCodEx store is defined in the `src/redux/store.js` script. The middleware used by the ReCodEx store is defined in the `src/redux/store.js`
Several open source libraries are used: script. Several open source libraries are used:
- [redux-promise-middleware](https://github.com/pburtchaell/redux-promise-middleware) - [redux-promise-middleware](https://github.com/pburtchaell/redux-promise-middleware)
- [redux-thunk](https://github.com/gaearon/redux-thunk) - [redux-thunk](https://github.com/gaearon/redux-thunk)
@ -3500,12 +3502,14 @@ Several open source libraries are used:
We created two other custom middleware functions for our needs: We created two other custom middleware functions for our needs:
- **API middleware** -- The middleware filters out all actions with the *type* set to `recodex-api/CALL`, - **API middleware** -- The middleware filters out all actions with the *type*
sends a real HTTP request according to the information in the action. set to `recodex-api/CALL`, sends a real HTTP request according to the
- **Access Token Middleware** -- This middleware persists the access token each time after the information in the action.
user signs into the application into the local storage and the cookies. The token - **Access Token Middleware** -- This middleware persists the access token each
is removed when the user decides to sign out. The middleware also attaches the token time after the user signs into the application into the local storage and the
to each `recodex-api/CALL` action when it does not have an access token set explicitly. cookies. The token is removed when the user decides to sign out. The
middleware also attaches the token to each `recodex-api/CALL` action when it
does not have an access token set explicitly.
##### Accessing The Store Using Selectors ##### Accessing The Store Using Selectors
@ -3513,96 +3517,106 @@ We created two other custom middleware functions for our needs:
#### Routing #### Routing
The page should not be reloaded after the initial render but the current location The page should not be reloaded after the initial render but the current
of the user in the system must be reflected in the URL. This is achieved through the location of the user in the system must be reflected in the URL. This is
achieved through the
[react-router](https://github.com/ReactTraining/react-router) and [react-router](https://github.com/ReactTraining/react-router) and
[react-router-redux](https://github.com/reactjs/react-router-redux) libraries. [react-router-redux](https://github.com/reactjs/react-router-redux) libraries.
These libraries use `pushState` These libraries use `pushState` method of the `history` object, a living
method of the `history` object, a living standard supported by all of the modern browsers. standard supported by all of the modern browsers. The mapping of the URLs to
The mapping of the URLs to the components is defined in the `src/pages/routes.js` file. the components is defined in the `src/pages/routes.js` file. To create links
To create links between pages, use either the `Link` component from the `react-router` library between pages, use either the `Link` component from the `react-router` library
or dispatch an action created using the `push` action creator from the `react-router-redux` or dispatch an action created using the `push` action creator from the
library. All the navigations are mapped to redux actions and can be handled by any reducer. `react-router-redux` library. All the navigations are mapped to redux actions
and can be handled by any reducer.
Having up-to-date URLs gives the users the possibility to reload the page if some
error occurs on the page and land at the same page as he or she would expect. Users can also Having up-to-date URLs gives the users the possibility to reload the page if
send links to the very page they want to. some error occurs on the page and land at the same page as he or she would
expect. Users can also send links to the very page they want to.
### Creating HTTP Requests ### Creating HTTP Requests
All of the HTTP requests are made by dispatching a specific action which will be processed All of the HTTP requests are made by dispatching a specific action which will be
by our custom *API middleware*. The action must have the *type* property set to processed by our custom *API middleware*. The action must have the *type*
`recodex-api/CALL`. The middleware catches the action and it sends a real HTTP request property set to `recodex-api/CALL`. The middleware catches the action and it
created according to the information in the `request` property of the action: sends a real HTTP request created according to the information in the `request`
property of the action:
- **type** -- Type prefix of the actions which will be dispatched automatically during the
lifecycle of the request (pending, fulfilled, failed). - **type** -- Type prefix of the actions which will be dispatched automatically
- **endpoint** -- The URI to which the request should be sent. All endpoints will be prefixed during the lifecycle of the request (pending, fulfilled, failed).
with the base URL of the API server. - **endpoint** -- The URI to which the request should be sent. All endpoints
- **method** (*optional*) -- A string containing the name of the HTTP method which should be used. will be prefixed with the base URL of the API server.
The default method is `GET`. - **method** (*optional*) -- A string containing the name of the HTTP method
- **query** (*optional*) -- An object containing key-value pairs which will be put which should be used. The default method is `GET`.
in the URL of the request in the query part of the URL. - **query** (*optional*) -- An object containing key-value pairs which will be
- **headers** (*optional*) -- An object containing key-value paris which will be appended put in the URL of the request in the query part of the URL.
to the headers of the HTTP request. - **headers** (*optional*) -- An object containing key-value pairs which will be
- **accessToken** (*optional*) -- Explicitly set the access token for the request. The token appended to the headers of the HTTP request.
will be put in the *Authorization* header. - **accessToken** (*optional*) -- Explicitly set the access token for the
- **body** (*optional*) -- An object or an array which will be recursively flattened into request. The token will be put in the *Authorization* header.
the `FormData` structure with correct usage of square brackets for nested (associative) - **body** (*optional*) -- An object or an array which will be recursively
arrays. It is worth mentioning that the keys must not contain a colon in the string. flattened into the `FormData` structure with correct usage of square brackets
- **doNotProcess** (*optional*) -- A boolean value which can disable the default procesing for nested (associative) arrays. It is worth mentioning that the keys must not
of the response to the request which includes showing a notification to the user in case contain a colon in the string.
of a failiure of the request. All requests are processed in the way described above - **doNotProcess** (*optional*) -- A boolean value which can disable the default
by default. processing of the response to the request which includes showing a
notification to the user in case of a failure of the request. All requests
The HTTP requests are sent using the `fetch` API which returns a *Promise* of the request. are processed in the way described above by default.
This promise is put into a new action and creates a new action containing the promise
and the type specified in the `request` description. This action is then caught by the The HTTP requests are sent using the `fetch` API which returns a *Promise* of
promise middleware and the promise middleware dispatches actions whenever the state of the request. This promise is put into a new action and creates a new action
the promise changes during its the lifecycle. The new actions have specific types: containing the promise and the type specified in the `request` description. This
action is then caught by the promise middleware and the promise middleware
- {$TYPE}_PENDING -- Dispatched immediatelly after the action is processed by the promise dispatches actions whenever the state of the promise changes during its the
middleware. The `payload` property of the action contains the body of the request. lifecycle. The new actions have specific types:
- {$TYPE}_PENDING -- Dispatched immediately after the action is processed by
the promise middleware. The `payload` property of the action contains the body
of the request.
- {$TYPE}_FAILED -- Dispatched if the promise of the request is rejected. - {$TYPE}_FAILED -- Dispatched if the promise of the request is rejected.
- {$TYPE}_FULFILLED -- Dispatched when the response to the request is received and the - {$TYPE}_FULFILLED -- Dispatched when the response to the request is received
promise is resolved. The `payload` property of the action contains the body of the and the promise is resolved. The `payload` property of the action contains the
HTTP response parsed as JSON. body of the HTTP response parsed as JSON.
### Routine CRUD Operations ### Routine CRUD Operations
For routine CRUD (Create, Read, Update, Delete) operations which are common to most For routine CRUD (Create, Read, Update, Delete) operations which are common to
of the resources used in the ReCodEx (e.g., groups, users, assignments, solutions, most of the resources used in the ReCodEx (e.g., groups, users, assignments,
solution evaluations, source code files) a set of functions called *Resource manager* solutions, solution evaluations, source code files) a set of functions called
was implemented. It contains a factory which creates basic actions (e.g., `fetchResource`, *Resource manager* was implemented. It contains a factory which creates basic
`addResource`, `updateResource`, `removeResource`, `fetchMany`) and handlers for actions (e.g., `fetchResource`, `addResource`, `updateResource`,
all of the lifecycle actions created by both the API middleware and the promise middleware `removeResource`, `fetchMany`) and handlers for all of the lifecycle actions
which can be used to create a basic reducer. created by both the API middleware and the promise middleware which can be used
to create a basic reducer.
The *resource manager* is spread over several files in the `src/redux/helpers/resourceManager`
directory and is covered with unit tests in scripts located at The *resource manager* is spread over several files in the
`test/redux/helpers/resourceManager`. `src/redux/helpers/resourceManager` directory and is covered with unit tests in
scripts located at `test/redux/helpers/resourceManager`.
### Servier-side Rendering
### Server-side Rendering
To speed-up the initial time of rendering of the web application a technique called server-side
rendering (SSR) is used. The same code which is executed in the web browser of the client can run To speed-up the initial time of rendering of the web application a technique
on the server using [Node.js](https://nodejs.org). React can serialize its HTML output into called server-side rendering (SSR) is used. The same code which is executed in
a string which can be sent to the client and can be displayed before the (potentially large) the web browser of the client can run on the server using
JavaScript source code starts being executed by the browser. The redux store is in fact [Node.js](https://nodejs.org). React can serialize its HTML output into a string
just a large JSON tree which can be easily serialized as well. which can be sent to the client and can be displayed before the (potentially
large) JavaScript source code starts being executed by the browser. The redux
If the user is logged in then the access token should be in the cookies of the web browser store is in fact just a large JSON tree which can be easily serialized as well.
and it should be attached to the HTTP request when the user navigates to the ReCodEx
web page. This token is then put into the redux store and so the user is logged in on If the user is logged in then the access token should be in the cookies of the
the server. web browser and it should be attached to the HTTP request when the user
navigates to the ReCodEx web page. This token is then put into the redux store
The whole logic of the SSR is in a single file called `src/server.js`. It contains only and so the user is logged in on the server.
a definition of a simple HTTP server (using the [express](http://expressjs.com/) framework)
and some necessary boilerplate of the routing library. The whole logic of the SSR is in a single file called `src/server.js`. It
contains only a definition of a simple HTTP server (using the
All the components which are associated to the matched route can have a class property `loadAsync` [express](http://expressjs.com/) framework) and some necessary boilerplate of
which should contain a function returning a *Promise*. The SRR calls all these functions and delays the routing library.
the response of the HTTP server until all of the promises are resolved (or some of them fails).
All the components which are associated to the matched route can have a class
property `loadAsync` which should contain a function returning a *Promise*. The
SRR calls all these functions and delays the response of the HTTP server until
all of the promises are resolved (or some of them fails).
## Communication Protocol ## Communication Protocol
@ -3611,7 +3625,7 @@ Detailed communication inside the ReCodEx system is captured in the following
image and described in sections below. Red connections are through ZeroMQ image and described in sections below. Red connections are through ZeroMQ
sockets, blue are through WebSockets and green are through HTTP(S). All ZeroMQ sockets, blue are through WebSockets and green are through HTTP(S). All ZeroMQ
messages are sent as multipart with one string (command, option) per part, with messages are sent as multipart with one string (command, option) per part, with
no empty frames (unles explicitly specified otherwise). no empty frames (unless explicitly specified otherwise).
![Communication schema](https://github.com/ReCodEx/wiki/raw/master/images/Backend_Connections.png) ![Communication schema](https://github.com/ReCodEx/wiki/raw/master/images/Backend_Connections.png)
@ -3667,7 +3681,7 @@ message from broker to worker must be target the socket identity of the worker
message frames: message frames:
- `job_id` -- identifier of current job - `job_id` -- identifier of current job
- `command` -- what is happening now. - `command` -- what is happening now.
- DOWNLOADED -- submission successfuly fetched from fileserver - DOWNLOADED -- submission successfully fetched from fileserver
- FAILED -- something bad happened and job was not executed at all - FAILED -- something bad happened and job was not executed at all
- UPLOADED -- results are uploaded to fileserver - UPLOADED -- results are uploaded to fileserver
- STARTED -- evaluation of tasks started - STARTED -- evaluation of tasks started
@ -3726,7 +3740,7 @@ capable to send corresponding credentials with each request.
#### Worker Side #### Worker Side
Workers comunicate with the file server in both directions -- they download the Workers communicate with the file server in both directions -- they download the
submissions of the student and then upload evaluation results. Internally, submissions of the student and then upload evaluation results. Internally,
worker is using libcurl C library with very similar setup. In both cases it can worker is using libcurl C library with very similar setup. In both cases it can
verify HTTPS certificate (on Linux against system cert list, on Windows against verify HTTPS certificate (on Linux against system cert list, on Windows against
@ -3777,7 +3791,7 @@ broker and workers. The current architecture prefers the broker to do all the
communication so that the workers do not have to know too many network services. communication so that the workers do not have to know too many network services.
Monitor is treated as a somewhat optional part of whole solution, so no special Monitor is treated as a somewhat optional part of whole solution, so no special
effort on communication realibility was made. effort on communication reliability was made.
#### Commands from Monitor to Broker: #### Commands from Monitor to Broker:
@ -3972,7 +3986,7 @@ completing. Surely, the list is not complete and may change in time.
- Write alternative commandline frontend. A lot of users want to submit their - Write alternative commandline frontend. A lot of users want to submit their
solutions directly from command line. That is newly possible in ReCodEx solutions directly from command line. That is newly possible in ReCodEx
because of REST API, but no suitable frontend exists yet. There is unfinished because of REST API, but no suitable frontend exists yet. There is unfinished
attempt to create one in NodeJS, accessible in ReCodEx organisation on attempt to create one in NodeJS, accessible in ReCodEx organization on
[GitHub](https://github.com/ReCodEx/cli). The goal is to finish this project [GitHub](https://github.com/ReCodEx/cli). The goal is to finish this project
or create alternative tool. We would like to see one written in Python. or create alternative tool. We would like to see one written in Python.
- Create mobile frontend. It would be really nice to have mobile application - Create mobile frontend. It would be really nice to have mobile application
@ -3991,7 +4005,7 @@ completing. Surely, the list is not complete and may change in time.
- Finish .NET sandbox for Windows. We developed initial sandbox for Windows - Finish .NET sandbox for Windows. We developed initial sandbox for Windows
environment, [WrapSharp](https://github.com/ReCodEx/wrapsharp). WrapSharp is environment, [WrapSharp](https://github.com/ReCodEx/wrapsharp). WrapSharp is
only for .NET platform assemblies and most notably for C# programs and cannot only for .NET platform assemblies and most notably for C# programs and cannot
be used generraly for sandboxing on Windows. The goal is to finish be used generally for sandboxing on Windows. The goal is to finish
implementation and do a really detailed security audit of the project. Also, implementation and do a really detailed security audit of the project. Also,
integration to the worker is not fully done yet. integration to the worker is not fully done yet.
- SIS integration. A very nice feature is (semi)automatic creation of groups and - SIS integration. A very nice feature is (semi)automatic creation of groups and

Write Preview
Loading…
Cancel
Save