LEdoian
/
recodex-wiki
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Typos

Browse Source
master
Petr Stefan 8 years ago
parent 2cc1147f6d
commit d12fde67dc
1 changed files with 172 additions and 158 deletions
Show Stats Download Patch File Download Diff File

328
Rewritten-docs.md
Unescape Escape View File

@ -537,8 +537,8 @@ For a job evaluation, the tasks need to be executed sequentially in a specified
order. Running independent tasks is possible, but there are complications --
exact time measurement requires a controlled environment with as few
interruptions as possible from other processes. It would be possible to run
tasks that do not need exact time measuremet in parallel, but in this case a
synchronization mechanism has to be developed to exclude paralellism for
tasks that do not need exact time measurement in parallel, but in this case a
synchronization mechanism has to be developed to exclude parallelism for
measured tasks. Usually, there are about four times more unmeasured tasks than
tasks with time measurement, but measured tasks tend to be much longer. With
[Amdahl's law](https://en.wikipedia.org/wiki/Amdahl's_law) in mind, the
@ -546,7 +546,7 @@ parallelism does not seem to provide a notable benefit in overall execution
speed and brings trouble with synchronization. Moreover, most of the internal
tasks are also limited by IO speed (most notably copying and downloading files
and reading archives). However, if there are performance issues, this approach
could be reconsiderred, along with using a ram disk for storing supplementary
could be reconsidered, along with using a ram disk for storing supplementary
files.
It seems that connecting tasks into directed acyclic graph (DAG) can handle all
@ -662,7 +662,7 @@ To avoid assigning points for insufficient solutions (like only printing "File
error" which is the valid answer in two tests), a minimal point threshold can be
specified. If the solution is to get less points than specified, it will get
zero points instead. This functionality can be embedded into grading computation
algoritm itself, but it would have to be present in each implementation
algorithm itself, but it would have to be present in each implementation
separately, which is not maintainable. Because of this the threshold feature is
separated from score computation.
@ -869,7 +869,7 @@ for this approach than RPC. Moreover, we rarely need to receive replies to our
requests immediately.
RabbitMQ seems well suited for many use cases, but implementing a job routing
mechanism between heterogenous workers would be complicated -- we would probably
mechanism between heterogeneous workers would be complicated -- we would probably
have to create a separate load balancing service, which cancels the advantage of
a message broker already being provided by the framework. It is also written in
Erlang, which nobody from our team understands.
@ -988,8 +988,8 @@ considered ones are:
- *XML* -- broadly used general markup language which is flavoured with document
type definition (DTD) which can express and check XML file structure, so it
does not have to be checked within application. But XML with its tags can be
sometimes quite 'chatty' and extensive which is not desirable. And overally
XML with all its features and properties can be a bit heavy-weight.
sometimes quite 'chatty' and extensive which is not desirable. And overly XML
with all its features and properties can be a bit heavy-weight.
- *JSON* -- a notation which was developed to represent javascript objects. As
such it is quite simple, there can be expressed only: key-value structures,
arrays and primitive values. Structure and hierarchy of data is solved by
@ -998,7 +998,7 @@ considered ones are:
key-value structures which can be grouped into sections. This is not enough
to represent a job and its tasks hierarchy.
- *YAML* -- format which is very similar to JSON with its capabilities. But with
small difference in structure and hirarchy of configuration which is solved
small difference in structure and hierarchy of configuration which is solved
not with braces but with indentation. This means that YAML is easily readable
by both human and machine.
- *specific format* -- newly created format used just for job configuration.
@ -1230,7 +1230,7 @@ one. Implementation of first one is quite straightforward and clear. So let us
discuss what should be happening in execution subsystem.
After successful arrival of the job from broker to the listening thread, the job
is immediatelly redirected to execution thread. In there worker has to prepare
is immediately redirected to execution thread. In there worker has to prepare
new execution environment, solution archive has to be downloaded from fileserver
and extracted. Job configuration is located within these files and loaded into
internal structures and executed. After that, results are uploaded back to
@ -1589,9 +1589,9 @@ To speed up the development process of the PHP server application we decided to
use a web framework. After evaluating and trying several frameworks, such as
Lumen, Laravel, and Symfony, we ended up using Nette.
- **Lumen** and **Laravel** seemed promissing but the default ORM framework
- **Lumen** and **Laravel** seemed promising but the default ORM framework
Eloquent is an implementation of ActiveRecord which we wanted to avoid. It
was also suprisingly complicated to implement custom middleware for validation
was also surprisingly complicated to implement custom middleware for validation
of access tokens in the headers of incoming HTTP requests.
- **Symfony** is a very good framework and has Doctrine "built-in". The reason
why we did not use Symfony in the end was our lack of experience with this
@ -1625,7 +1625,7 @@ both the URL and the HTTP method of the request.
#### Authentication
To make certain data and actions acessible only for some specific users, there
To make certain data and actions accessible only for some specific users, there
must be a way how these users can prove their identity. We decided to avoid PHP
sessions to make the server stateless (session ID is stored in the cookies of
the HTTP requests and responses). The server issues a specific token for the
@ -1697,7 +1697,7 @@ several advantages:
- no installation or setup is required on the device of the user
- works on all platforms including mobile devices
- when a new version is released, all the clients will use this version without
any need for manual instalation of the update
any need for manual installation of the update
One of the downsides is the large number of different web browsers (including
the older versions of a specific browser) and their different interpretation
@ -1754,7 +1754,7 @@ worth considering:
- **Angular 2** - it is a new framework which was developed by Google. This
framework is very complex and provides the developer with many tools which
make creating a website very straigtforward. The code can be written in pure
make creating a website very straightforward. The code can be written in pure
JavaScript (ES5) or using the TypeScript language which is then transpiled
into JavaScript. Creating a web application in Angular 2 is based on creating
and composing components. The previous version of Angular is not compatible
@ -1772,7 +1772,7 @@ worth considering:
We decided to use React and Redux over Angular 2 for several reasons:
- There is a large community arround these libraries and there is a large number
- There is a large community around these libraries and there is a large number
of tutorials, libraries, and other resources available online.
- Many of the web frontend developers are familiar with React and Redux and
contributing to the project should be easy for them.
@ -1939,7 +1939,7 @@ description of dashboard will be provided later on with according roles.
## Student
Student is a default role for every newly registered user. This role has quite
limited capabilites in ReCodEx. Generally, a student can only submit solutions
limited capabilities in ReCodEx. Generally, a student can only submit solutions
of exercises in some particular groups. These groups should correspond to
courses he/she attends.
@ -2201,7 +2201,7 @@ updated. To remove assigned bonus points, submit just the zero number. The bonus
points are not additive, newer value overrides older values.
It is useful to give a feedback about the solution back to the user. For this
you can use the "Commens and notes" box. Make sure that the messages are not
you can use the "Comments and notes" box. Make sure that the messages are not
private, so that the student can see them. More detailed description of this box
can be nicely used the "Comments and notes" box. Make sure that the messages are
not private, so the student can see them. More detailed description of this box
@ -2660,7 +2660,7 @@ communication. It was designed to maintain a heavy load of messages by making
only small actions in the main communication thread and asynchronous execution
of other actions.
The responsibilites of broker are:
The responsibilities of broker are:
- allowing workers to register themselves and keep track of their capabilities
- tracking status of each worker and handle cases when they crash
@ -2672,19 +2672,19 @@ The responsibilites of broker are:
### Internal Structure
The main work of the broker is to handle incomming messages. For that a
_reactor_ subcomponent is written to bind events on sockets to handler classes.
There are currently two handlers -- one that handles the main functionality and
the other that sends status reports to the REST API asynchronously. This
prevents broker freezes when synchronously waiting for responses of HTTP
requests, especially when some kind of error happens on the server.
The main work of the broker is to handle incoming messages. For that a _reactor_
subcomponent is written to bind events on sockets to handler classes. There are
currently two handlers -- one that handles the main functionality and the other
that sends status reports to the REST API asynchronously. This prevents broker
freezes when synchronously waiting for responses of HTTP requests, especially
when some kind of error happens on the server.
Main handler takes care of requests from workers and API servers:
- *init* -- initial connection from worker to broker
- *done* -- currently processed job on worker was executed and is done
- *ping* -- worker prooving that it is still alive
- *progress* -- job progress state from worker which is immediatelly forwarded
- *ping* -- worker proving that it is still alive
- *progress* -- job progress state from worker which is immediately forwarded
to monitor
- *eval* -- request from API server to execute given job
@ -2727,7 +2727,7 @@ Following types of failures are distinguished:
student solution is not considered as a job failure.
Jobs that failed internally are reassigned until a limit on the amount of
reassingments (configurable with the `max_request_failures` option) is reached.
reassignments (configurable with the `max_request_failures` option) is reached.
External failures are reported to the frontend immediately.
**Worker failure** -- when a worker crash is detected, an attempt to reassign
@ -2757,7 +2757,7 @@ Broker implementation depends on several open-source C and C++ libraries.
- **spdlog** -- Spdlog is small, fast and modern logging library used for system
logging. It is highly customizable and configurable from the
configuration of the broker.
- **yaml-cpp** -- Yaml-cpp is used for parsing borker configuration text file in
- **yaml-cpp** -- Yaml-cpp is used for parsing broker configuration text file in
YAML format.
- **boost-filesystem** -- Boost filesystem is used for managing logging
directory (create if necessary) and parsing filesystem paths from strings as
@ -2773,7 +2773,7 @@ Broker implementation depends on several open-source C and C++ libraries.
## Fileserver
The fileserver component provides a shared file storage between the frontend and
the backend. It is writtend in Python 3 using Flask web framework. Fileserver
the backend. It is written in Python 3 using Flask web framework. Fileserver
stores files in configurable filesystem directory, provides file deduplication
and HTTP access. To keep the stored data safe, the fileserver should not be
visible from public internet. Instead, it should be accessed indirectly through
@ -2832,7 +2832,7 @@ evaluation request, worker has to do following:
### Internal Structure
Worker is logicaly divided into two parts:
Worker is logically divided into two parts:
- **Listener** -- communicates with broker through ZeroMQ. On startup, it
introduces itself to the broker. Then it receives new jobs, passes them to
@ -2879,7 +2879,7 @@ administrator.
Student submissions are executed in a sandbox environment to prevent them from
damaging the host system and also to restrict the amount of used resources.
Currently, only the Isolate sandbox support is implemented, but it is possible
to add support for another sandox.
to add support for another sandbox.
Every sandbox, regardless of the concrete implementation, has to be a command
line application taking parameters with arguments, standard input or file.
@ -2948,7 +2948,7 @@ To allow passing these additional values an extended judge interface can be
implemented:
- Parameters: There are two mandatory positional parameters which have to be
files for comparision
files for comparison
- Results:
- _comparison OK_
- exitcode: 0
@ -2978,7 +2978,7 @@ them are multi-platform, so both Linux and Windows builds are possible.
- **libcurl** -- Libcurl is used for all HTTP communication, that is downloading
and uploading files. Due to lack of documentation of all C++ bindings the
plain C API is used.
- **libarchive** -- Libarchive is ised for compressing and extracting archives.
- **libarchive** -- Libarchive is used for compressing and extracting archives.
Actual supported formats depends on installed packages on target system, but
at least ZIP and TAR.GZ should be available.
- **cppzmq** -- Cppzmq is a simple C++ wrapper for core ZeroMQ C API. It
@ -3023,7 +3023,7 @@ unencrypted connection and will not show the progress to the users.
Monitor runs in 2 threads. _Thread 1_ is the main thread, which initializes all
components (logger for example), starts the other thread and runs the ZeroMQ
part of the application. This thread receives and parses incomming messages from
part of the application. This thread receives and parses incoming messages from
broker and forwards them to _thread 2_ sending logic.
_Thread 2_ is responsible for managing all of WebSocket connections
@ -3033,12 +3033,12 @@ all events from other threads (actually only `send_message` method invocation)
must be called within the event loop (via `asyncio.loop.call_soon_threadsafe`
function). Please note, that most of the Python interpreters use [Global
Interpreter Lock](https://wiki.python.org/moin/GlobalInterpreterLock), so there
is actualy no parallelism in the performance point of view, but proper
is actually no parallelism in the performance point of view, but proper
synchronization is still required.
### Handling of Incomming Messages
### Handling of Incoming Messages
Incomming ZeroMQ progress message is received and parsed to JSON format (same as
Incoming ZeroMQ progress message is received and parsed to JSON format (same as
our WebSocket communication format). JSON string is then passed to _thread 2_
for asynchronous sending. Each message has an identifier of channel where to
send it to.
@ -3141,7 +3141,7 @@ its own attribute (database column). Current supported options are `darkTheme`,
Every user has a role in the system. The basic ones are student, supervisor and
administrator, but new roles can be created by adding `Role` entities. Roles can
have permissions associated with them. These associations are represented by
`Permission` entitites. Each permission consists of a role, resource, action and
`Permission` entities. Each permission consists of a role, resource, action and
an `isAllowed` flag. If the `isAllowed` flag is set to true, the permission is
positive (lets the role access the resource), and if it is false, it denies
access. The `Resource` entity contains just a string identifier of a resource
@ -3386,7 +3386,7 @@ does not have to worry about it.
### Job Configuration Parsing and Modifying
Even in the API the job configuration file can be loaded in the corresponding
internal stuctures. This is necessary because there has to be possibility to
internal structures. This is necessary because there has to be possibility to
modify particular job details, such as the job identification or the fileserver
address, during the submission.
@ -3403,7 +3403,7 @@ should have *get* counterparts. Job configuration is serialized through
`__toString()` methods.
For loading of the job configuration there is separate `Storage` class which can
be used for loading, saving or archivating of job configuration. For parsing the
be used for loading, saving or archiving of job configuration. For parsing the
storage uses the `Loader` class which does all the checks and loads the data
from given strings in the appropriate structures. In case of parser error
`App\Exceptions\JobConfigLoadingException` is thrown.
@ -3458,41 +3458,43 @@ production.
### State Management
Web application is a SPA (Single Page Application. When the user accesses the page,
the source codes are downloaded and are interpreted by the web browser. The communication
between the browser and the server then runs in the background without reloading the page.
Web application is a SPA (Single Page Application. When the user accesses the
page, the source codes are downloaded and are interpreted by the web browser.
The communication between the browser and the server then runs in the background
without reloading the page.
The application keeps its internal state which can be altered by the actions of the user
(e.g., clicking on links and buttons, filling input fields of forms) and by the outcomes
of HTTP requests to the API server. This internal state is kept in memory of the web
browser and is not persisted in any way -- when the page is refreshed, the internal state
is deleted and a new one is created from scratch (i.e., all of the data is fetched from the
API server again).
The application keeps its internal state which can be altered by the actions of
the user (e.g., clicking on links and buttons, filling input fields of forms)
and by the outcomes of HTTP requests to the API server. This internal state is
kept in memory of the web browser and is not persisted in any way -- when the
page is refreshed, the internal state is deleted and a new one is created from
scratch (i.e., all of the data is fetched from the API server again).
The only part of the state which is persisted is the token of the logged in user. This token
is kept in cookies and in the local storage. Keeping the token in the cookies is necessary
for server-side rendering.
The only part of the state which is persisted is the token of the logged in
user. This token is kept in cookies and in the local storage. Keeping the token
in the cookies is necessary for server-side rendering.
#### Redux
The in-memory state is handled by the *redux* library. This library is storngly insipred
by the [Flux](https://facebook.github.io/flux/) architecture but it has some specifics.
The whole state is in a single serializable tree structure called the *store*. This store
can be modified only by dispatching *actions* which are Plain Old JavaScript Oblects (POJO)
which are processed by *reducers*. A reducer is a pure function which takes the state
object and the action object and it creates a new state. This process is very easy to
reason about and is also very easy to test using unit tests. Please read the
[redux documentation](http://redux.js.org/) for detailed information about the library.
The in-memory state is handled by the *redux* library. This library is strongly
inspired by the [Flux](https://facebook.github.io/flux/) architecture but it has
some specifics. The whole state is in a single serializable tree structure
called the *store*. This store can be modified only by dispatching *actions*
which are Plain Old JavaScript Oblects (POJO) which are processed by *reducers*.
A reducer is a pure function which takes the state object and the action object
and it creates a new state. This process is very easy to reason about and is
also very easy to test using unit tests. Please read the [redux
documentation](http://redux.js.org/) for detailed information about the library.
![Redux state handling schema](https://github.com/ReCodEx/wiki/raw/master/images/redux.png)
##### Redux Middleware
A middleware in redux is a function which can process actions before they are passed
to the reducers to update the state.
A middleware in redux is a function which can process actions before they are
passed to the reducers to update the state.
The middleware used by the ReCodEx store is defined in the `src/redux/store.js` script.
Several open source libraries are used:
The middleware used by the ReCodEx store is defined in the `src/redux/store.js`
script. Several open source libraries are used:
- [redux-promise-middleware](https://github.com/pburtchaell/redux-promise-middleware)
- [redux-thunk](https://github.com/gaearon/redux-thunk)
@ -3500,12 +3502,14 @@ Several open source libraries are used:
We created two other custom middleware functions for our needs:
- **API middleware** -- The middleware filters out all actions with the *type* set to `recodex-api/CALL`,
sends a real HTTP request according to the information in the action.
- **Access Token Middleware** -- This middleware persists the access token each time after the
user signs into the application into the local storage and the cookies. The token
is removed when the user decides to sign out. The middleware also attaches the token
to each `recodex-api/CALL` action when it does not have an access token set explicitly.
- **API middleware** -- The middleware filters out all actions with the *type*
set to `recodex-api/CALL`, sends a real HTTP request according to the
information in the action.
- **Access Token Middleware** -- This middleware persists the access token each
time after the user signs into the application into the local storage and the
cookies. The token is removed when the user decides to sign out. The
middleware also attaches the token to each `recodex-api/CALL` action when it
does not have an access token set explicitly.
##### Accessing The Store Using Selectors
@ -3513,96 +3517,106 @@ We created two other custom middleware functions for our needs:
#### Routing
The page should not be reloaded after the initial render but the current location
of the user in the system must be reflected in the URL. This is achieved through the
The page should not be reloaded after the initial render but the current
location of the user in the system must be reflected in the URL. This is
achieved through the
[react-router](https://github.com/ReactTraining/react-router) and
[react-router-redux](https://github.com/reactjs/react-router-redux) libraries.
These libraries use `pushState`
method of the `history` object, a living standard supported by all of the modern browsers.
The mapping of the URLs to the components is defined in the `src/pages/routes.js` file.
To create links between pages, use either the `Link` component from the `react-router` library
or dispatch an action created using the `push` action creator from the `react-router-redux`
library. All the navigations are mapped to redux actions and can be handled by any reducer.
Having up-to-date URLs gives the users the possibility to reload the page if some
error occurs on the page and land at the same page as he or she would expect. Users can also
send links to the very page they want to.
These libraries use `pushState` method of the `history` object, a living
standard supported by all of the modern browsers. The mapping of the URLs to
the components is defined in the `src/pages/routes.js` file. To create links
between pages, use either the `Link` component from the `react-router` library
or dispatch an action created using the `push` action creator from the
`react-router-redux` library. All the navigations are mapped to redux actions
and can be handled by any reducer.
Having up-to-date URLs gives the users the possibility to reload the page if
some error occurs on the page and land at the same page as he or she would
expect. Users can also send links to the very page they want to.
### Creating HTTP Requests
All of the HTTP requests are made by dispatching a specific action which will be processed
by our custom *API middleware*. The action must have the *type* property set to
`recodex-api/CALL`. The middleware catches the action and it sends a real HTTP request
created according to the information in the `request` property of the action:
- **type** -- Type prefix of the actions which will be dispatched automatically during the
lifecycle of the request (pending, fulfilled, failed).
- **endpoint** -- The URI to which the request should be sent. All endpoints will be prefixed
with the base URL of the API server.
- **method** (*optional*) -- A string containing the name of the HTTP method which should be used.
The default method is `GET`.
- **query** (*optional*) -- An object containing key-value pairs which will be put
in the URL of the request in the query part of the URL.
- **headers** (*optional*) -- An object containing key-value paris which will be appended
to the headers of the HTTP request.
- **accessToken** (*optional*) -- Explicitly set the access token for the request. The token
will be put in the *Authorization* header.
- **body** (*optional*) -- An object or an array which will be recursively flattened into
the `FormData` structure with correct usage of square brackets for nested (associative)
arrays. It is worth mentioning that the keys must not contain a colon in the string.
- **doNotProcess** (*optional*) -- A boolean value which can disable the default procesing
of the response to the request which includes showing a notification to the user in case
of a failiure of the request. All requests are processed in the way described above
by default.
The HTTP requests are sent using the `fetch` API which returns a *Promise* of the request.
This promise is put into a new action and creates a new action containing the promise
and the type specified in the `request` description. This action is then caught by the
promise middleware and the promise middleware dispatches actions whenever the state of
the promise changes during its the lifecycle. The new actions have specific types:
- {$TYPE}_PENDING -- Dispatched immediatelly after the action is processed by the promise
middleware. The `payload` property of the action contains the body of the request.
All of the HTTP requests are made by dispatching a specific action which will be
processed by our custom *API middleware*. The action must have the *type*
property set to `recodex-api/CALL`. The middleware catches the action and it
sends a real HTTP request created according to the information in the `request`
property of the action:
- **type** -- Type prefix of the actions which will be dispatched automatically
during the lifecycle of the request (pending, fulfilled, failed).
- **endpoint** -- The URI to which the request should be sent. All endpoints
will be prefixed with the base URL of the API server.
- **method** (*optional*) -- A string containing the name of the HTTP method
which should be used. The default method is `GET`.
- **query** (*optional*) -- An object containing key-value pairs which will be
put in the URL of the request in the query part of the URL.
- **headers** (*optional*) -- An object containing key-value pairs which will be
appended to the headers of the HTTP request.
- **accessToken** (*optional*) -- Explicitly set the access token for the
request. The token will be put in the *Authorization* header.
- **body** (*optional*) -- An object or an array which will be recursively
flattened into the `FormData` structure with correct usage of square brackets
for nested (associative) arrays. It is worth mentioning that the keys must not
contain a colon in the string.
- **doNotProcess** (*optional*) -- A boolean value which can disable the default
processing of the response to the request which includes showing a
notification to the user in case of a failure of the request. All requests
are processed in the way described above by default.
The HTTP requests are sent using the `fetch` API which returns a *Promise* of
the request. This promise is put into a new action and creates a new action
containing the promise and the type specified in the `request` description. This
action is then caught by the promise middleware and the promise middleware
dispatches actions whenever the state of the promise changes during its the
lifecycle. The new actions have specific types:
- {$TYPE}_PENDING -- Dispatched immediately after the action is processed by
the promise middleware. The `payload` property of the action contains the body
of the request.
- {$TYPE}_FAILED -- Dispatched if the promise of the request is rejected.
- {$TYPE}_FULFILLED -- Dispatched when the response to the request is received and the
promise is resolved. The `payload` property of the action contains the body of the
HTTP response parsed as JSON.
- {$TYPE}_FULFILLED -- Dispatched when the response to the request is received
and the promise is resolved. The `payload` property of the action contains the
body of the HTTP response parsed as JSON.
### Routine CRUD Operations
For routine CRUD (Create, Read, Update, Delete) operations which are common to most
of the resources used in the ReCodEx (e.g., groups, users, assignments, solutions,
solution evaluations, source code files) a set of functions called *Resource manager*
was implemented. It contains a factory which creates basic actions (e.g., `fetchResource`,
`addResource`, `updateResource`, `removeResource`, `fetchMany`) and handlers for
all of the lifecycle actions created by both the API middleware and the promise middleware
which can be used to create a basic reducer.
The *resource manager* is spread over several files in the `src/redux/helpers/resourceManager`
directory and is covered with unit tests in scripts located at
`test/redux/helpers/resourceManager`.
### Servier-side Rendering
To speed-up the initial time of rendering of the web application a technique called server-side
rendering (SSR) is used. The same code which is executed in the web browser of the client can run
on the server using [Node.js](https://nodejs.org). React can serialize its HTML output into
a string which can be sent to the client and can be displayed before the (potentially large)
JavaScript source code starts being executed by the browser. The redux store is in fact
just a large JSON tree which can be easily serialized as well.
If the user is logged in then the access token should be in the cookies of the web browser
and it should be attached to the HTTP request when the user navigates to the ReCodEx
web page. This token is then put into the redux store and so the user is logged in on
the server.
The whole logic of the SSR is in a single file called `src/server.js`. It contains only
a definition of a simple HTTP server (using the [express](http://expressjs.com/) framework)
and some necessary boilerplate of the routing library.
All the components which are associated to the matched route can have a class property `loadAsync`
which should contain a function returning a *Promise*. The SRR calls all these functions and delays
the response of the HTTP server until all of the promises are resolved (or some of them fails).
For routine CRUD (Create, Read, Update, Delete) operations which are common to
most of the resources used in the ReCodEx (e.g., groups, users, assignments,
solutions, solution evaluations, source code files) a set of functions called
*Resource manager* was implemented. It contains a factory which creates basic
actions (e.g., `fetchResource`, `addResource`, `updateResource`,
`removeResource`, `fetchMany`) and handlers for all of the lifecycle actions
created by both the API middleware and the promise middleware which can be used
to create a basic reducer.
The *resource manager* is spread over several files in the
`src/redux/helpers/resourceManager` directory and is covered with unit tests in
scripts located at `test/redux/helpers/resourceManager`.
### Server-side Rendering
To speed-up the initial time of rendering of the web application a technique
called server-side rendering (SSR) is used. The same code which is executed in
the web browser of the client can run on the server using
[Node.js](https://nodejs.org). React can serialize its HTML output into a string
which can be sent to the client and can be displayed before the (potentially
large) JavaScript source code starts being executed by the browser. The redux
store is in fact just a large JSON tree which can be easily serialized as well.
If the user is logged in then the access token should be in the cookies of the
web browser and it should be attached to the HTTP request when the user
navigates to the ReCodEx web page. This token is then put into the redux store
and so the user is logged in on the server.
The whole logic of the SSR is in a single file called `src/server.js`. It
contains only a definition of a simple HTTP server (using the
[express](http://expressjs.com/) framework) and some necessary boilerplate of
the routing library.
All the components which are associated to the matched route can have a class
property `loadAsync` which should contain a function returning a *Promise*. The
SRR calls all these functions and delays the response of the HTTP server until
all of the promises are resolved (or some of them fails).
## Communication Protocol
@ -3611,7 +3625,7 @@ Detailed communication inside the ReCodEx system is captured in the following
image and described in sections below. Red connections are through ZeroMQ
sockets, blue are through WebSockets and green are through HTTP(S). All ZeroMQ
messages are sent as multipart with one string (command, option) per part, with
no empty frames (unles explicitly specified otherwise).
no empty frames (unless explicitly specified otherwise).
![Communication schema](https://github.com/ReCodEx/wiki/raw/master/images/Backend_Connections.png)
@ -3667,7 +3681,7 @@ message from broker to worker must be target the socket identity of the worker
message frames:
- `job_id` -- identifier of current job
- `command` -- what is happening now.
- DOWNLOADED -- submission successfuly fetched from fileserver
- DOWNLOADED -- submission successfully fetched from fileserver
- FAILED -- something bad happened and job was not executed at all
- UPLOADED -- results are uploaded to fileserver
- STARTED -- evaluation of tasks started
@ -3726,7 +3740,7 @@ capable to send corresponding credentials with each request.
#### Worker Side
Workers comunicate with the file server in both directions -- they download the
Workers communicate with the file server in both directions -- they download the
submissions of the student and then upload evaluation results. Internally,
worker is using libcurl C library with very similar setup. In both cases it can
verify HTTPS certificate (on Linux against system cert list, on Windows against
@ -3777,7 +3791,7 @@ broker and workers. The current architecture prefers the broker to do all the
communication so that the workers do not have to know too many network services.
Monitor is treated as a somewhat optional part of whole solution, so no special
effort on communication realibility was made.
effort on communication reliability was made.
#### Commands from Monitor to Broker:
@ -3972,7 +3986,7 @@ completing. Surely, the list is not complete and may change in time.
- Write alternative commandline frontend. A lot of users want to submit their
solutions directly from command line. That is newly possible in ReCodEx
because of REST API, but no suitable frontend exists yet. There is unfinished
attempt to create one in NodeJS, accessible in ReCodEx organisation on
attempt to create one in NodeJS, accessible in ReCodEx organization on
[GitHub](https://github.com/ReCodEx/cli). The goal is to finish this project
or create alternative tool. We would like to see one written in Python.
- Create mobile frontend. It would be really nice to have mobile application
@ -3991,7 +4005,7 @@ completing. Surely, the list is not complete and may change in time.
- Finish .NET sandbox for Windows. We developed initial sandbox for Windows
environment, [WrapSharp](https://github.com/ReCodEx/wrapsharp). WrapSharp is
only for .NET platform assemblies and most notably for C# programs and cannot
be used generraly for sandboxing on Windows. The goal is to finish
be used generally for sandboxing on Windows. The goal is to finish
implementation and do a really detailed security audit of the project. Also,
integration to the worker is not fully done yet.
- SIS integration. A very nice feature is (semi)automatic creation of groups and

Write Preview
Loading…
Cancel
Save