diff --git a/Rewritten-docs.md b/Rewritten-docs.md index 616e736..2a4d3d4 100644 --- a/Rewritten-docs.md +++ b/Rewritten-docs.md @@ -443,7 +443,7 @@ duties related with lecturing of labs are already associated with supervisors, so this role does not seem useful. In addition, no one requested more than three level privilege scheme. -School labs are lessons for some students lead by supervisors. All students ina +School labs are lessons for some students lead by supervisors. All students in a lab have the same homework and supervisors evaluate their solutions. This organization has to be carried over into the new system. Virtual groups are a counterpart to real-life labs. This concept was already discussed in the @@ -537,8 +537,8 @@ For a job evaluation, the tasks need to be executed sequentially in a specified order. Running independent tasks is possible, but there are complications -- exact time measurement requires a controlled environment with as few interruptions as possible from other processes. It would be possible to run -tasks that do not need exact time measuremet in parallel, but in this case a -synchronization mechanism has to be developed to exclude paralellism for +tasks that do not need exact time measurement in parallel, but in this case a +synchronization mechanism has to be developed to exclude parallelism for measured tasks. Usually, there are about four times more unmeasured tasks than tasks with time measurement, but measured tasks tend to be much longer. With [Amdahl's law](https://en.wikipedia.org/wiki/Amdahl's_law) in mind, the @@ -546,7 +546,7 @@ parallelism does not seem to provide a notable benefit in overall execution speed and brings trouble with synchronization. Moreover, most of the internal tasks are also limited by IO speed (most notably copying and downloading files and reading archives). However, if there are performance issues, this approach -could be reconsiderred, along with using a ram disk for storing supplementary +could be reconsidered, along with using a ram disk for storing supplementary files. It seems that connecting tasks into directed acyclic graph (DAG) can handle all @@ -662,7 +662,7 @@ To avoid assigning points for insufficient solutions (like only printing "File error" which is the valid answer in two tests), a minimal point threshold can be specified. If the solution is to get less points than specified, it will get zero points instead. This functionality can be embedded into grading computation -algoritm itself, but it would have to be present in each implementation +algorithm itself, but it would have to be present in each implementation separately, which is not maintainable. Because of this the threshold feature is separated from score computation. @@ -869,7 +869,7 @@ for this approach than RPC. Moreover, we rarely need to receive replies to our requests immediately. RabbitMQ seems well suited for many use cases, but implementing a job routing -mechanism between heterogenous workers would be complicated -- we would probably +mechanism between heterogeneous workers would be complicated -- we would probably have to create a separate load balancing service, which cancels the advantage of a message broker already being provided by the framework. It is also written in Erlang, which nobody from our team understands. @@ -988,8 +988,8 @@ considered ones are: - *XML* -- broadly used general markup language which is flavoured with document type definition (DTD) which can express and check XML file structure, so it does not have to be checked within application. But XML with its tags can be - sometimes quite 'chatty' and extensive which is not desirable. And overally - XML with all its features and properties can be a bit heavy-weight. + sometimes quite 'chatty' and extensive which is not desirable. And overly XML + with all its features and properties can be a bit heavy-weight. - *JSON* -- a notation which was developed to represent javascript objects. As such it is quite simple, there can be expressed only: key-value structures, arrays and primitive values. Structure and hierarchy of data is solved by @@ -998,7 +998,7 @@ considered ones are: key-value structures which can be grouped into sections. This is not enough to represent a job and its tasks hierarchy. - *YAML* -- format which is very similar to JSON with its capabilities. But with - small difference in structure and hirarchy of configuration which is solved + small difference in structure and hierarchy of configuration which is solved not with braces but with indentation. This means that YAML is easily readable by both human and machine. - *specific format* -- newly created format used just for job configuration. @@ -1230,7 +1230,7 @@ one. Implementation of first one is quite straightforward and clear. So let us discuss what should be happening in execution subsystem. After successful arrival of the job from broker to the listening thread, the job -is immediatelly redirected to execution thread. In there worker has to prepare +is immediately redirected to execution thread. In there worker has to prepare new execution environment, solution archive has to be downloaded from fileserver and extracted. Job configuration is located within these files and loaded into internal structures and executed. After that, results are uploaded back to @@ -1589,9 +1589,9 @@ To speed up the development process of the PHP server application we decided to use a web framework. After evaluating and trying several frameworks, such as Lumen, Laravel, and Symfony, we ended up using Nette. -- **Lumen** and **Laravel** seemed promissing but the default ORM framework +- **Lumen** and **Laravel** seemed promising but the default ORM framework Eloquent is an implementation of ActiveRecord which we wanted to avoid. It - was also suprisingly complicated to implement custom middleware for validation + was also surprisingly complicated to implement custom middleware for validation of access tokens in the headers of incoming HTTP requests. - **Symfony** is a very good framework and has Doctrine "built-in". The reason why we did not use Symfony in the end was our lack of experience with this @@ -1625,7 +1625,7 @@ both the URL and the HTTP method of the request. #### Authentication -To make certain data and actions acessible only for some specific users, there +To make certain data and actions accessible only for some specific users, there must be a way how these users can prove their identity. We decided to avoid PHP sessions to make the server stateless (session ID is stored in the cookies of the HTTP requests and responses). The server issues a specific token for the @@ -1697,7 +1697,7 @@ several advantages: - no installation or setup is required on the device of the user - works on all platforms including mobile devices - when a new version is released, all the clients will use this version without -any need for manual instalation of the update + any need for manual installation of the update One of the downsides is the large number of different web browsers (including the older versions of a specific browser) and their different interpretation @@ -1754,7 +1754,7 @@ worth considering: - **Angular 2** - it is a new framework which was developed by Google. This framework is very complex and provides the developer with many tools which - make creating a website very straigtforward. The code can be written in pure + make creating a website very straightforward. The code can be written in pure JavaScript (ES5) or using the TypeScript language which is then transpiled into JavaScript. Creating a web application in Angular 2 is based on creating and composing components. The previous version of Angular is not compatible @@ -1772,7 +1772,7 @@ worth considering: We decided to use React and Redux over Angular 2 for several reasons: -- There is a large community arround these libraries and there is a large number +- There is a large community around these libraries and there is a large number of tutorials, libraries, and other resources available online. - Many of the web frontend developers are familiar with React and Redux and contributing to the project should be easy for them. @@ -1939,7 +1939,7 @@ description of dashboard will be provided later on with according roles. ## Student Student is a default role for every newly registered user. This role has quite -limited capabilites in ReCodEx. Generally, a student can only submit solutions +limited capabilities in ReCodEx. Generally, a student can only submit solutions of exercises in some particular groups. These groups should correspond to courses he/she attends. @@ -2201,7 +2201,7 @@ updated. To remove assigned bonus points, submit just the zero number. The bonus points are not additive, newer value overrides older values. It is useful to give a feedback about the solution back to the user. For this -you can use the "Commens and notes" box. Make sure that the messages are not +you can use the "Comments and notes" box. Make sure that the messages are not private, so that the student can see them. More detailed description of this box can be nicely used the "Comments and notes" box. Make sure that the messages are not private, so the student can see them. More detailed description of this box @@ -2660,7 +2660,7 @@ communication. It was designed to maintain a heavy load of messages by making only small actions in the main communication thread and asynchronous execution of other actions. -The responsibilites of broker are: +The responsibilities of broker are: - allowing workers to register themselves and keep track of their capabilities - tracking status of each worker and handle cases when they crash @@ -2672,19 +2672,19 @@ The responsibilites of broker are: ### Internal Structure -The main work of the broker is to handle incomming messages. For that a -_reactor_ subcomponent is written to bind events on sockets to handler classes. -There are currently two handlers -- one that handles the main functionality and -the other that sends status reports to the REST API asynchronously. This -prevents broker freezes when synchronously waiting for responses of HTTP -requests, especially when some kind of error happens on the server. +The main work of the broker is to handle incoming messages. For that a _reactor_ +subcomponent is written to bind events on sockets to handler classes. There are +currently two handlers -- one that handles the main functionality and the other +that sends status reports to the REST API asynchronously. This prevents broker +freezes when synchronously waiting for responses of HTTP requests, especially +when some kind of error happens on the server. Main handler takes care of requests from workers and API servers: - *init* -- initial connection from worker to broker - *done* -- currently processed job on worker was executed and is done -- *ping* -- worker prooving that it is still alive -- *progress* -- job progress state from worker which is immediatelly forwarded +- *ping* -- worker proving that it is still alive +- *progress* -- job progress state from worker which is immediately forwarded to monitor - *eval* -- request from API server to execute given job @@ -2727,7 +2727,7 @@ Following types of failures are distinguished: student solution is not considered as a job failure. Jobs that failed internally are reassigned until a limit on the amount of - reassingments (configurable with the `max_request_failures` option) is reached. + reassignments (configurable with the `max_request_failures` option) is reached. External failures are reported to the frontend immediately. **Worker failure** -- when a worker crash is detected, an attempt to reassign @@ -2757,7 +2757,7 @@ Broker implementation depends on several open-source C and C++ libraries. - **spdlog** -- Spdlog is small, fast and modern logging library used for system logging. It is highly customizable and configurable from the configuration of the broker. -- **yaml-cpp** -- Yaml-cpp is used for parsing borker configuration text file in +- **yaml-cpp** -- Yaml-cpp is used for parsing broker configuration text file in YAML format. - **boost-filesystem** -- Boost filesystem is used for managing logging directory (create if necessary) and parsing filesystem paths from strings as @@ -2773,7 +2773,7 @@ Broker implementation depends on several open-source C and C++ libraries. ## Fileserver The fileserver component provides a shared file storage between the frontend and -the backend. It is writtend in Python 3 using Flask web framework. Fileserver +the backend. It is written in Python 3 using Flask web framework. Fileserver stores files in configurable filesystem directory, provides file deduplication and HTTP access. To keep the stored data safe, the fileserver should not be visible from public internet. Instead, it should be accessed indirectly through @@ -2832,7 +2832,7 @@ evaluation request, worker has to do following: ### Internal Structure -Worker is logicaly divided into two parts: +Worker is logically divided into two parts: - **Listener** -- communicates with broker through ZeroMQ. On startup, it introduces itself to the broker. Then it receives new jobs, passes them to @@ -2879,7 +2879,7 @@ administrator. Student submissions are executed in a sandbox environment to prevent them from damaging the host system and also to restrict the amount of used resources. Currently, only the Isolate sandbox support is implemented, but it is possible -to add support for another sandox. +to add support for another sandbox. Every sandbox, regardless of the concrete implementation, has to be a command line application taking parameters with arguments, standard input or file. @@ -2948,7 +2948,7 @@ To allow passing these additional values an extended judge interface can be implemented: - Parameters: There are two mandatory positional parameters which have to be - files for comparision + files for comparison - Results: - _comparison OK_ - exitcode: 0 @@ -2978,7 +2978,7 @@ them are multi-platform, so both Linux and Windows builds are possible. - **libcurl** -- Libcurl is used for all HTTP communication, that is downloading and uploading files. Due to lack of documentation of all C++ bindings the plain C API is used. -- **libarchive** -- Libarchive is ised for compressing and extracting archives. +- **libarchive** -- Libarchive is used for compressing and extracting archives. Actual supported formats depends on installed packages on target system, but at least ZIP and TAR.GZ should be available. - **cppzmq** -- Cppzmq is a simple C++ wrapper for core ZeroMQ C API. It @@ -3023,7 +3023,7 @@ unencrypted connection and will not show the progress to the users. Monitor runs in 2 threads. _Thread 1_ is the main thread, which initializes all components (logger for example), starts the other thread and runs the ZeroMQ -part of the application. This thread receives and parses incomming messages from +part of the application. This thread receives and parses incoming messages from broker and forwards them to _thread 2_ sending logic. _Thread 2_ is responsible for managing all of WebSocket connections @@ -3033,12 +3033,12 @@ all events from other threads (actually only `send_message` method invocation) must be called within the event loop (via `asyncio.loop.call_soon_threadsafe` function). Please note, that most of the Python interpreters use [Global Interpreter Lock](https://wiki.python.org/moin/GlobalInterpreterLock), so there -is actualy no parallelism in the performance point of view, but proper +is actually no parallelism in the performance point of view, but proper synchronization is still required. -### Handling of Incomming Messages +### Handling of Incoming Messages -Incomming ZeroMQ progress message is received and parsed to JSON format (same as +Incoming ZeroMQ progress message is received and parsed to JSON format (same as our WebSocket communication format). JSON string is then passed to _thread 2_ for asynchronous sending. Each message has an identifier of channel where to send it to. @@ -3141,7 +3141,7 @@ its own attribute (database column). Current supported options are `darkTheme`, Every user has a role in the system. The basic ones are student, supervisor and administrator, but new roles can be created by adding `Role` entities. Roles can have permissions associated with them. These associations are represented by -`Permission` entitites. Each permission consists of a role, resource, action and +`Permission` entities. Each permission consists of a role, resource, action and an `isAllowed` flag. If the `isAllowed` flag is set to true, the permission is positive (lets the role access the resource), and if it is false, it denies access. The `Resource` entity contains just a string identifier of a resource @@ -3386,7 +3386,7 @@ does not have to worry about it. ### Job Configuration Parsing and Modifying Even in the API the job configuration file can be loaded in the corresponding -internal stuctures. This is necessary because there has to be possibility to +internal structures. This is necessary because there has to be possibility to modify particular job details, such as the job identification or the fileserver address, during the submission. @@ -3403,7 +3403,7 @@ should have *get* counterparts. Job configuration is serialized through `__toString()` methods. For loading of the job configuration there is separate `Storage` class which can -be used for loading, saving or archivating of job configuration. For parsing the +be used for loading, saving or archiving of job configuration. For parsing the storage uses the `Loader` class which does all the checks and loads the data from given strings in the appropriate structures. In case of parser error `App\Exceptions\JobConfigLoadingException` is thrown. @@ -3458,41 +3458,43 @@ production. ### State Management -Web application is a SPA (Single Page Application. When the user accesses the page, -the source codes are downloaded and are interpreted by the web browser. The communication -between the browser and the server then runs in the background without reloading the page. +Web application is a SPA (Single Page Application. When the user accesses the +page, the source codes are downloaded and are interpreted by the web browser. +The communication between the browser and the server then runs in the background +without reloading the page. -The application keeps its internal state which can be altered by the actions of the user -(e.g., clicking on links and buttons, filling input fields of forms) and by the outcomes -of HTTP requests to the API server. This internal state is kept in memory of the web -browser and is not persisted in any way -- when the page is refreshed, the internal state -is deleted and a new one is created from scratch (i.e., all of the data is fetched from the -API server again). +The application keeps its internal state which can be altered by the actions of +the user (e.g., clicking on links and buttons, filling input fields of forms) +and by the outcomes of HTTP requests to the API server. This internal state is +kept in memory of the web browser and is not persisted in any way -- when the +page is refreshed, the internal state is deleted and a new one is created from +scratch (i.e., all of the data is fetched from the API server again). -The only part of the state which is persisted is the token of the logged in user. This token -is kept in cookies and in the local storage. Keeping the token in the cookies is necessary -for server-side rendering. +The only part of the state which is persisted is the token of the logged in +user. This token is kept in cookies and in the local storage. Keeping the token +in the cookies is necessary for server-side rendering. #### Redux -The in-memory state is handled by the *redux* library. This library is storngly insipred -by the [Flux](https://facebook.github.io/flux/) architecture but it has some specifics. -The whole state is in a single serializable tree structure called the *store*. This store -can be modified only by dispatching *actions* which are Plain Old JavaScript Oblects (POJO) -which are processed by *reducers*. A reducer is a pure function which takes the state -object and the action object and it creates a new state. This process is very easy to -reason about and is also very easy to test using unit tests. Please read the -[redux documentation](http://redux.js.org/) for detailed information about the library. +The in-memory state is handled by the *redux* library. This library is strongly +inspired by the [Flux](https://facebook.github.io/flux/) architecture but it has +some specifics. The whole state is in a single serializable tree structure +called the *store*. This store can be modified only by dispatching *actions* +which are Plain Old JavaScript Oblects (POJO) which are processed by *reducers*. +A reducer is a pure function which takes the state object and the action object +and it creates a new state. This process is very easy to reason about and is +also very easy to test using unit tests. Please read the [redux +documentation](http://redux.js.org/) for detailed information about the library. ![Redux state handling schema](https://github.com/ReCodEx/wiki/raw/master/images/redux.png) ##### Redux Middleware -A middleware in redux is a function which can process actions before they are passed -to the reducers to update the state. +A middleware in redux is a function which can process actions before they are +passed to the reducers to update the state. -The middleware used by the ReCodEx store is defined in the `src/redux/store.js` script. -Several open source libraries are used: +The middleware used by the ReCodEx store is defined in the `src/redux/store.js` +script. Several open source libraries are used: - [redux-promise-middleware](https://github.com/pburtchaell/redux-promise-middleware) - [redux-thunk](https://github.com/gaearon/redux-thunk) @@ -3500,12 +3502,14 @@ Several open source libraries are used: We created two other custom middleware functions for our needs: -- **API middleware** -- The middleware filters out all actions with the *type* set to `recodex-api/CALL`, - sends a real HTTP request according to the information in the action. -- **Access Token Middleware** -- This middleware persists the access token each time after the - user signs into the application into the local storage and the cookies. The token - is removed when the user decides to sign out. The middleware also attaches the token - to each `recodex-api/CALL` action when it does not have an access token set explicitly. +- **API middleware** -- The middleware filters out all actions with the *type* + set to `recodex-api/CALL`, sends a real HTTP request according to the + information in the action. +- **Access Token Middleware** -- This middleware persists the access token each + time after the user signs into the application into the local storage and the + cookies. The token is removed when the user decides to sign out. The + middleware also attaches the token to each `recodex-api/CALL` action when it + does not have an access token set explicitly. ##### Accessing The Store Using Selectors @@ -3513,96 +3517,106 @@ We created two other custom middleware functions for our needs: #### Routing -The page should not be reloaded after the initial render but the current location -of the user in the system must be reflected in the URL. This is achieved through the +The page should not be reloaded after the initial render but the current +location of the user in the system must be reflected in the URL. This is +achieved through the [react-router](https://github.com/ReactTraining/react-router) and [react-router-redux](https://github.com/reactjs/react-router-redux) libraries. -These libraries use `pushState` -method of the `history` object, a living standard supported by all of the modern browsers. -The mapping of the URLs to the components is defined in the `src/pages/routes.js` file. -To create links between pages, use either the `Link` component from the `react-router` library -or dispatch an action created using the `push` action creator from the `react-router-redux` -library. All the navigations are mapped to redux actions and can be handled by any reducer. - -Having up-to-date URLs gives the users the possibility to reload the page if some -error occurs on the page and land at the same page as he or she would expect. Users can also -send links to the very page they want to. +These libraries use `pushState` method of the `history` object, a living +standard supported by all of the modern browsers. The mapping of the URLs to +the components is defined in the `src/pages/routes.js` file. To create links +between pages, use either the `Link` component from the `react-router` library +or dispatch an action created using the `push` action creator from the +`react-router-redux` library. All the navigations are mapped to redux actions +and can be handled by any reducer. + +Having up-to-date URLs gives the users the possibility to reload the page if +some error occurs on the page and land at the same page as he or she would +expect. Users can also send links to the very page they want to. ### Creating HTTP Requests -All of the HTTP requests are made by dispatching a specific action which will be processed -by our custom *API middleware*. The action must have the *type* property set to -`recodex-api/CALL`. The middleware catches the action and it sends a real HTTP request -created according to the information in the `request` property of the action: - -- **type** -- Type prefix of the actions which will be dispatched automatically during the - lifecycle of the request (pending, fulfilled, failed). -- **endpoint** -- The URI to which the request should be sent. All endpoints will be prefixed - with the base URL of the API server. -- **method** (*optional*) -- A string containing the name of the HTTP method which should be used. - The default method is `GET`. -- **query** (*optional*) -- An object containing key-value pairs which will be put - in the URL of the request in the query part of the URL. -- **headers** (*optional*) -- An object containing key-value paris which will be appended - to the headers of the HTTP request. -- **accessToken** (*optional*) -- Explicitly set the access token for the request. The token - will be put in the *Authorization* header. -- **body** (*optional*) -- An object or an array which will be recursively flattened into - the `FormData` structure with correct usage of square brackets for nested (associative) - arrays. It is worth mentioning that the keys must not contain a colon in the string. -- **doNotProcess** (*optional*) -- A boolean value which can disable the default procesing - of the response to the request which includes showing a notification to the user in case - of a failiure of the request. All requests are processed in the way described above - by default. - -The HTTP requests are sent using the `fetch` API which returns a *Promise* of the request. -This promise is put into a new action and creates a new action containing the promise -and the type specified in the `request` description. This action is then caught by the -promise middleware and the promise middleware dispatches actions whenever the state of -the promise changes during its the lifecycle. The new actions have specific types: - -- {$TYPE}_PENDING -- Dispatched immediatelly after the action is processed by the promise - middleware. The `payload` property of the action contains the body of the request. +All of the HTTP requests are made by dispatching a specific action which will be +processed by our custom *API middleware*. The action must have the *type* +property set to `recodex-api/CALL`. The middleware catches the action and it +sends a real HTTP request created according to the information in the `request` +property of the action: + +- **type** -- Type prefix of the actions which will be dispatched automatically + during the lifecycle of the request (pending, fulfilled, failed). +- **endpoint** -- The URI to which the request should be sent. All endpoints + will be prefixed with the base URL of the API server. +- **method** (*optional*) -- A string containing the name of the HTTP method + which should be used. The default method is `GET`. +- **query** (*optional*) -- An object containing key-value pairs which will be + put in the URL of the request in the query part of the URL. +- **headers** (*optional*) -- An object containing key-value pairs which will be + appended to the headers of the HTTP request. +- **accessToken** (*optional*) -- Explicitly set the access token for the + request. The token will be put in the *Authorization* header. +- **body** (*optional*) -- An object or an array which will be recursively + flattened into the `FormData` structure with correct usage of square brackets + for nested (associative) arrays. It is worth mentioning that the keys must not + contain a colon in the string. +- **doNotProcess** (*optional*) -- A boolean value which can disable the default + processing of the response to the request which includes showing a + notification to the user in case of a failure of the request. All requests + are processed in the way described above by default. + +The HTTP requests are sent using the `fetch` API which returns a *Promise* of +the request. This promise is put into a new action and creates a new action +containing the promise and the type specified in the `request` description. This +action is then caught by the promise middleware and the promise middleware +dispatches actions whenever the state of the promise changes during its the +lifecycle. The new actions have specific types: + +- {$TYPE}_PENDING -- Dispatched immediately after the action is processed by + the promise middleware. The `payload` property of the action contains the body + of the request. - {$TYPE}_FAILED -- Dispatched if the promise of the request is rejected. -- {$TYPE}_FULFILLED -- Dispatched when the response to the request is received and the - promise is resolved. The `payload` property of the action contains the body of the - HTTP response parsed as JSON. +- {$TYPE}_FULFILLED -- Dispatched when the response to the request is received + and the promise is resolved. The `payload` property of the action contains the + body of the HTTP response parsed as JSON. ### Routine CRUD Operations -For routine CRUD (Create, Read, Update, Delete) operations which are common to most -of the resources used in the ReCodEx (e.g., groups, users, assignments, solutions, -solution evaluations, source code files) a set of functions called *Resource manager* -was implemented. It contains a factory which creates basic actions (e.g., `fetchResource`, -`addResource`, `updateResource`, `removeResource`, `fetchMany`) and handlers for -all of the lifecycle actions created by both the API middleware and the promise middleware -which can be used to create a basic reducer. - -The *resource manager* is spread over several files in the `src/redux/helpers/resourceManager` -directory and is covered with unit tests in scripts located at -`test/redux/helpers/resourceManager`. - -### Servier-side Rendering - -To speed-up the initial time of rendering of the web application a technique called server-side -rendering (SSR) is used. The same code which is executed in the web browser of the client can run -on the server using [Node.js](https://nodejs.org). React can serialize its HTML output into -a string which can be sent to the client and can be displayed before the (potentially large) -JavaScript source code starts being executed by the browser. The redux store is in fact -just a large JSON tree which can be easily serialized as well. - -If the user is logged in then the access token should be in the cookies of the web browser -and it should be attached to the HTTP request when the user navigates to the ReCodEx -web page. This token is then put into the redux store and so the user is logged in on -the server. - -The whole logic of the SSR is in a single file called `src/server.js`. It contains only -a definition of a simple HTTP server (using the [express](http://expressjs.com/) framework) -and some necessary boilerplate of the routing library. - -All the components which are associated to the matched route can have a class property `loadAsync` -which should contain a function returning a *Promise*. The SRR calls all these functions and delays -the response of the HTTP server until all of the promises are resolved (or some of them fails). +For routine CRUD (Create, Read, Update, Delete) operations which are common to +most of the resources used in the ReCodEx (e.g., groups, users, assignments, +solutions, solution evaluations, source code files) a set of functions called +*Resource manager* was implemented. It contains a factory which creates basic + actions (e.g., `fetchResource`, `addResource`, `updateResource`, + `removeResource`, `fetchMany`) and handlers for all of the lifecycle actions + created by both the API middleware and the promise middleware which can be used + to create a basic reducer. + +The *resource manager* is spread over several files in the +`src/redux/helpers/resourceManager` directory and is covered with unit tests in +scripts located at `test/redux/helpers/resourceManager`. + +### Server-side Rendering + +To speed-up the initial time of rendering of the web application a technique +called server-side rendering (SSR) is used. The same code which is executed in +the web browser of the client can run on the server using +[Node.js](https://nodejs.org). React can serialize its HTML output into a string +which can be sent to the client and can be displayed before the (potentially +large) JavaScript source code starts being executed by the browser. The redux +store is in fact just a large JSON tree which can be easily serialized as well. + +If the user is logged in then the access token should be in the cookies of the +web browser and it should be attached to the HTTP request when the user +navigates to the ReCodEx web page. This token is then put into the redux store +and so the user is logged in on the server. + +The whole logic of the SSR is in a single file called `src/server.js`. It +contains only a definition of a simple HTTP server (using the +[express](http://expressjs.com/) framework) and some necessary boilerplate of +the routing library. + +All the components which are associated to the matched route can have a class +property `loadAsync` which should contain a function returning a *Promise*. The +SRR calls all these functions and delays the response of the HTTP server until +all of the promises are resolved (or some of them fails). ## Communication Protocol @@ -3611,7 +3625,7 @@ Detailed communication inside the ReCodEx system is captured in the following image and described in sections below. Red connections are through ZeroMQ sockets, blue are through WebSockets and green are through HTTP(S). All ZeroMQ messages are sent as multipart with one string (command, option) per part, with -no empty frames (unles explicitly specified otherwise). +no empty frames (unless explicitly specified otherwise). ![Communication schema](https://github.com/ReCodEx/wiki/raw/master/images/Backend_Connections.png) @@ -3667,7 +3681,7 @@ message from broker to worker must be target the socket identity of the worker message frames: - `job_id` -- identifier of current job - `command` -- what is happening now. - - DOWNLOADED -- submission successfuly fetched from fileserver + - DOWNLOADED -- submission successfully fetched from fileserver - FAILED -- something bad happened and job was not executed at all - UPLOADED -- results are uploaded to fileserver - STARTED -- evaluation of tasks started @@ -3726,7 +3740,7 @@ capable to send corresponding credentials with each request. #### Worker Side -Workers comunicate with the file server in both directions -- they download the +Workers communicate with the file server in both directions -- they download the submissions of the student and then upload evaluation results. Internally, worker is using libcurl C library with very similar setup. In both cases it can verify HTTPS certificate (on Linux against system cert list, on Windows against @@ -3777,7 +3791,7 @@ broker and workers. The current architecture prefers the broker to do all the communication so that the workers do not have to know too many network services. Monitor is treated as a somewhat optional part of whole solution, so no special -effort on communication realibility was made. +effort on communication reliability was made. #### Commands from Monitor to Broker: @@ -3972,7 +3986,7 @@ completing. Surely, the list is not complete and may change in time. - Write alternative commandline frontend. A lot of users want to submit their solutions directly from command line. That is newly possible in ReCodEx because of REST API, but no suitable frontend exists yet. There is unfinished - attempt to create one in NodeJS, accessible in ReCodEx organisation on + attempt to create one in NodeJS, accessible in ReCodEx organization on [GitHub](https://github.com/ReCodEx/cli). The goal is to finish this project or create alternative tool. We would like to see one written in Python. - Create mobile frontend. It would be really nice to have mobile application @@ -3991,7 +4005,7 @@ completing. Surely, the list is not complete and may change in time. - Finish .NET sandbox for Windows. We developed initial sandbox for Windows environment, [WrapSharp](https://github.com/ReCodEx/wrapsharp). WrapSharp is only for .NET platform assemblies and most notably for C# programs and cannot - be used generraly for sandboxing on Windows. The goal is to finish + be used generally for sandboxing on Windows. The goal is to finish implementation and do a really detailed security audit of the project. Also, integration to the worker is not fully done yet. - SIS integration. A very nice feature is (semi)automatic creation of groups and