diff --git a/Rewritten-docs.md b/Rewritten-docs.md index 9466270..85a8563 100644 --- a/Rewritten-docs.md +++ b/Rewritten-docs.md @@ -2571,7 +2571,9 @@ used. -# The Backend +# Implementation + +## The backend The backend is the part which is hidden to the user and which has only one purpose: evaluate user’s solutions of their assignments. @@ -2583,8 +2585,6 @@ one purpose: evaluate user’s solutions of their assignments. @todo: describe how the backend receives the inputs and how it communicates the results -## Components - Whole backend is not just one service/component, it is quite complex system on its own. @todo: describe the inner parts of the Backend (and refer to the Wiki @@ -2594,6 +2594,11 @@ for the technical description of the components) @todo: gets stuff done, single point of failure and center point of ReCodEx universe +@todo: what to mention: + - job scheduling, worker queues + - API notification using curl, authentication using HTTP Basic Auth + - asynchronous resending progress messages + ### Fileserver @todo: stores particular data from frontend and backend, hashing, HTTP API @@ -2601,219 +2606,16 @@ for the technical description of the components) ### Worker @todo: describe a bit of internal structure in general + - two threads + - number of ZeroMQ sockets, using it also for internal communication + - how sandboxes are fitted into worker, unix syscalls, #ifndef + - libcurl for fetchning, why not to use some object binding + - working with local filesystem, directory structure + - hardware groups in detail @todo: describe how jobs are generally executed -### Monitor - -@todo: not necessary component which can be omitted, proxy-like service - -## Backend internal communication - -@todo: internal backend communication, what communicates with what and why - -The Frontend -============ - -The frontend is the part which is visible to the user of ReCodEx and -which holds the state of the system – the user accounts, their roles in -the system, the database of exercises, the assignments of these -exercises to groups of users (i.e., students), and the solutions and -evaluations of them. - -Frontend is split into three parts: - -- the server-side REST API (“API”) which holds the business logic and - keeps the state of the system consistent - -- the relational database (“DB”) which persists the state of the - system - -- the client side application (“client”) which simplifies access to - the API for the common users - -The centerpiece of this architecture is the API. This component receives -requests from the users and from the Backend, validates them and -modifies the state of the system and persists this modified state in the -DB. - -We have created a web application which can communicate with the API -server and present the information received from the server to the user -in a convenient way. The client can be though any application, which can -send HTTP requests and receive the HTTP responses. Users can use general -applications like [cURL](https://github.com/curl/curl/), -[Postman](https://www.getpostman.com/), or create their own specific -client for ReCodEx API. - -Frontend capabilities ---------------------- - -@todo: describe what the frontend is capable of and how it really works, -what are the limitations and how it can be extended - -Terminology ------------ - -This project was created for the needs of a university and this fact is -reflected into the terminology used throughout the Frontend. A list of -important terms’ definitions follows to make the meaning unambiguous. - -### User and user roles - -*User* is a person who uses the application. User is granted access to -the application once he or she creates an account directly through the -API or the web application. There are several types of user accounts -depending on the set of permissions – a so called “role” – they have -been granted. Each user receives only the most basic set of permissions -after he or she creates an account and this role can be changed only by -the administrators of the service: - -- *Student* is the most basic role. Student can become member of a - group and submit his solutions to his assignments. - -- *Supervisor* can be entitled to manage a group of students. - Supervisor can assign exercises to the students who are members of - his groups and review their solutions submitted to - these assignments. - -- *Super-admin* is a user with unlimited rights. This user can perform - any action in the system. - -There are two implicit changes of roles: - -- Once a *student* is added to a group as its supervisor, his role is - upgraded to a *supervisor* role. - -- Once a *supervisor* is removed from the lasts group where he is a - supervisor then his role is downgraded to a *student* role. - -These mechanisms do not prevent a single user being a supervisor of one -group and student of a different group as supervisors’ permissions are -superset of students’ permissions. - -### Login - -*Login* is a set of user’s credentials he must submit to verify he can -be allowed to access the system as a specific user. We distinguish two -types of logins: local and external. - -- *Local login* is user’s email address and a password he chooses - during registration. - -- *External login* is a mapping of a user profile to an account of - some authentication service (e.g., [CAS](https://ldap1.cuni.cz/)). - -### Instance - -*An instance* of ReCodEx is in fact just a set of groups and user -accounts. An instance should correspond to a real entity as a -university, a high-school, an IT company or an HR agency. This approach -enables the system to be shared by multiple independent organizations -without interfering with each other. - -Usage of the system by the users of an instance can be limited by -possessing a valid licence. It is up to the administrators of the system -to determine the conditions under which they will assign licences to the -instances. - -### Group - -*Group* corresponds to a school class or some other unit which gathers -users who will be assigned the same set exercises. Each group can have -multiple supervisors who can manage the students and the list of -assignments. - -Groups can form a tree hierarchy of arbitrary depth. This is inspired by the -hierarchy of school classes belonging to the same subject over several school -years. For example, there can be a top level group for a programming class that -contains subgroups for every school year. These groups can then by divided into -actual student groups with respect to lab attendance. Supervisors can create -subgroups of their groups and further manage these subgroups. - -### Exercise - -*An exercise* consists of textual assignment of a task and a definition -of how a solution to this exercise should be processed and evaluated in -a specific runtime environment (i.e., how to compile a submitted source -code and how to test the correctness of the program). It is a template -which can be instantiated as an *assignment* by a supervisor of a group. - -### Assignment - -An assignment is an instance of an *exercise* assigned to a specific -*group*. An assignment can modify the text of the task assignment and it -has some additional information which is specific to the group (e.g., a -deadline, the number of points gained for a correct solution, additional -hints for the students in the assignment). The text of the assignment -can be edited and supervisors can translate the assignment into another -language. - -### Solution - -*A solution* is a set of files which a user submits to a given -*assignment*. - -### Submission - -*A submission* corresponds to a *solution* being evaluated by the -Backend. A single *solution* can be submitted repeatedly (e.g., when the -Backend encounters an error or when the supervisor changes the assignment). - -### Evaluation - -*An evaluation* is the processed report received from the Backend after -a *submission* is processed. Evaluation contains points given to the -user based on the quality of his solution measured by the Backend and -the settings of the assignment. Supervisors can review the evaluation -and add bonus points (both positive and negative) if the student -deserves some. - -### Runtime environment - -*A runtime environment* defines the used programming language or tools -which are needed to process and evaluate a solution. Examples of a -runtime environment can be: - -- *Linux + GCC* -- *Linux + Mono* -- *Windows + .NET 4* -- *Bison + Yacc* - -### Limits - -A correct *solution* of an *assignment* has to pass all specified tests (mostly -checks that it yields the correct output for various inputs) and typically must -also be effective in some sense. The Backend measures the time and memory -consumption of the solution while running. This consumption of resources can be -*limited* and the solution will receive fewer points if it exceeds the given -limits in some test cases defined by the *exercise*. - -User management ---------------- - -@todo: roles and their rights, adding/removing different users, how the -role of a specific user changes - -Instances and hierarchy of groups ---------------------------------- - -@todo: What is an instance, how to create one, what are the licences and -how do they work. Why can the groups form hierarchies and what are the -benefits – what it means to be an admin of a group, hierarchy of roles -in the group hierarchy. - -Exercises database ------------------- - -@todo: How the exercises are stored, accessed, who can edit what - -### Creating a new exercise - -@todo Localized assignments, default settings - -### Runtime environments and hardware groups - -@todo read this later and see if it still makes sense +#### Runtime environments ReCodEx is designed to utilize a rather diverse set of workers -- there can be differences in many aspects, such as the actual hardware running the worker @@ -2837,69 +2639,28 @@ However, limits can differ between runtime environments -- formally speaking, limits are a function of three arguments: an assignment, a hardware group and a runtime environment. -### Reference solutions - -@todo: how to add one, how to evaluate it - -The task of determining appropriate resource limits for exercises is difficult -to do correctly. To aid exercise authors and group supervisors, ReCodEx supports -assigning reference solutions to exercises. Those are example programs that -should cover the main approaches to the implementation. For example, searching -for an integer in an ordered array can be done with a linear search, or better, -using a binary search. - -Reference solutions can be evaluated on demand, using a selected hardware group. -The evaluation results are stored and can be used later to determine limits. In -our example problem, we could configure the limits so that the linear -search-based program doesn't finish in time on larger inputs, but a binary -search does. - -Note that separate reference solutions should be supplied for all supported -runtime environments. - -### Exercise assignments - -@todo: Creating instances of an exercise for a specific group of users, -capabilities of settings. Editing limits according to the reference -solution. - -Evaluation process ------------------- - -@todo: How the evaluation process works on the Frontend side. - -### Uploading files and file storage - -@todo: One by one upload endpoint. Explain different types of the -Uploaded files. - -### Automatic detection of the runtime environment - -@todo: Users must submit correctly named files – assuming the RTE from -the extensions. - -REST API implementation ------------------------ - -@todo: What is the REST API, what are the basic principles – GET, POST, -Headers, JSON. +### Monitor -### Authentication and authorization scopes +@todo: not necessary component which can be omitted, proxy-like service -@todo: How authentication works – signed JWT, headers, expiration, -refreshing. Token scopes usage. +### Cleaner -### HTTP requests handling +@todo: if it is something what to say here -@todo: Router and routes with specific HTTP methods, preflight, required -headers +## The frontend -### HTTP responses format +### REST API -@todo: Describe the JSON structure convention of success and error -responses +@todo: what to mention + - basic - GET, POST, JSON, Header, ... + - endpoint structure, Swager UI + - handling requests, preflight, checking roles with annotation + - Uploading files and file storage - one by one upload endpoint. Explain + different types of the Uploaded files. + - Automatic detection of the runtime environment - users must submit + correctly named files, assuming the RTE from the extensions -### Used technologies +#### Used technologies @todo: PHP7 – how it is used for typehints, Nette framework – how it is used for routing, Presenters actions endpoints, exceptions and @@ -2909,7 +2670,7 @@ problem with the extension and how we reported it and how to treat it in the future when the bug is solved. Relational database – we use MariaDB, Doctine enables us to switch the engine to a different engine if needed -### Data model +#### Data model @todo: Describe the code-first approach using the Doctrine entities, how the entities map onto the database schema (refer to the attached schemas @@ -2928,65 +2689,22 @@ grouping of entities and how they are related: @todo: Tell the user about the generated API reference and how the Swagger UI can be used to access the API directly. -Web Application ---------------- - -@todo: What is the purpose of the web application and how it interacts -with the REST API. +### Web application -### Used technologies +@todo: what to mention: + - used libraries, JSX, ... + - usage in user doc + - server side rendering + - maybe more ... -@todo: Briefly introduce the used technologies like React, Redux and the -build process. For further details refer to the GitHub wiki - -### How to use the application - -@todo: Describe the user documentation and the FAQ page. - -Backend-Frontend communication protocol -======================================= +## Communication protocol @todo: describe the exact methods and respective commands for the communication -Initiation of a job evaluation ------------------------------- - @todo: How does the Frontend initiate the evaluation and how the Backend can accept it or decline it -Job processing progress monitoring ----------------------------------- - -When evaluating a job the worker sends progress messages on predefined points of -evaluation chain. The sending place can be on very beginning of the job, when -submit archive is downloaded or at the end of each simple task with its state -(completed, failed, skipped). These messages are sent to broker through existing -ZeroMQ connection. Detailed format of messages can be found on [communication -page](https://github.com/ReCodEx/wiki/wiki/Overall-architecture#commands-from-worker-to-broker). - -Broker only resends received progress messages to the monitor component via -ZeroMQ socket. The output message format is the same as the input format. - -Monitor parses received messages to JSON format, which is easy to work with in -JavaScript inside web application. All messages are cached (one queue per job) -and can be obtained multiple times through WebSocket communication channel. The -cache is cleared 5 minutes after receiving last message. - -Publishing of the results -------------------------- - -After job finish the worker packs results directory into single archive and -uploads it to the fileserver through HTTP protocol. The target URL is obtained -from API in headers on job initiation. Then "job done" notification request is -performed to API via broker. Special submissions (reference or asynchronous -submissions) are loaded immediately, other types are loaded on-demand on first -results request. - -Loading results means fetching archive from fileserver, parsing the main YAML -file generated by worker and saving data to the database. Also, points are -assigned by score calculator. -