From e4bb23be7cf2a3e2f5e33d7c76fed128d4d6bd35 Mon Sep 17 00:00:00 2001 From: Simon Rozsival Date: Mon, 12 Dec 2016 12:56:45 +0100 Subject: [PATCH] Created Rewritten docs (markdown) --- Rewritten-docs.md | 593 ++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 593 insertions(+) create mode 100644 Rewritten-docs.md diff --git a/Rewritten-docs.md b/Rewritten-docs.md new file mode 100644 index 0000000..2de4cd3 --- /dev/null +++ b/Rewritten-docs.md @@ -0,0 +1,593 @@ +Introduction +============ + +@todo: Describe who we are and what is the nature of the project. + +Analysis +-------- + +@todo: Describe how the idea of ReCodEx originated and how we came up +with the stuff we implemented. + +Structure of the project +------------------------ + +The ReCodEx project is divided into two logical parts – the *Backend* +and the *Frontend* – which interact which each other and which cover the +whole area of code examination. Both of these logical parts are +independent of each other in the sense of being installed on separate +machines on different locations and that one of the parts can be +replaced with different implementation and as long as the communication +protocols are preserved, the system will continue to work as expected. + +*Backend* is the part which is responsible solely for the process of +evaluation a solution of an exercise. Each evaluation of a solution is +referred to as a *job*. For each job, the system expects a configuration +document of the job, supplementary files for the exercise (e.g., test +inputs, expected outputs, predefined header files), and the solution of +the exercise (typically source codes created by a student). There might +be some specific requirements for the job, such as a specific runtime +environment, specific version of a compiler or the job must be evaluated +on a processor with a specific number of cores. The backend +infrastructure decides whether it will accept a job or decline it based +on the specified requirements. In case it accepts the job, it will be +placed in a queue and processed as soon as possible. The backend +publishes the progress of processing of the queued jobs and the results +of the evaluations can be queried after the job processing is finished. +The backend produces a log of the evaluation and scores the solution +based on the job configuration document. + +*Frontend* on the other hand is responsible for the communication with +the users and provides them a convenient access to the Backend +infrastructure. The Frontend manages user accounts and gathers them into +units called groups. There is a database of exercises which can be +assigned to the groups and the users of these groups can submit their +solutions for these assignments. The Frontend will initiate evaluation +of these solutions by the Backend and it will store the results +afterwards. The results will be visible to authorized users and the +results will be awarded with points according to the score given by the +Backend in the evaluation process. The supervisors of the groups can +edit the parameters of the assignments, review the solutions and the +evaluations in detail and award the solutions with bonus points (both +positive and negative) and discuss about the solution with the author of +the solution. Some of the users can be entitled to create new exercises +and extend the database of exercises which can be assigned to the groups +later on. + +The Frontend developed as part of this project was created with the +needs of the Faculty of Mathematics and Physics of the Charles +university in Prague in mind. The users are the students and their +teachers, groups correspond to the different courses, the teachers are +the supervisors of these groups. We believe that this model is +applicable to the needs of other universities, schools, and IT +companies, which can use the same system for their needs. It is also +possible to develop their own frontend with their own user management +system for their specific needs and use the possibilities of the Backend +without any changes, as was mentioned in the previous paragraphs. + +In the latter parts of the documentation, both of the Backend and +Frontend parts will be introduced separately and covered in more detail. +The communication protocol between these two logical parts will be +described as well. + +The Backend +=========== + +The backend is the part which is hidden to the user and which has only +one purpose: evaluate user’s solutions of their assignments. + +@todo: describe the configuration inputs of the Backend + +@todo: describe the outputs of the Backend + +@todo: describe the inner parts of the Backend (and refer to the Wiki +for the technical description of the components) + +@todo: describe how the backend receives the inputs and how it +communicates the results + +The Frontend +============ + +The frontend is the part which is visible to the user of ReCodEx and +which holds the state of the system – the user accounts, their roles in +the system, the database of exercises, the assignments of these +exercises to groups of users (i.e., students), and the solutions and +evaluations of them. + +Frontend is split into three parts: + +- the server-side REST API (“API”) which holds the business logic and + keeps the state of the system consistent + +- the relational database (“DB”) which persists the state of the + system + +- the client side application (“client”) which simplifies access to + the API for the common users + +The centerpiece of this architecture is the API. This component receives +requests from the users and from the Backend, validates them and +modifies the state of the system and persists this modified state in the +DB. + +We have created a web application which can communicate with the API +server and present the information received from the server to the user +in a convenient way. The client can be though any application, which can +send HTTP requests and receive the HTTP responses. Users can use general +applications like [cURL](https://github.com/curl/curl/), +[Postman](https://www.getpostman.com/), or create their own specific +client for ReCodEx API. + +Frontend capabilities +--------------------- + +@todo: describe what the frontend is capable of and how it really works, +what are the limitations and how it can be extended + +Terminology +----------- + +This project was created for the needs of a university and this fact is +reflected into the terminology used throughout the Frontend. A list of +important terms’ definitions follows to make the meaning unambiguous. + +### User and user roles + +*User* is a person who uses the application. User is granted access to +the application once he or she creates an account directly through the +API or the web application. There are several types of user accounts +depending on the set of permissions – a so called “role” – they have +been granted. Each user receives only the most basic set of permissions +after he or she creates an account and this role can be changed only by +the administrators of the service: + +- *Student* is the most basic role. Student can become member of a + group and submit his solutions to his assignments. + +- *Supervisor* can be entitled to manage a group of students. + Supervisor can assign exercises to the students who are members of + his groups and review their solutions submitted to + these assignments. + +- *Super-admin* is a user with unlimited rights. This user can perform + any action in the system. + +There are two implicit changes of roles: + +- Once a *student* is added to a group as its supervisor, his role is + upgraded to a *supervisor* role. + +- Once a *supervisor* is removed from the lasts group where he is a + supervisor then his role is downgraded to a *student* role. + +These mechanisms do not prevent a single user being a supervisor of one +group and student of a different group as supervisors’ permissions are +superset of students’ permissions. + +### Login + +*Login* is a set of user’s credentials he must submit to verify he can +be allowed to access the system as a specific user. We distinguish two +types of logins: local and external. + +- *Local login* is user’s email address and a password he chooses + during registration. + +- *External login* is a mapping of a user profile to an account of + some authentication service (e.g., [CAS](https://ldap1.cuni.cz/)). + +### Instance + +*An instance* of ReCodEx is in fact just a set of groups and user +accounts. An instance should correspond to a real entity as a +university, a high-school, an IT company or an HR agency. This approach +enables the system to be shared by multiple independent organizations +without interfering with each other. + +Usage of the system by the users of an instance can be limited by +possessing a valid license. It is up to the administrators of the system +to determine the conditions under which they will assign licenses to the +instances. + +### Group + +*Group* corresponds to a school class or some other unit which gathers +users who will be assigned the same set exercises. Each group can have +multiple supervisors who can manage the students and the list of +assignments. + +Groups can form a tree hierarchy. This is inspired by the hierarchy of +school classes belonging to the same subject over several school years. +Supervisors can create subgroups of their groups and further manage +these subgroups. + +### Exercise + +*An exercise* consists of textual assignment of a task and a definition +of how a solution to this exercise should be processed and evaluated in +a specific runtime environment (i.e., how to compile a submitted source +code and how to test the correctness of the program). It is a template +which can be instantiated as an *assignment* by a supervisor of a group. + +### Assignment + +An assignment is an instance of an *exercise* assigned to a specific +*group*. An assignment can modify the text of the task assignment and it +has some additional information which is specific to the group (e.g., a +deadline, the number of points gained for a correct solution, additional +hints for the students in the assignment). The text of the assignment +can be edited and supervisors can translate the assignment into another +language. + +### Solution + +*A solution* is a set of files which a user submits to a given +*assignment*. + +### Submission + +*A submission* corresponds to a *solution* being evaluated by the +Backend. A single *solution* can be submitted repeatedly (i.e., when the +Backend encounters an error). + +### Evaluation + +*An evaluation* is the processed report received from the Backend after +a *submission* is processed. Evaluation contains points given to the +user based on the quality of his solution measured by the Backend and +the settings of the assignment. Supervisors can review the evaluation +and add bonus points (both positive and negative) if the student +deserves some. + +### Runtime environment + +*A runtime environment* defines the used programming language or tools +which are needed to process and evaluate a solution. Examples of a +runtime environment can be “*Linux + GCC*”, “*Linux + Mono*”, “*Windows ++ .NET 4*”, “*Bison + Yacc.”* + +### Limits + +A correct *solution* of an *assignment* must obviously master the +expected behavior and typically must also be effective in some sense. +The Backend measures the time and memory consumption of the solution +while running. This consumption of resources can be *limited* and the +solution will receive fewer points if it exceeds the given limits in +some test cases defined by the *exercise*. + +User management +--------------- + +@todo: roles and their rights, adding/removing different users, how the +role of a specific user changes + +Instances and hierarchy of groups +--------------------------------- + +@todo: What is an instance, how to create one, what are the licenses and +how do they work. Why can the groups form hierarchies and what are the +benefits – what it means to be an admin of a group, hierarchy of roles +in the group hierarchy. + +Exercises database +------------------ + +@todo: How the exercises are stored, accessed, who can edit what + +### Creating a new exercise + +@todo Localized assignments, default settings + +### Runtime environments and hardware groups + +@todo: describe the concept of RE and HG + +### Reference solutions + +@todo: What it is, what is it used for, how to add one, how to evaluate +it + +### Exercise assignments + +@todo: Creating instances of an exercise for a specific group of users, +capabilities of settings. Editing limits according to the reference +solution. + +Evaluation process +------------------ + +@todo: How the evaluation process works on the Frontend side. + +### Uploading files and file storage + +@todo: One by one upload endpoint. Explain different types of the +Uploaded files. + +### Automatic detection of the runtime environment + +@todo: Users must submit correctly named files – assuming the RTE from +the extensions. + +REST API implementation +----------------------- + +@todo: What is the REST API, what are the basic principles – GET, POST, +Headers, JSON. + +### Authentication and authorization scopes + +@todo: How authentication works – signed JWT, headers, expiration, +refreshing. Token scopes usage. + +### HTTP requests handling + +@todo: Router and routes with specific HTTP methods, preflight, required +headers + +### HTTP responses format + +@todo: Describe the JSON structure convention of success and error +responses + +### Used technologies + +@todo: PHP7 – how it is used for typehints, Nette framework – how it is +used for routing, Presenters actions endpoints, exceptions and +ErrorPresenter, Doctrine 2 – database abstraction, entities and +repositories + conventions, Communication over ZMQ – describe the +problem with the extension and how we reported it and how to treat it in +the future when the bug is solved. Relational database – we use MariaDB, +Doctine enables us to switch the engine to a different engine if needed + +### Data model + +@todo: Describe the code-first approach using the Doctrine entities, how +the entities map onto the database schema (refer to the attached schemas +of entities and relational database models), describe the logical +grouping of entities and how they are related: + +- user + settings + logins + ACL + +- instance + licences + groups + group membership + +- exercise + assignments + localized assignments + runtime + environments + hardware groups + +- submission + solution + reference solution + solution evaluation + +- comment threads + comments + +### API endpoints + +@todo: Tell the user about the generated API reference and how the +Swagger UI can be used to access the API directly. + +Web Application +--------------- + +@todo: What is the purpose of the web application and how it interacts +with the REST API. + +### Used technologies + +@todo: Briefly introduce the used technologies like React, Redux and the +build process. For further details refer to the GitHub wiki + +### How to use the application + +@todo: Describe the user documentation and the FAQ page. + +Backend-Frontend communication protocol +======================================= + +@todo: describe the exact methods and respective commands for the +communication + +Initiation of a job evaluation +------------------------------ + +@todo: How does the Frontend initiate the evaluation and how the Backend +can accept it or decline it + +Job processing progress monitoring +---------------------------------- + +@todo: How does the Backend push the progress of evaluation through the +Monitor to the Frontend + +Publishing of the results +------------------------- + +@todo: How does the Backend publish the results and how it notifies the +Frontend; How the Frontend can request the results if the notification +does not arrive + +User documentation +================== + +Web Application +--------------- + +@todo: Describe different scenarios of the usage of the Web App + +### Terminology + +@todo: Describe the terminology: Instance, User, Group, Student, +Supervisor, Admin + +### Web application requirements + +@todo: Describe the requirements of running the web application (modern +web browser, enabled CSS, JavaScript, Cookies & Local storage) + +### Scenario \#1: Becoming a user of ReCodEx + +#### How to create a user account? + +You can create an account if you click on the “*Create account*” menu +item in the left sidebar. You can choose between two types of +registration methods – by creating a local account with a specific +password, or pairing your new account with an existing CAS UK account. + +If you decide a new “*local*” account using the “*Create ReCodEx +account*” form, you will have to provide your details and choose a +password for your account. You will later sign in using your email +address as your username and the password you select. + +If you decide to use the CAS UK, then we will verify your credentials +and access your name and email stored in the system and create your +account based on this information. You can change your personal +information or email later on the “*Settings*” page. + +When crating your account both ways, you must select an instance your +account will belong to by default. The instance you will select will be +most likely your university or other organization you are a member of. + +#### How to get into ReCodEx? + +To log in, go to the homepage of ReCodEx and in the left sidebar choose +the menu item “*Sign in*”. Then you must enter your credentials into one +of the two forms – if you selected a password during registration, then +you should sign with your email and password in the first form called +“*Sign into ReCodEx*”. If you registered using the Charles University +Authentication Service (CAS), you should put your student’s number and +your CAS password into the second form called “Sign into ReCodEx using +CAS UK”. + +#### How do I sign out of ReCodEx? + +If you don’t use ReCodEx for a whole day, you will be logged out +automatically. However, we recommend you sign out of the application +after you finished your interaction with it. The logout button is placed +in the top section of the left sidebar right under your name. You will +have to expand the sidebar with a button next to the “*ReCodEx*” title +(shown in the picture below). + +![](media/image1.png){width="3.3131944444444446in" +height="0.6173611111111111in"} + +#### What to do when you cannot remember your password? + +If you can’t remember your password and you don’t use CAS UK +authentication, then you can reset your password. You will find a link +saying “*You cannot remember what your password was? Reset your +password.*” under the sign in form. After you click on this link, you +will be asked to submit your email address. An email with a link +containing a special token will be sent to the address you fill in. We +make sure that the person who requested password resetting is really +you. When you click on the link (or you copy & paste it into your web +browser) you will be able to select a new password for your account. The +token is valid only for a couple of minutes, so do not forget to reset +the password as soon as possible, or you will have to request a new link +with a valid token. + +If you sign in through CAS UK, then please follow the instructions +provided by the administrators of the service described on their +website. + +#### How to configure your account? + +There are several options you have to edit your user account. + +- changing your personal information (i.e., name) + +- changing your credentials (email and password) + +- updating your preferences (e.g., source code viewer/editor settings, + default language) + +You can access the settings page through the “*Settings*” button right +under your name in the left sidebar. + +### Scenario \#2: User is a student + +@todo: describe what it means to be a “student” and what are the +student’s rights + +#### How to join a group for my class? + +@todo: How to join a specific group + +#### Which assignments do I have to solve? + +@todo: Where the student can find the list of the assignment he is +expected to solve, what is the first and second deadline. + +#### Where can I see details of my classes’ group? + +@todo: Where can the user see groups description and details, what +information is available. + +#### How to submit a solution of an assignment? + +@todo: How does a student submit his solution through the web app + +#### Where are the results of my solutions? + +@todo: When the results are ready and what the results mean and what to +do about them, when the user is convinced, that his solution is correct +although the results say different + +#### How can I discuss my solution with my teacher/group’s supervisor directly through the web application? + +@todo: Describe the comments thread behavior (public/private comments), +who else can see the comments, how notifications work (*not implemented +yet*!). + +### Scenario \#3: User is supervisor of a group + +@todo: describe what it means to be a “supervisor” of a group and what +are the supervisors rights + +#### How do I become a supervisor of a group? + +@todo: How does a user become a supervisor of a group? + +#### How to add or remove a student to my group? + +@todo: How to add a specific student to a given group + +#### How do I add another supervisor to my group? + +@todo: who can add another supervisor, what would be the rights of the +second supervisor + +#### How do I create a subgroup of my group? + +@todo: What it means to create a subgroup and how to do it. + +#### How do I assign an exercise to my students? + +@todo: Describe how to access the database of the exercises and what are +the possibilities of assignment setup – availability, deadlines, points, +score configuration, limits + +#### How do I configure the limits of an assignment and how to choose appropriate limits? + +@todo: Describe the form and explain the concept of reference solutions. +How to evaluate the reference solutions for the exercise right now (to +get the up-to-date information). + +#### How can I assign some exercises only to some students of the group? + +@todo: Describe how to achieve this using subgroups + +#### How can I see my students’ solutions? + +@todo Describe where all the students’ solutions for a given assignment +can be found, where to look for all solutions of a given student, how to +see results of a specific student’s solution’s evaluation result. + +#### Can I assign points to my students’ solutions manually instead of depending on automatic scoring? + +@todo If and how to change the score of a solution – assignment +settings, setting points, bonus points, accepting a solution (*not +implemented yet!*). Describe how the student and supervisor will still +be able to see the percentage received from the automatic scoring, but +the awarded points will be overridden. + +#### How can I discuss student’s solution with him/her directly through the web application? + +@todo: Describe the comments thread behavior (public/private comments), +who else can see the comments