From 462fc20cbd71d40c403b77829870ddbf5c29b626 Mon Sep 17 00:00:00 2001 From: Petr Stefan Date: Tue, 3 Jan 2017 11:41:26 +0100 Subject: [PATCH] Structure of the project improved --- Rewritten-docs.md | 201 ++++++++++++++++++++++------------------------ 1 file changed, 95 insertions(+), 106 deletions(-) diff --git a/Rewritten-docs.md b/Rewritten-docs.md index ee0b6b9..fef14e3 100644 --- a/Rewritten-docs.md +++ b/Rewritten-docs.md @@ -423,67 +423,97 @@ overview which part succeeded and which failed (optionally with reason like ### Structure of the project -The ReCodEx project is divided into two logical parts – the *Backend* -and the *Frontend* – which interact which each other and which cover the -whole area of code examination. Both of these logical parts are -independent of each other in the sense of being installed on separate -machines at different locations and that one of the parts can be -replaced with a different implementation and as long as the communication -protocols are preserved, the system will continue working as expected. - -*Backend* is the part which is responsible solely for the process of -evaluation a solution of an exercise. Each evaluation of a solution is -referred to as a *job*. For each job, the system expects a configuration -document of the job, supplementary files for the exercise (e.g., test -inputs, expected outputs, predefined header files), and the solution of -the exercise (typically source codes created by a student). There might -be some specific requirements for the job, such as a specific runtime -environment, specific version of a compiler or the job must be evaluated -on a processor with a specific number of cores. The backend -infrastructure decides whether it will accept a job or decline it based -on the specified requirements. In case it accepts the job, it will be -placed in a queue and it will be processed as soon as possible. The backend -publishes the progress of processing of the queued jobs and the results -of the evaluations can be queried after the job processing is finished. -The backend produces a log of the evaluation and scores the solution -based on the job configuration document. - -*Frontend* on the other hand is responsible for the communication with -the users and provides them a convenient access to the Backend -infrastructure. The Frontend manages user accounts and gathers them into -units called groups. There is a database of exercises which can be -assigned to the groups and the users of these groups can submit their -solutions for these assignments. The Frontend will initiate evaluation -of these solutions by the Backend and it will store the results -afterwards. The results will be visible to authorized users and the -results will be awarded with points according to the score given by the -Backend in the evaluation process. The supervisors of the groups can -edit the parameters of the assignments, review the solutions and the -evaluations in detail and award the solutions with bonus points (both -positive and negative) and discuss about the solution with the author of -the solution. Some of the users can be entitled to create new exercises -and extend the database of exercises which can be assigned to the groups -later on. - -The Frontend developed as part of this project was created with the -needs of the Faculty of Mathematics and Physics of the Charles -university in Prague in mind. The users are the students and their -teachers, groups correspond to the different courses, the teachers are -the supervisors of these groups. We believe that this model is -applicable to the needs of other universities, schools, and IT -companies, which can use the same system for their needs. It is also -possible to develop their own frontend with their own user management -system for their specific needs and use the possibilities of the Backend -without any changes, as was mentioned in the previous paragraphs. - -In the latter parts of the documentation, both of the Backend and -Frontend parts will be introduced separately and covered in more detail. -The communication protocol between these two logical parts will be -described as well. - -@todo: move "General backend implementation" here - -@todo: move "General frontend implementation" here +There are numerous ways how to divide some sort of system into separated +services, from one single component to many and many single-purpose components. +Having only one big service is not feasible, not scalable enough and mainly it +would be one big blob of code which somehow works and is very complex, so this +is not the way. The quite opposite, having a lot of single-purpose components is +also somehow impractical. It is scalable by default and all services would have +quite simple code but on the other hand communication requirements for such +solution would be insane. So there has to be chosen approach which is somehow in +the middle, that means services have to communicate in manner which will not +bring network down, code basis should be reasonable and the whole system has to +be scalable enough. With this being said there can be discussion over particular +division for ReCodEx system. + +The ReCodEx project is divided into two logical parts – the *backend* and the +*frontend* – which interact which each other and which cover the whole area of +code examination. Both of these logical parts are independent of each other in +the sense of being installed on separate machines at different locations and +that one of the parts can be replaced with a different implementation and as +long as the communication protocols are preserved, the system will continue +working as expected. + +*Backend* is the part which is responsible solely for the process of evaluation +a solution of an exercise. Each evaluation of a solution is referred to as a +*job*. For each job, the system expects a configuration document of the job, +supplementary files for the exercise (e.g., test inputs, expected outputs, +predefined header files), and the solution of the exercise (typically source +codes created by a student). There might be some specific requirements for the +job, such as a specific runtime environment, specific version of a compiler or +the job must be evaluated on a processor with a specific number of cores. The +backend infrastructure decides whether it will accept a job or decline it based +on the specified requirements. In case it accepts the job, it will be placed in +a queue and it will be processed as soon as possible. The backend publishes the +progress of processing of the queued jobs and the results of the evaluations can +be queried after the job processing is finished. The backend produces a log of +the evaluation and scores the solution based on the job configuration document. + +From the scalable point of view there are two necessary components, the one +which will execute jobs and component which will distribute jobs to the +instances of the first one. This ensures scalability in manner of parallel +execution of numerous jobs which is exactly what is needed. Implementation of +these services are called **broker** and **worker**, first one handles +distribution, latter execution. These components should be enough to fulfil all +above said, but for the sake of simplicity and better communication gateways +with frontend two other components were added, **fileserver** and **monitor**. +Fileserver is simple component whose purpose is to store files which are +exchanged between frontend and backend. Monitor is also quite simple service +which is able to serve job progress state from worker to web application. These +two additional services are on the edge of frontend and backend (like gateways) +but logically they are more connected with backend, so it is considered they +belong there. + +*Frontend* on the other hand is responsible for the communication with the users +and provides them a convenient access to the backend infrastructure. The +frontend manages user accounts and gathers them into units called groups. There +is a database of exercises which can be assigned to the groups and the users of +these groups can submit their solutions for these assignments. The frontend will +initiate evaluation of these solutions by the backend and it will store the +results afterwards. The results will be visible to authorized users and the +results will be awarded with points according to the score given by the Backend +in the evaluation process. The supervisors of the groups can edit the parameters +of the assignments, review the solutions and the evaluations in detail and award +the solutions with bonus points (both positive and negative) and discuss about +the solution with the author of the solution. Some of the users can be entitled +to create new exercises and extend the database of exercises which can be +assigned to the groups later on. + +There are two main purposes of frontend -- holding the state of whole system +(database of users, exercises, solutions, points, etc.) and presenting the state +to users through some kind of an user interface (e.g., a web application, mobile +application, or a command-line tool). According to contemporary trends in +development of frontend parts of applications, we decided to split the frontend +in two logical parts -- a server side and a client side. The server side is +responsible for managing the state and the client side gives instructions to the +server side based on the inputs from the user. This decoupling gives us the +ability to create multiple client side tools which may address different needs +of the users. + +The frontend developed as part of this project is a web application created with +the needs of the Faculty of Mathematics and Physics of the Charles university in +Prague in mind. The users are the students and their teachers, groups +correspond to the different courses, the teachers are the supervisors of these +groups. We believe that this model is applicable to the needs of other +universities, schools, and IT companies, which can use the same system for their +needs. It is also possible to develop their own frontend with their own user +management system for their specific needs and use the possibilities of the +Backend without any changes, as was mentioned in the previous paragraphs. + +In the latter parts of the documentation, both of the backend and frontend parts +will be introduced separately and covered in more detail. The communication +protocol between these two logical parts will be described as well. + ### Evaluation unit executed on backend @@ -574,34 +604,7 @@ This discussion is a never ending story which is done through the whole development process. Some of the most important implementation problems or interesting observations will be discussed in this chapter. -### General backend implementation - -There are numerous ways how to divide some sort of system into separated -services, from one single component to many and many single-purpose components. -Having only one big service is not feasible, not scalable enough and mainly it -would be one big blob of code which somehow works and is very complex, so this -is not the way. The quite opposite, having a lot of single-purpose components is -also somehow impractical. It is scalable by default and all services would have -quite simple code but on the other hand communication requirements for such -solution would be insane. So there has to be chosen approach which is somehow in -the middle, that means services have to communicate in manner which will not -bring network down, code basis should be reasonable and the whole system has to -be scalable enough. With this being said there can be discussion over particular -division for ReCodEx system. - -From the scalable point of view there are two necessary components, the one -which will execute jobs and component which will distribute jobs to the -instances of the first one. This ensures scalability in manner of parallel -execution of numerous jobs which is exactly what is needed. Implementation of -these services are called 'broker' and 'worker', first one handles distribution, -latter execution. These components should be enough to fulfil all above said, -but for the sake of simplicity and better communication gateways with frontend -two other components were added, 'fileserver' and 'monitor'. Fileserver is -simple component whose purpose is to store files which are exchanged between -frontend and backend. Monitor is also quite simple service which is able to -serve job progress state from worker to web application. These two additional -services are on the edge of frontend and backend (like gateways) but logically -they are more connected with backend, so it is considered they belong there. +### Backend communication @todo: what type of communication within backend could be used, mention some frameworks, queue managers, protocols, which was considered @@ -760,21 +763,7 @@ reception of last message. The client gets all already received messages at time of connection with no message loss. -### General frontend implementation - -The frontend is the part of the whole system, which is directly available to the user. -It must hold the whole state of the system - the login credentials, users' data, hierarchy -of groups, available exercises, exercises assignments, users' solutions, and much more. -User must be able to access and modify this state through some kind of an user interface -(e.g., a web application, mobile application, or a command-line tool). We want to implement -a web application as part of this project, but we want to make it possible for other -developers to contribute to the platform and to create their specific clients for -their specific needs. - -We decided to split the frontend in two logical parts: a server side and a client side. -The server side is responsible for managing the state and the client side gives instructions -to the server side based on the inputs from the user. This decoupling gives us the ability -to create multiple client side tools which may address different needs of the users. +### Frontend communication The first thing we need to address is the communication protocol of this client-server architecture. There are several options: @@ -807,7 +796,7 @@ used, there are many tools available for development and testing, and it is unde it should be easy for a new developer with some experience in client-side applications to get to know with the ReCodEx API and develop a client application. -#### API server +### API server The API server must handle HTTP requests and manage the state of the application in some kind of a database. It must also be able to communicate with the backend over ZeroMQ. @@ -858,7 +847,7 @@ Nette framework which makes usage of Doctrine 2 very straighforward. @todo: on demand loading of students submission, in-time loading of every other submission, why -#### Web-app +### Web-app @todo: what technologies can be used on client frontend side, why react was used