@ -337,6 +337,70 @@ like "memory limit exceeded") and amount of awarded points.
### Solution concepts analysis
@todo: what problems were solved on abstract and high levels, how they can be solved and what was the final solution
- which problems are they?
#### Structure of the project
@todo: move "General backend implementation" here
@todo: move "General frontend implementation" here
The ReCodEx project is divided into two logical parts – the *Backend*
and the *Frontend*– which interact which each other and which cover the
whole area of code examination. Both of these logical parts are
independent of each other in the sense of being installed on separate
machines on different locations and that one of the parts can be
replaced with different implementation and as long as the communication
protocols are preserved, the system will continue to work as expected.
*Backend* is the part which is responsible solely for the process of
evaluation a solution of an exercise. Each evaluation of a solution is
referred to as a *job*. For each job, the system expects a configuration
document of the job, supplementary files for the exercise (e.g., test
inputs, expected outputs, predefined header files), and the solution of
the exercise (typically source codes created by a student). There might
be some specific requirements for the job, such as a specific runtime
environment, specific version of a compiler or the job must be evaluated
on a processor with a specific number of cores. The backend
infrastructure decides whether it will accept a job or decline it based
on the specified requirements. In case it accepts the job, it will be
placed in a queue and processed as soon as possible. The backend
publishes the progress of processing of the queued jobs and the results
of the evaluations can be queried after the job processing is finished.
The backend produces a log of the evaluation and scores the solution
based on the job configuration document.
*Frontend* on the other hand is responsible for the communication with
the users and provides them a convenient access to the Backend
infrastructure. The Frontend manages user accounts and gathers them into
units called groups. There is a database of exercises which can be
assigned to the groups and the users of these groups can submit their
solutions for these assignments. The Frontend will initiate evaluation
of these solutions by the Backend and it will store the results
afterwards. The results will be visible to authorized users and the
results will be awarded with points according to the score given by the
Backend in the evaluation process. The supervisors of the groups can
edit the parameters of the assignments, review the solutions and the
evaluations in detail and award the solutions with bonus points (both
positive and negative) and discuss about the solution with the author of
the solution. Some of the users can be entitled to create new exercises
and extend the database of exercises which can be assigned to the groups
later on.
The Frontend developed as part of this project was created with the
needs of the Faculty of Mathematics and Physics of the Charles
university in Prague in mind. The users are the students and their
teachers, groups correspond to the different courses, the teachers are
the supervisors of these groups. We believe that this model is
applicable to the needs of other universities, schools, and IT
companies, which can use the same system for their needs. It is also
possible to develop their own frontend with their own user management
system for their specific needs and use the possibilities of the Backend
without any changes, as was mentioned in the previous paragraphs.
In the latter parts of the documentation, both of the Backend and
Frontend parts will be introduced separately and covered in more detail.
The communication protocol between these two logical parts will be
described as well.
#### Evaluation unit executed on backend
@ -351,13 +415,40 @@ like "memory limit exceeded") and amount of awarded points.
### Implementation analysis
Developing project like ReCodEx have to have some discussion over implementation details and how to solve some particular problems properly. This discussion is never ending story which is done through whole development process. Some of the most important implementation problems or interesting observations will be discussed in this chapter.
Developing project like ReCodEx have to have some discussion over implementation
details and how to solve some particular problems properly. This discussion is
never ending story which is done through whole development process. Some of the
most important implementation problems or interesting observations will be
discussed in this chapter.
#### General backend implementation
There are numerous ways how to divide some sort of system into separated services, from one single component to many and many single-purpose components. Having only one big service is not feasible, not scalable enough and mainly it would be one big blob of code which somehow works and is very complex, so this is not the way. The quite opposite, having a lot of single-purpose components is also somehow impractical. It is scalable by default and all services would have quite simple code but on the other hand communication requirements for such solution would be insane. So there has to be chosen approach which is somehow in the middle, that means services have to communicate in manner which will not bring network down, code basis should be reasonable and the whole system has to be scalable enough. With this being said there can be discussion over particular division for ReCodEx system.
From the scalable point of view there are two necessary components, the one which will execute jobs and component which will distribute jobs to the instances of the first one. This ensures scalability in manner of parallel execution of numerous jobs which is exactly what is needed. Implementation of these services are called 'broker' and 'worker', first one handles distribution, latter execution. These components should be enough to fulfil all above said, but for the sake of simplicity and better communication gateways with frontend two other components were added, 'fileserver' and 'monitor'. Fileserver is simple component whose purpose is to store files which are exchanged between frontend and backend. Monitor is also quite simple service which is able to serve job progress state from worker to web application. These two additional services are on the edge of frontend and backend (like gateways) but logically they are more connected with backend, so it is considered they belong there.
There are numerous ways how to divide some sort of system into separated
services, from one single component to many and many single-purpose components.
Having only one big service is not feasible, not scalable enough and mainly it
would be one big blob of code which somehow works and is very complex, so this
is not the way. The quite opposite, having a lot of single-purpose components is
also somehow impractical. It is scalable by default and all services would have
quite simple code but on the other hand communication requirements for such
solution would be insane. So there has to be chosen approach which is somehow in
the middle, that means services have to communicate in manner which will not
bring network down, code basis should be reasonable and the whole system has to
be scalable enough. With this being said there can be discussion over particular
division for ReCodEx system.
From the scalable point of view there are two necessary components, the one
which will execute jobs and component which will distribute jobs to the
instances of the first one. This ensures scalability in manner of parallel
execution of numerous jobs which is exactly what is needed. Implementation of
these services are called 'broker' and 'worker', first one handles distribution,
latter execution. These components should be enough to fulfil all above said,
but for the sake of simplicity and better communication gateways with frontend
two other components were added, 'fileserver' and 'monitor'. Fileserver is
simple component whose purpose is to store files which are exchanged between
frontend and backend. Monitor is also quite simple service which is able to
serve job progress state from worker to web application. These two additional
services are on the edge of frontend and backend (like gateways) but logically
they are more connected with backend, so it is considered they belong there.
@todo: what type of communication within backend could be used, mention some frameworks, queue managers, protocols, which was considered
@ -436,66 +527,6 @@ From the scalable point of view there are two necessary components, the one whic
@todo: please think about more stuff about api and web-app... thanks ;-)
Structure of the project
------------------------
The ReCodEx project is divided into two logical parts – the *Backend*
and the *Frontend*– which interact which each other and which cover the
whole area of code examination. Both of these logical parts are
independent of each other in the sense of being installed on separate
machines on different locations and that one of the parts can be
replaced with different implementation and as long as the communication
protocols are preserved, the system will continue to work as expected.
*Backend* is the part which is responsible solely for the process of
evaluation a solution of an exercise. Each evaluation of a solution is
referred to as a *job*. For each job, the system expects a configuration
document of the job, supplementary files for the exercise (e.g., test
inputs, expected outputs, predefined header files), and the solution of
the exercise (typically source codes created by a student). There might
be some specific requirements for the job, such as a specific runtime
environment, specific version of a compiler or the job must be evaluated
on a processor with a specific number of cores. The backend
infrastructure decides whether it will accept a job or decline it based
on the specified requirements. In case it accepts the job, it will be
placed in a queue and processed as soon as possible. The backend
publishes the progress of processing of the queued jobs and the results
of the evaluations can be queried after the job processing is finished.
The backend produces a log of the evaluation and scores the solution
based on the job configuration document.
*Frontend* on the other hand is responsible for the communication with
the users and provides them a convenient access to the Backend
infrastructure. The Frontend manages user accounts and gathers them into
units called groups. There is a database of exercises which can be
assigned to the groups and the users of these groups can submit their
solutions for these assignments. The Frontend will initiate evaluation
of these solutions by the Backend and it will store the results
afterwards. The results will be visible to authorized users and the
results will be awarded with points according to the score given by the
Backend in the evaluation process. The supervisors of the groups can
edit the parameters of the assignments, review the solutions and the
evaluations in detail and award the solutions with bonus points (both
positive and negative) and discuss about the solution with the author of
the solution. Some of the users can be entitled to create new exercises
and extend the database of exercises which can be assigned to the groups
later on.
The Frontend developed as part of this project was created with the
needs of the Faculty of Mathematics and Physics of the Charles
university in Prague in mind. The users are the students and their
teachers, groups correspond to the different courses, the teachers are
the supervisors of these groups. We believe that this model is
applicable to the needs of other universities, schools, and IT
companies, which can use the same system for their needs. It is also
possible to develop their own frontend with their own user management
system for their specific needs and use the possibilities of the Backend
without any changes, as was mentioned in the previous paragraphs.
In the latter parts of the documentation, both of the Backend and
Frontend parts will be introduced separately and covered in more detail.
The communication protocol between these two logical parts will be
Petr Stefan
8 years ago