@ -394,21 +394,21 @@ group, how points are assigned for solutions, ...
@todo: merge with next chapter (Solution concept analysis)
@todo: merge with next chapter (Solution concept analysis)
None of the existing systems we came across is capable of all the required
None of the existing projects we came across is capable of all the required
features of the new system. There is no grading system which is designed to
features for the new system. There is no grading system which is designed to
support a complicated evaluation pipeline, so this part is an unexplored field
support a complicated evaluation pipeline, so this unexplored field has to be
and has to be designed with caution. Also, no project is modern and extensible
designed with caution. Also, no project is modern and extensible enough so it
so it could be used as a base for ReCodEx. After considering all these facts,
could be used as a base for the new system. After considering all these facts,
it was clear that a new system has to be written from scratch. This implies,
it is clear that a new system has to be written from scratch. This implies, that
that only a subset of all the features will be implemented in the first version,
only a subset of all the features will be implemented in the first version, the
the other in the following ones.
others coming in the following releases.
Gathered features are categorized based on priorities for the whole system. The
Gathered features are categorized based on priorities for the whole system. The
highest priority has main functionality similar to current CodEx. It is a base
highest priority has main functionality similar to current CodEx. It is a base
line to be useful in production environment, but a new design allows to easily
line to be useful in production environment, but a new design allows to easily
develop further. On top of that, most of ideas from faculty staff belongs to
develop further. On top of that, most of ideas from faculty staff belongs to
second priority bucket, which will be implemented as part of the project. Most
second priority bucket, which will be implemented as part of the project. The
advanced tasks from this category are advanced low-level evaluation
most complicated tasks from this category are advanced low-level evaluation
configuration format, using modern tools, connecting to a university systems and
configuration format, using modern tools, connecting to a university systems and
merging separate system instances into single one. Other tasks are scheduled for
merging separate system instances into single one. Other tasks are scheduled for
next releases after successful project defense. Namely, these are high-level
next releases after successful project defense. Namely, these are high-level
@ -418,30 +418,49 @@ side) and command-line submit tool. Plagiarism detection is not likely to be
part of any release in near future unless someone other makes the engine. The
part of any release in near future unless someone other makes the engine. The
detection problem is too hard to be solved as part of this project.
detection problem is too hard to be solved as part of this project.
We named the project as **ReCodEx -- ReCodEx Code Examiner**. The name should
We named the new project as **ReCodEx -- ReCodEx Code Examiner**. The name
point to the old CodEx, but also reflect the new approach to solve issues.
should point to the old CodEx, but also reflect the new approach to solve
**Re** as part of the name means redesigned, rewritten, renewed, or restarted.
issues. **Re** as part of the name means redesigned, rewritten, renewed, or
restarted.
At this point there is a clear idea how the new system will be used and what are
At this point there is a clear idea how the new system will be used and what are
the major enhancements for future releases. With this in mind, the overall
the major enhancements for future releases. With this in mind, the overall
architecture can be sketched. From the previous research, we set up several
architecture can be sketched. From the previous research, several goals are set
goals, which the new system should have. They mostly reflect drawbacks of the
up for the new project. They mostly reflect drawbacks of the current version of
current version of CodEx and some reasonable wishes of university users. Most
CodEx and some reasonable wishes of university users. Most notable features are
notable features are following:
following:
- modern HTML5 web frontend written in JavaScript using a suitable framework
- modern HTML5 web frontend written in JavaScript using a suitable framework
- REST API implemented in PHP, communicating with database, evaluation backend
- REST API implemented in PHP, communicating with database, evaluation backend
and a file server
and a file server
- evaluation backend implemented as a distributed system on top of a message
- evaluation backend implemented as a distributed system on top of a message
queue framework (ZeroMQ) with master-worker architecture <!-- @todo: WTF is
queue framework (ZeroMQ) with master-worker architecture
worker??? The concept has not been introduced yet! -->
- multi-platform worker supporting Linux and Windows environment (latter
- worker with basic support of the Windows environment (without sandbox, no
without sandbox, no general purpose suitable tool available yet)
general purpose suitable tool available yet)
- evaluation procedure configured in a YAML file, compound of small tasks
- evaluation procedure configured in a YAML file, compound of small tasks
connected into an arbitrary oriented acyclic graph
connected into an arbitrary oriented acyclic graph
The reasons supporting these decisions are explained in the rest of analysis
## Solution concepts analysis
chapter. Also a lot of smaller design choices are mentioned including possible
options, what is picked to implement and why. But first, discuss basic concepts
of the system.
### Basic concepts
The system is designed as a web application. The requirements say, that user
interface must be accessible from students' computers without installation
additional software. This immediately implies that users are connected to the
internet, so it is used as communication medium. Today, there are two main ways
of designing graphical user interface -- native app or web page. Make nice and
multi-platform application with graphical interface is almost impossible because
of large number of different environments. Also, these applications often
requires installation or at least downloading its files (sources or binaries).
However, creating web application is easier, because every personal computer has
a internet browser installed. Also, browsers supports sort of unified and
standardized environment of HTML5 and JavaScript. CodEx is also web application
and everybody seems happy about it. There are other communicating channels every
user have available like email or git, but they are totally inappropriate for
designing user interfaces on top of them.
@todo: what problems were solved on abstract and high levels, how they can be solved and what was the final solution
@todo: what problems were solved on abstract and high levels, how they can be solved and what was the final solution
@ -457,6 +476,7 @@ notable features are following:
- judges, discuss what they possibly can do and what it can be used for (returning for instance 2 numbers instead of 1 and why we return just one)
- judges, discuss what they possibly can do and what it can be used for (returning for instance 2 numbers instead of 1 and why we return just one)
- discuss points assigned to solution, why are there bonus points, explain minimal point threshold
- discuss points assigned to solution, why are there bonus points, explain minimal point threshold
- discuss several ways how points can be assigned to solution, propose basic systems but also general systems which can use outputs from judges or other executed programs, there is need for variables or other concept, explain why
- discuss several ways how points can be assigned to solution, propose basic systems but also general systems which can use outputs from judges or other executed programs, there is need for variables or other concept, explain why
- where is kept the state (MariaDB)
- and many many more general concepts which can be discussed and solved... please append more of them if something comes to your mind... thanks
- and many many more general concepts which can be discussed and solved... please append more of them if something comes to your mind... thanks
@ -520,7 +540,7 @@ is a database of exercises which can be assigned to the groups and the users of
these groups can submit their solutions for these assignments. The frontend will
these groups can submit their solutions for these assignments. The frontend will
initiate evaluation of these solutions by the backend and it will store the
initiate evaluation of these solutions by the backend and it will store the
results afterwards. The results will be visible to authorized users and the
results afterwards. The results will be visible to authorized users and the
results will be awarded with points according to the score given by the Backend
results will be awarded with points according to the score given by the backend
in the evaluation process. The supervisors of the groups can edit the parameters
in the evaluation process. The supervisors of the groups can edit the parameters
of the assignments, review the solutions and the evaluations in detail and award
of the assignments, review the solutions and the evaluations in detail and award
the solutions with bonus points (both positive and negative) and discuss about
the solutions with bonus points (both positive and negative) and discuss about
@ -541,13 +561,13 @@ of the users.
The frontend developed as part of this project is a web application created with
The frontend developed as part of this project is a web application created with
the needs of the Faculty of Mathematics and Physics of the Charles university in
the needs of the Faculty of Mathematics and Physics of the Charles university in
Prague in mind. The users are the students and their teachers, groups
Prague in mind. The users are the students and their teachers, groups correspond
correspond to the different courses, the teachers are the supervisors of these
to the different courses, the teachers are the supervisors of these groups. We
groups. We believe that this model is applicable to the needs of other
believe that this model is applicable to the needs of other universities,
universities, schools, and IT companies, which can use the same system for their
schools, and IT companies, which can use the same system for their needs. It is
needs. It is also possible to develop their own frontend with their own user
also possible to develop their own frontend with their own user management
management system for their specific needs and use the possibilities of the
system for their specific needs and use the possibilities of the backend without
Backend without any changes, as was mentioned in the previous paragraphs.
any changes, as was mentioned in the previous paragraphs.
In the latter parts of the documentation, both of the backend and frontend parts
In the latter parts of the documentation, both of the backend and frontend parts
will be introduced separately and covered in more detail. The communication
will be introduced separately and covered in more detail. The communication
@ -645,7 +665,11 @@ interesting observations will be discussed in this chapter.
### Backend communication
### Backend communication
@todo: what type of communication within backend could be used, mention some frameworks, queue managers, protocols, which was considered
@todo: what type of communication within backend could be used, mention some
frameworks, queue managers, protocols, which was considered - why we don't use
CORBA, RabbitMQ and why is ZeroMQ great
@todo: maybe merge frontend communication into this
Petr Stefan
8 years ago