LEdoian
/
recodex-wiki
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Intro structure updated

Browse Source
master
Petr Stefan 8 years ago
parent 4c4599d5de
commit 273161711f
1 changed files with 70 additions and 83 deletions
Show Stats Download Patch File Download Diff File

153
Rewritten-docs.md
Unescape Escape View File

@ -115,7 +115,21 @@ currently used at the university (CodEx), so its flaws and weaknesses can be
addressed. Furthermore, many teachers desire to use and test the new system and addressed. Furthermore, many teachers desire to use and test the new system and
they are willing to consult ideas or problems during development with us. they are willing to consult ideas or problems during development with us.
### Intended usage ## Current system
Currently used grading solution at the Faculty of Mathematics and Physics of
the Charles University in Prague which was implemented in 2006 by a group
of students. It is called [CodEx -- The Code Examiner](http://codex.ms.mff.cuni.cz/project/)
and it has been used with some improvements since then. The original plan was
to use the system only for basic programming courses, but there was a demand
for adapting it for many different subjects.
CodEx is based on dynamic analysis. It features a web-based interface, where
supervisors can assign exercises to their students and the students have a time
window to submit their solutions. Each solution is compiled and run in sandbox
(MO-Eval). The metrics which are checked are: correctness of the output, time
and memory limits. It supports programs written in C, C++, C#, Java, Pascal,
Python and Haskell.
The whole system is intended to help both teachers (supervisors) and students. The whole system is intended to help both teachers (supervisors) and students.
To achieve this, it is crucial to keep in mind the typical usage scenarios of To achieve this, it is crucial to keep in mind the typical usage scenarios of
@ -123,26 +137,7 @@ the system and to try to make these tasks as simple as possible.
The system has a database of users. Each user is assigned a role, which The system has a database of users. Each user is assigned a role, which
corresponds to his/her privileges. There are user groups reflecting the corresponds to his/her privileges. There are user groups reflecting the
structure of lectured courses. Groups can be hierarchically ordered to reflect structure of lectured courses.
additional metadata such as the academic year. For example, a reasonable group
hierarchy could look like this:
```
Summer term 2016
|-- Language C# and .NET platform
|   |-- Labs Monday 10:30
|   `-- Labs Thursday 9:00
|-- Programming I
|   |-- Labs Monday 14:00
...
```
In this example, students are members of the leaf groups and the higher level
nodes are just for keeping related groups together. The structure can be
modified and altered to fit specific needs of the university or any other
organization, even a flat structure is possible. One user can be a member of
multiple groups and have a different role in each of them (a student can attend
labs for several courses while also teaching one).
A database of exercises (algorithmic problems) is another part of the project. A database of exercises (algorithmic problems) is another part of the project.
Each exercise consists of a text describing the problem in multiple language Each exercise consists of a text describing the problem in multiple language
@ -155,12 +150,50 @@ points, a configuration for calculating the score, a maximum number of
submissions, and a list of supported runtime environments (e.g. programming submissions, and a list of supported runtime environments (e.g. programming
languages) including specific time and memory limits for each one. languages) including specific time and memory limits for each one.
Typical use cases for supported user roles are illustrated on following UML Typical use cases for supported user roles are following:
diagram:
- **student**
- join a group
- get assignments in group
- submit solution to assignment
- view solution results
- **supervisor**
- create exercise
- assign exercise to group, modify assignment
- view all results in group
- alter automatic solution grading
- **administrator**
- create groups
- alter user privileges
- check system logs, upgrades and other management
Current system is old, but robust. There were no major security incidents
during its production usage. However, from today's perspective there are
several drawbacks. The main ones are:
![System use case diagram](https://github.com/ReCodEx/wiki/raw/master/images/System_use_case.png) - **web interface** -- The web interface is simple and fully functional. But
rapid development in web technologies opens new horizons of how web interface
can be made.
- **web API** -- CodEx offers a very limited XML API based on outdated
technologies that is not sufficient for users who would like to create custom
interfaces such as a command line tool or mobile application.
- **sandboxing** -- MO-Eval sandbox is based on principle of monitoring system
calls and blocking the bad ones. This can be easily done for single-threaded
applications, but proves difficult with multi-threaded ones. In present day,
parallelism is a very important area of computing, so there is requirement to
test multi-threaded applications too.
- **instances** -- Different ways of CodEx usage scenarios requires separate
instances (Programming I and II, Java, C#, etc.). This configuration is not
user friendly (students have to register in each instance separately) and
burdens administrators with unnecessary work. CodEx architecture does not
allow sharing hardware between instances, which results in an inefficient use
of hardware for evaluation.
- **task extensibility** -- There is a need to test and evaluate complicated
programs for classes such as Parallel programming or Compiler principles,
which have a more difficult evaluation chain than simple
compilation/execution/evaluation provided by CodEx.
#### Exercise evaluation chain ### Exercise evaluation chain
The most important part of the system is evaluation of solutions submitted by The most important part of the system is evaluation of solutions submitted by
students. Concepts of consecutive steps from source code to final results students. Concepts of consecutive steps from source code to final results
@ -203,27 +236,23 @@ from administrators and supervisors. The ideas were gathered mostly from our
personal experience with the system and from meetings with faculty staff personal experience with the system and from meetings with faculty staff
involved with the current system. involved with the current system.
For clear arrangement all the requirements and wishes are presented grouped by In general, CodEx features should be preserved, so only differences are
categories. presented here. For clear arrangement all the requirements and wishes are
presented grouped by categories.
### System features ### System features
System features represents directly accessible functionality to users of the System features represents directly accessible functionality to users of the
system. They describe the evaluation system in general and also university system. They describe the evaluation system in general and also university
addons (mostly administrative features). addons (mostly administrative features).
#### End user requirements #### End user requirements
- users have their own accounts in the system - group hierarchy -- @todo: copy text from
- system users can be members of multiple groups (reflecting courses or labs) [here](https://github.com/ReCodEx/wiki/wiki/Rewritten-docs/87e4bcd39a4fca3eadbb4748e9a3b6ced2bd7150#intended-usage)
- there is a database of exercises; teachers can create exercises including - there is a database of exercises; teachers can create exercises including
textual description, sample inputs and correct reference outputs (for example textual description, sample inputs and correct reference outputs (for example
"sum all numbers from given file and write the result to the standard output") "sum all numbers from given file and write the result to the standard output")
- there is a list of assigned exercises in each group and an interface to submit
a solution; teachers can assign an existing exercise to their class with some
specific properties set (deadlines, etc.)
- users can see a list of submitted solutions for each assignment with
corresponding results
- teachers can specify way of computation grading points which will be awarded - teachers can specify way of computation grading points which will be awarded
to the students depending on the quality of his/her solution for each to the students depending on the quality of his/her solution for each
assignment extra assignment extra
@ -237,7 +266,7 @@ addons (mostly administrative features).
- localization of all texts (UI and exercises) - localization of all texts (UI and exercises)
- Markdown support for creating exercise texts - Markdown support for creating exercise texts
- tagging exercises in database and search by these tags - tagging exercises in database and search by these tags
- comments, comments, comments (exercises, tests, solutions, ...) - comments of exercises, tests and solutions
- plagiarism detection - plagiarism detection
#### Administrative requirements #### Administrative requirements
@ -263,12 +292,12 @@ addons (mostly administrative features).
another tool and perform additional tests another tool and perform additional tests
- use of modern technologies with state-of-the-art compilers - use of modern technologies with state-of-the-art compilers
### Technical details ### Nonfunctional requirements
Technical details are requirements of technical character with no direct mapping Nonfunctional requirements are requirements of technical character with no
to visible parts of system. In ideal word, users should not know about these if direct mapping to visible parts of system. In ideal word, users should not know
they work properly, but would be at least annoyed if these requirements were not about these if they work properly, but would be at least annoyed if these
met. Most notably they are these ones: requirements were not met. Most notably they are these ones:
- user interface of the system accessible on users' computers without - user interface of the system accessible on users' computers without
installation of any kind of additional software installation of any kind of additional software
@ -300,48 +329,6 @@ This is not a complete list of available evaluators, but only a few projects
which are used these days and can be an inspiration for our project. Each which are used these days and can be an inspiration for our project. Each
project from the list has a brief description and some key features mentioned. project from the list has a brief description and some key features mentioned.
### CodEx
Currently used grading solution at the Faculty of Mathematics and Physics of
the Charles University in Prague which was implemented in 2006 by a group
of students. It is called [CodEx -- The Code Examiner](http://codex.ms.mff.cuni.cz/project/)
and it has been used with some improvements since then. The original plan was
to use the system only for basic programming courses, but there was a demand
for adapting it for many different subjects.
CodEx is based on dynamic analysis. It features a web-based interface, where
supervisors can assign exercises to their students and the students have a time
window to submit their solutions. Each solution is compiled and run in sandbox
(MO-Eval). The metrics which are checked are: correctness of the output, time
and memory limits. It supports programs written in C, C++, C#, Java, Pascal,
Python and Haskell.
Current system is old, but robust. There were no major security incidents
during its production usage. However, from today's perspective there are
several drawbacks. The main ones are:
- **web interface** -- The web interface is simple and fully functional. But
rapid development in web technologies opens new horizons of how web interface
can be made.
- **web API** -- CodEx offers a very limited XML API based on outdated
technologies that is not sufficient for users who would like to create custom
interfaces such as a command line tool or mobile application.
- **sandboxing** -- MO-Eval sandbox is based on principle of monitoring system
calls and blocking the bad ones. This can be easily done for single-threaded
applications, but proves difficult with multi-threaded ones. In present day,
parallelism is a very important area of computing, so there is requirement to
test multi-threaded applications too.
- **instances** -- Different ways of CodEx usage scenarios requires separate
instances (Programming I and II, Java, C#, etc.). This configuration is not
user friendly (students have to register in each instance separately) and
burdens administrators with unnecessary work. CodEx architecture does not
allow sharing hardware between instances, which results in an inefficient use
of hardware for evaluation.
- **task extensibility** -- There is a need to test and evaluate complicated
programs for classes such as Parallel programming or Compiler principles,
which have a more difficult evaluation chain than simple
compilation/execution/evaluation provided by CodEx.
### Progtest ### Progtest
[Progtest](https://progtest.fit.cvut.cz/) is private project of [FIT [Progtest](https://progtest.fit.cvut.cz/) is private project of [FIT

Write Preview
Loading…
Cancel
Save