Analysis updated

master
Petr Stefan 8 years ago
parent b25c4f5c6a
commit a91396ef94

@ -36,11 +36,79 @@ solution. The system is now ready for production testing at our university.
Analysis
--------
### Current solution at MFF UK
### Assignment
The major goal of this project is to create grading application that will be
used for programming classes at the Faculty of Mathematics and Physics, Charles
University. However, the application should have modular design and be easily
extendable, so other kinds of usage will be possible.
This project have a great starting point -- there is an old grading system used
these days at out university, so its mistakes and weaknesses can be adressed, and
a lot of teachers willing to use the new system. The ideas were collected both
from our personal experience with old system and from requests of some teachers.
**Common grading system features:**
- creating exercises including textual description, sample inputs and correct
reference outputs (for example "sum all numbers from given file and write the
result to the standard output")
- assign the exercise to a group of users with some additional properties like
deadlines set
- user interface for interaction with the system, mainly for showing assigned
exercises, uploading solution sources and presenting evaluated results
- safe environment to execute student solutions withing prescribed time and
memory limits and check corectness of outputs
- assigning points to users depending of correctness of his/her solution
- user management with support of roles (at least two -- _student_ and
_supervisor_)
- administrative interface for manual checking of solutions, overriding
automatically obtained amount of points and view overall statistics about
users
Schools and educating institutions needs a bit specific setup. System
configuration needs to reflect structure of courses and user hierarchy
(_student_, _supervisor_ and _administrator_). These requirement are satisfied
with currently used system at the university, but there are some new ones, which
escalated during long production lifetime of old system. From student
perspective there are not many thing to improve, but a lot of them come from
administrator and supervisors. Collected ideas are from meeting with faculty
staff involved in current system.
**Wanted features for new system:**
- keep it simple as possible
- login through university authentication system
- support of multiple programming environments at once to avoid unacceptable
workload for administrator and hardware occupation
- localization (both UI and exercises)
- Markdown support for exercise texts
- tagging exercises and search by tags
- comments, comments, comments (exercises, tests, solutions, ...)
- edit student solution and privately resubmit it
- resubmit solution with saving all results
- mark one student solution as accepted (used for grading this assignment)
- web and command-line submit tool
- SIS integration for fetching personal user data
- plagiarism detection
- advanced low-level evaluation flow configuration with high-level abstraction
layer for ordinary configuration cases
- use of modern technologies with state-of-the-art compilers
@todo: some conclusion
The ideas presented above are not completely new. There was a group of
students, who already implemented an evaluation solution for student's
homeworks in 2006. Its name is [CodEx - The Code
### Related projects
First of all, some code evaluating projects were found and examined. It is not
a complete list of such evaluators, but just a few projects which are used
these days and can be an inspiration for our project. Each project from the
list has a brief description and some key features mentioned.
#### CodEx
There already is a grading solution at MFF UK, which was inplemented in 2006 by
group of students. Its name is [CodEx - The Code
Examiner](http://codex.ms.mff.cuni.cz/project/) and it has been used with some
improvements since then. The original plan was to use the system only for basic
programming courses, but there is demand for adapting it for many different
@ -78,34 +146,7 @@ several drawbacks. The main ones are:
programs for classes such as Parallel programming or Compiler principles,
which have a more difficult evaluation chain than simple
compilation/execution/evaluation provided by CodEx.
Supervisors and current users of CodEx gives us following ideas for
improvements:
- keep it simple
- localization (both UI and exercises)
- Markdown support for exercise texts
- tagging exercises and search by tags
- comments, comments, comments (exercises, tests, solutions, ...)
- edit student solution and privately resubmit it
- resubmit solution with saving all results
- mark one student solution as accepted (used for grading this assignment)
- command-line submit tool
- SIS integration for personal user data
- plagiarism detection
After considering all these facts, it is clear that CodEx cannot be used
anymore. The project is too old to just maintain it and extend for modern
technologies. Thus, it needs to be completely rewritten or another solution
must be found.
### Related projects
First of all, some code evaluating projects were found and examined. It is not
a complete list of such evaluators, but just a few projects which are used
these days and can be an inspiration for our project. Each project from the
list has a brief description and some key features mentioned.
#### Progtest
[Progtest](https://progtest.fit.cvut.cz/) is private project from FIT ČVUT in
@ -157,6 +198,11 @@ recruiters and also some universities.
### ReCodEx goals
After considering all these facts, it is clear that CodEx cannot be used
anymore. The project is too old to just maintain it and extend for modern
technologies. Thus, it needs to be completely rewritten or another solution
must be found.
From the research above, we set up several goals, which a new system should
have. They mostly reflect drawbacks of current version of CodEx and wishes of
MFF users. No existing tool fits our needs, for example no examined project

Loading…
Cancel
Save