master
Petr Stefan 8 years ago
parent 00ae6eaeb0
commit 72282b24ed

@ -46,15 +46,15 @@ extendable, so other kinds of usage will be possible.
This project have a great starting point -- there is an old grading system used
these days at out university, so its mistakes and weaknesses can be adressed, and
a lot of teachers willing to use the new system. The ideas were collected both
from our personal experience with old system and from requests of some teachers.
from our personal experience with old system and from teachers' requests.
**Common grading system features:**
- creating exercises including textual description, sample inputs and correct
reference outputs (for example "sum all numbers from given file and write the
result to the standard output")
- assign the exercise to a group of users with some additional properties like
deadlines set
- assign the exercise to a group of users with some additional properties set
(deadlines, etc.)
- user interface for interaction with the system, mainly for showing assigned
exercises, uploading solution sources and presenting evaluated results
- safe environment to execute student solutions withing prescribed time and
@ -63,7 +63,7 @@ from our personal experience with old system and from requests of some teachers.
- user management with support of roles (at least two -- _student_ and
_supervisor_)
- administrative interface for manual checking of solutions, overriding
automatically obtained amount of points and view overall statistics about
automatically assigned amount of points and view overall statistics about
users
Schools and educating institutions needs a bit specific setup. System
@ -72,21 +72,20 @@ configuration needs to reflect structure of courses and user hierarchy
with currently used system at the university, but there are some new ones, which
escalated during long production lifetime of old system. From student
perspective there are not many thing to improve, but a lot of them come from
administrator and supervisors. Collected ideas are from meeting with faculty
staff involved in current system.
administrator and supervisors. Collected ideas are mainly from meeting with
faculty staff involved in current system.
**Wanted features for new system:**
- keep it simple as possible
- login through university authentication system
- support of multiple programming environments at once to avoid unacceptable
- support for multiple programming environments at once to avoid unacceptable
workload for administrator and hardware occupation
- localization (both UI and exercises)
- Markdown support for exercise texts
- tagging exercises and search by tags
- comments, comments, comments (exercises, tests, solutions, ...)
- edit student solution and privately resubmit it
- resubmit solution with saving all results
- resubmit solution with saving all (including temporary) results
- mark one student solution as accepted (used for grading this assignment)
- web and command-line submit tool
- SIS integration for fetching personal user data
@ -95,7 +94,15 @@ staff involved in current system.
layer for ordinary configuration cases
- use of modern technologies with state-of-the-art compilers
@todo: some conclusion
The survey shows that the system is used in many different ways, but the core
functionality is the same for all. When the system is ready, it is likely that
new ideas are developed. Thus the system must be designed to be easily
extendable, so everyone can develop his dream feature. This also means, that
widely used programming languages and techniques should be used, so users can
quickly understand the code and make changes.
To find out current state in the field of automatic grading systems, let's do a
short survey at universities, programming contests or online tools.
### Related projects
@ -107,8 +114,8 @@ list has a brief description and some key features mentioned.
#### CodEx
There already is a grading solution at MFF UK, which was inplemented in 2006 by
group of students. Its name is [CodEx - The Code
There already is a grading solution at MFF UK, which was implemented in 2006 by
group of students. Its name is [CodEx -- The Code
Examiner](http://codex.ms.mff.cuni.cz/project/) and it has been used with some
improvements since then. The original plan was to use the system only for basic
programming courses, but there is demand for adapting it for many different
@ -198,28 +205,23 @@ recruiters and also some universities.
### ReCodEx goals
After considering all these facts, it is clear that CodEx cannot be used
anymore. The project is too old to just maintain it and extend for modern
technologies. Thus, it needs to be completely rewritten or another solution
must be found.
From the research above, we set up several goals, which a new system should
have. They mostly reflect drawbacks of current version of CodEx and wishes of
MFF users. No existing tool fits our needs, for example no examined project
provides complex execution/evaluation pipeline to support needs of courses like
Compiler principles. Modifying CodEx is also not an option -- the required
scope of a new solution is too big. To sum up, a new evaluation system has to
be written, with only small parts of reused code from CodEx (for example
judges).
From the survey above it is clear, that none of the existing systems is capable
of all the features collected for the new system. No grading system is designed
to support complicated evaluation pipeline, so this part is unexplored field and
has to be designed with caution. Also, no project is modern and extendable in a
way that it can be used as a base for ReCodEx. After considering all these
facts, it is clear that the new system has to be written from scratch. This
implies, that only subset of all features will be implemented in the first
version, the others following later.
The new project is **ReCodEx -- ReCodEx Code Examiner**. The name should point
to CodEx, previous evaluation solution, but also reflect new approach to solve
issues. **Re** as part of the name means redesigned, rewritten, renewed or
restarted.
Official assignment of the project is available at [web of software project
committee](http://www.ksi.mff.cuni.cz/sw-projekty/zadani/recodex.pdf) (only in
Czech). Most notable features are following:
From the previous research, we set up several goals, which a new system should
have. They mostly reflect drawbacks of current version of CodEx and reasonable
wishes of university users. Most notable features are following:
- modern HTML5 web frontend written in Javascript using a suitable framework
- REST API implemented in PHP, communicating with database, backend and file
@ -231,6 +233,11 @@ Czech). Most notable features are following:
- evaluation procedure configured in YAML file, compound of small tasks
connected into arbitrary oriented acyclic graph
#### Usage design
@todo: describe detailed usage ... grading and this kind of stuff
### Implementation analysis
@todo: GENERALLY... what problems were solved, how they can be solved and what was the final solution

Loading…
Cancel
Save