master
Petr Stefan 8 years ago
parent 00ae6eaeb0
commit 72282b24ed

@ -46,15 +46,15 @@ extendable, so other kinds of usage will be possible.
This project have a great starting point -- there is an old grading system used This project have a great starting point -- there is an old grading system used
these days at out university, so its mistakes and weaknesses can be adressed, and these days at out university, so its mistakes and weaknesses can be adressed, and
a lot of teachers willing to use the new system. The ideas were collected both a lot of teachers willing to use the new system. The ideas were collected both
from our personal experience with old system and from requests of some teachers. from our personal experience with old system and from teachers' requests.
**Common grading system features:** **Common grading system features:**
- creating exercises including textual description, sample inputs and correct - creating exercises including textual description, sample inputs and correct
reference outputs (for example "sum all numbers from given file and write the reference outputs (for example "sum all numbers from given file and write the
result to the standard output") result to the standard output")
- assign the exercise to a group of users with some additional properties like - assign the exercise to a group of users with some additional properties set
deadlines set (deadlines, etc.)
- user interface for interaction with the system, mainly for showing assigned - user interface for interaction with the system, mainly for showing assigned
exercises, uploading solution sources and presenting evaluated results exercises, uploading solution sources and presenting evaluated results
- safe environment to execute student solutions withing prescribed time and - safe environment to execute student solutions withing prescribed time and
@ -63,7 +63,7 @@ from our personal experience with old system and from requests of some teachers.
- user management with support of roles (at least two -- _student_ and - user management with support of roles (at least two -- _student_ and
_supervisor_) _supervisor_)
- administrative interface for manual checking of solutions, overriding - administrative interface for manual checking of solutions, overriding
automatically obtained amount of points and view overall statistics about automatically assigned amount of points and view overall statistics about
users users
Schools and educating institutions needs a bit specific setup. System Schools and educating institutions needs a bit specific setup. System
@ -72,21 +72,20 @@ configuration needs to reflect structure of courses and user hierarchy
with currently used system at the university, but there are some new ones, which with currently used system at the university, but there are some new ones, which
escalated during long production lifetime of old system. From student escalated during long production lifetime of old system. From student
perspective there are not many thing to improve, but a lot of them come from perspective there are not many thing to improve, but a lot of them come from
administrator and supervisors. Collected ideas are from meeting with faculty administrator and supervisors. Collected ideas are mainly from meeting with
staff involved in current system. faculty staff involved in current system.
**Wanted features for new system:** **Wanted features for new system:**
- keep it simple as possible
- login through university authentication system - login through university authentication system
- support of multiple programming environments at once to avoid unacceptable - support for multiple programming environments at once to avoid unacceptable
workload for administrator and hardware occupation workload for administrator and hardware occupation
- localization (both UI and exercises) - localization (both UI and exercises)
- Markdown support for exercise texts - Markdown support for exercise texts
- tagging exercises and search by tags - tagging exercises and search by tags
- comments, comments, comments (exercises, tests, solutions, ...) - comments, comments, comments (exercises, tests, solutions, ...)
- edit student solution and privately resubmit it - edit student solution and privately resubmit it
- resubmit solution with saving all results - resubmit solution with saving all (including temporary) results
- mark one student solution as accepted (used for grading this assignment) - mark one student solution as accepted (used for grading this assignment)
- web and command-line submit tool - web and command-line submit tool
- SIS integration for fetching personal user data - SIS integration for fetching personal user data
@ -95,7 +94,15 @@ staff involved in current system.
layer for ordinary configuration cases layer for ordinary configuration cases
- use of modern technologies with state-of-the-art compilers - use of modern technologies with state-of-the-art compilers
@todo: some conclusion The survey shows that the system is used in many different ways, but the core
functionality is the same for all. When the system is ready, it is likely that
new ideas are developed. Thus the system must be designed to be easily
extendable, so everyone can develop his dream feature. This also means, that
widely used programming languages and techniques should be used, so users can
quickly understand the code and make changes.
To find out current state in the field of automatic grading systems, let's do a
short survey at universities, programming contests or online tools.
### Related projects ### Related projects
@ -107,8 +114,8 @@ list has a brief description and some key features mentioned.
#### CodEx #### CodEx
There already is a grading solution at MFF UK, which was inplemented in 2006 by There already is a grading solution at MFF UK, which was implemented in 2006 by
group of students. Its name is [CodEx - The Code group of students. Its name is [CodEx -- The Code
Examiner](http://codex.ms.mff.cuni.cz/project/) and it has been used with some Examiner](http://codex.ms.mff.cuni.cz/project/) and it has been used with some
improvements since then. The original plan was to use the system only for basic improvements since then. The original plan was to use the system only for basic
programming courses, but there is demand for adapting it for many different programming courses, but there is demand for adapting it for many different
@ -198,28 +205,23 @@ recruiters and also some universities.
### ReCodEx goals ### ReCodEx goals
After considering all these facts, it is clear that CodEx cannot be used From the survey above it is clear, that none of the existing systems is capable
anymore. The project is too old to just maintain it and extend for modern of all the features collected for the new system. No grading system is designed
technologies. Thus, it needs to be completely rewritten or another solution to support complicated evaluation pipeline, so this part is unexplored field and
must be found. has to be designed with caution. Also, no project is modern and extendable in a
way that it can be used as a base for ReCodEx. After considering all these
From the research above, we set up several goals, which a new system should facts, it is clear that the new system has to be written from scratch. This
have. They mostly reflect drawbacks of current version of CodEx and wishes of implies, that only subset of all features will be implemented in the first
MFF users. No existing tool fits our needs, for example no examined project version, the others following later.
provides complex execution/evaluation pipeline to support needs of courses like
Compiler principles. Modifying CodEx is also not an option -- the required
scope of a new solution is too big. To sum up, a new evaluation system has to
be written, with only small parts of reused code from CodEx (for example
judges).
The new project is **ReCodEx -- ReCodEx Code Examiner**. The name should point The new project is **ReCodEx -- ReCodEx Code Examiner**. The name should point
to CodEx, previous evaluation solution, but also reflect new approach to solve to CodEx, previous evaluation solution, but also reflect new approach to solve
issues. **Re** as part of the name means redesigned, rewritten, renewed or issues. **Re** as part of the name means redesigned, rewritten, renewed or
restarted. restarted.
Official assignment of the project is available at [web of software project From the previous research, we set up several goals, which a new system should
committee](http://www.ksi.mff.cuni.cz/sw-projekty/zadani/recodex.pdf) (only in have. They mostly reflect drawbacks of current version of CodEx and reasonable
Czech). Most notable features are following: wishes of university users. Most notable features are following:
- modern HTML5 web frontend written in Javascript using a suitable framework - modern HTML5 web frontend written in Javascript using a suitable framework
- REST API implemented in PHP, communicating with database, backend and file - REST API implemented in PHP, communicating with database, backend and file
@ -231,6 +233,11 @@ Czech). Most notable features are following:
- evaluation procedure configured in YAML file, compound of small tasks - evaluation procedure configured in YAML file, compound of small tasks
connected into arbitrary oriented acyclic graph connected into arbitrary oriented acyclic graph
#### Usage design
@todo: describe detailed usage ... grading and this kind of stuff
### Implementation analysis ### Implementation analysis
@todo: GENERALLY... what problems were solved, how they can be solved and what was the final solution @todo: GENERALLY... what problems were solved, how they can be solved and what was the final solution

Loading…
Cancel
Save