From a91396ef94b19df1825d13560b00499a51eacf8f Mon Sep 17 00:00:00 2001 From: Petr Stefan Date: Sun, 25 Dec 2016 18:19:27 +0100 Subject: [PATCH] Analysis updated --- Rewritten-docs.md | 110 ++++++++++++++++++++++++++++++++-------------- 1 file changed, 78 insertions(+), 32 deletions(-) diff --git a/Rewritten-docs.md b/Rewritten-docs.md index 353804b..01d39fd 100644 --- a/Rewritten-docs.md +++ b/Rewritten-docs.md @@ -36,11 +36,79 @@ solution. The system is now ready for production testing at our university. Analysis -------- -### Current solution at MFF UK +### Assignment + +The major goal of this project is to create grading application that will be +used for programming classes at the Faculty of Mathematics and Physics, Charles +University. However, the application should have modular design and be easily +extendable, so other kinds of usage will be possible. + +This project have a great starting point -- there is an old grading system used +these days at out university, so its mistakes and weaknesses can be adressed, and +a lot of teachers willing to use the new system. The ideas were collected both +from our personal experience with old system and from requests of some teachers. + +**Common grading system features:** + +- creating exercises including textual description, sample inputs and correct + reference outputs (for example "sum all numbers from given file and write the + result to the standard output") +- assign the exercise to a group of users with some additional properties like + deadlines set +- user interface for interaction with the system, mainly for showing assigned + exercises, uploading solution sources and presenting evaluated results +- safe environment to execute student solutions withing prescribed time and + memory limits and check corectness of outputs +- assigning points to users depending of correctness of his/her solution +- user management with support of roles (at least two -- _student_ and + _supervisor_) +- administrative interface for manual checking of solutions, overriding + automatically obtained amount of points and view overall statistics about + users + +Schools and educating institutions needs a bit specific setup. System +configuration needs to reflect structure of courses and user hierarchy +(_student_, _supervisor_ and _administrator_). These requirement are satisfied +with currently used system at the university, but there are some new ones, which +escalated during long production lifetime of old system. From student +perspective there are not many thing to improve, but a lot of them come from +administrator and supervisors. Collected ideas are from meeting with faculty +staff involved in current system. + +**Wanted features for new system:** + +- keep it simple as possible +- login through university authentication system +- support of multiple programming environments at once to avoid unacceptable + workload for administrator and hardware occupation +- localization (both UI and exercises) +- Markdown support for exercise texts +- tagging exercises and search by tags +- comments, comments, comments (exercises, tests, solutions, ...) +- edit student solution and privately resubmit it +- resubmit solution with saving all results +- mark one student solution as accepted (used for grading this assignment) +- web and command-line submit tool +- SIS integration for fetching personal user data +- plagiarism detection +- advanced low-level evaluation flow configuration with high-level abstraction + layer for ordinary configuration cases +- use of modern technologies with state-of-the-art compilers + +@todo: some conclusion + -The ideas presented above are not completely new. There was a group of -students, who already implemented an evaluation solution for student's -homeworks in 2006. Its name is [CodEx - The Code +### Related projects + +First of all, some code evaluating projects were found and examined. It is not +a complete list of such evaluators, but just a few projects which are used +these days and can be an inspiration for our project. Each project from the +list has a brief description and some key features mentioned. + +#### CodEx + +There already is a grading solution at MFF UK, which was inplemented in 2006 by +group of students. Its name is [CodEx - The Code Examiner](http://codex.ms.mff.cuni.cz/project/) and it has been used with some improvements since then. The original plan was to use the system only for basic programming courses, but there is demand for adapting it for many different @@ -78,34 +146,7 @@ several drawbacks. The main ones are: programs for classes such as Parallel programming or Compiler principles, which have a more difficult evaluation chain than simple compilation/execution/evaluation provided by CodEx. - -Supervisors and current users of CodEx gives us following ideas for -improvements: - -- keep it simple -- localization (both UI and exercises) -- Markdown support for exercise texts -- tagging exercises and search by tags -- comments, comments, comments (exercises, tests, solutions, ...) -- edit student solution and privately resubmit it -- resubmit solution with saving all results -- mark one student solution as accepted (used for grading this assignment) -- command-line submit tool -- SIS integration for personal user data -- plagiarism detection - -After considering all these facts, it is clear that CodEx cannot be used -anymore. The project is too old to just maintain it and extend for modern -technologies. Thus, it needs to be completely rewritten or another solution -must be found. - -### Related projects - -First of all, some code evaluating projects were found and examined. It is not -a complete list of such evaluators, but just a few projects which are used -these days and can be an inspiration for our project. Each project from the -list has a brief description and some key features mentioned. - + #### Progtest [Progtest](https://progtest.fit.cvut.cz/) is private project from FIT ČVUT in @@ -157,6 +198,11 @@ recruiters and also some universities. ### ReCodEx goals +After considering all these facts, it is clear that CodEx cannot be used +anymore. The project is too old to just maintain it and extend for modern +technologies. Thus, it needs to be completely rewritten or another solution +must be found. + From the research above, we set up several goals, which a new system should have. They mostly reflect drawbacks of current version of CodEx and wishes of MFF users. No existing tool fits our needs, for example no examined project