From b86567be3f46c7868e0d80cfbcc04fb46233850d Mon Sep 17 00:00:00 2001 From: Petr Stefan Date: Fri, 6 Jan 2017 14:38:04 +0100 Subject: [PATCH] Intro finished --- Rewritten-docs.md | 74 ++++++++++++++++++++++++----------------------- 1 file changed, 38 insertions(+), 36 deletions(-) diff --git a/Rewritten-docs.md b/Rewritten-docs.md index 85cce05..17b84d6 100644 --- a/Rewritten-docs.md +++ b/Rewritten-docs.md @@ -92,7 +92,7 @@ the university. The major goal of this project is to create a grading application that will be used for programming classes at the Faculty of Mathematics and Physics of the Charles University in Prague. However, the application should be designed in a -modular fashion to be easily extended ori even modified to make other ways of +modular fashion to be easily extended or even modified to make other ways of usage possible. The system should be capable of dynamic analysis of programming code. It means, @@ -106,7 +106,7 @@ that following four basic steps have to be supported: The project has a great starting point -- there is an old grading system currently used at the university (CodEx), so its flaws and weaknesses can be addressed. Furthermore, many teachers desire to use and test the new system and -they are willing to consult ideas or problems during development. +they are willing to consult ideas or problems during development with us. ### Intended usage @@ -114,7 +114,7 @@ The whole system is intended to help both teachers (supervisors) and students. To achieve this, it is crucial to keep in mind typical usage scenarios of the system and try to make these tasks as simple as possible. -The system has a database of users. Each user has assigned one role, which +The system has a database of users. Each user has assigned a role, which corresponds to his/her privileges. There are user groups reflecting structure of lectured courses. Groups can be hierarchically ordered to reflect additional metadata such as the academic year. For example, a reasonable group hierarchy @@ -135,8 +135,8 @@ entities are just for keeping the related groups together. The hierarchy structure can be modified and altered to fit specific needs of the university or any other organization, even the flat structure (i.e., no hierarchy) is possible. One user can be part of multiple groups and on the other hand one -group can have multiple users. Each user has a specific role for every group in -which is a member. +group can have multiple users. Each user can have a specific role for every +group in which is a member, overriding his/her default role in this context. Database of exercises (algorithmic problems) is another part of the project. Each exercise consists of a text in multiple language variants, an evaluation @@ -146,7 +146,7 @@ choose one of the available exercises and specifying additional properties. An assignment has a deadline (optionally a second deadline), a maximum amount of points, a configuration for calculating the final score, a maximum number of submissions, and a list of supported runtime environements (e.g., programming -languages) including specific time and memory limits for the sandboxed tasks. +languages) including specific time and memory limits for each one. Typical use cases for supported user roles are illustrated on following UML diagram: @@ -169,9 +169,9 @@ Java languages). Matching exercise configuration is then used for taking care of evaluation process. There is a pool of worker computers dedicated to processing jobs. Some of them -may have different environment to allow testing programs in different -conditions. Incoming jobs are scheduled to particular worker depending on its -capabilities and job requirements. +may have different environment to allow testing programs in more conditions. +Incoming jobs are scheduled to particular worker depending on its capabilities +and job requirements. Job processing itself stars with obtaining source files and job configuration. The configuration is parsed into small tasks with simple piece of work. @@ -183,19 +183,19 @@ are saved. Results from worker contains only output data from processed tasks (this could be return value, consumed time, ...). On top of that, one value is calculated to express overall quality of the tested job. It is used as points for final -student grading. Calculation method of this value may be different for each +student grading. Calculation method of this value may be different for each assignment. Data presented back to users include overview of job parts (which succeeded and which failed, optionally with reason like "memory limit exceeded") and achieved score (amount of awarded points). ## Requirements -There are bunch of different requirements for the system. Some of them are -necessary for any system for source code evaluation. Some of them are specific -for university deployment and some of them arose during the ten year long -lifetime of the old system. There are not many ways to improve CodEx -experience from the perspective of a student, but a lot of feature requests -came from administrators and supervisors. The ideas were gathered mostly our +There are bunch of different formal requirements for the system. Some of them +are necessary for any system for source code evaluation, some of them are +specific for university deployment and some of them arose during the ten year +long lifetime of the old system. There are not many ways to improve CodEx +experience from the perspective of a student, but a lot of feature requests come +from administrators and supervisors. The ideas were gathered mostly from our personal experience with the system and from meetings with faculty staff involved with the current system. @@ -204,12 +204,13 @@ categories. ### System features -System features are requirements directly accessible to users of the system. -They describe the evaluation system in general and also university addons -(mostly administrative features). +System features represents directly accessible functionality to users of the +system. They describe the evaluation system in general and also university +addons (mostly administrative features). #### Pure user requirements +- users have their own account in the system - system users can be members of multiple groups (reflecting courses or labs) - there is a database of exercises; teachers can create exercises including textual description, sample inputs and correct reference outputs (for example @@ -218,8 +219,8 @@ They describe the evaluation system in general and also university addons - there is a list of assigned exercises in each group and interface to submit a solution; teachers can assign an existing exercise to their class with some specific properties set (deadlines, etc.) -- there is a list of submitted solutions for each assignment with corresponding - results +- user can see a list of submitted solutions for each assignment with + corresponding results - teachers can specify way of computation grading points which will be awarted to the students depending on the quality of his/her solution for each assignment extra @@ -227,7 +228,7 @@ They describe the evaluation system in general and also university addons including all submitted solutions; also, each of the solution can be manually reviewed, commented and assigned additional points (positive or negative) - one particular solution can be marked as accepted (used for grading this - assignment) + assignment); otherwise, the solution with most points is used - teacher can edit student solution and privately resubmit it; optionally saving all results (including temporary ones) - localization of all texts (UI and exercises) @@ -241,14 +242,15 @@ They describe the evaluation system in general and also university addons - users can use an intuitive user interface for interaction with the system, mainly for viewing assigned exercises, uploading their own solutions to the assignments, and viewing the results of the solutions after an automatic - evaluation is finished; wanted two interfaces are web and command-line based + evaluation is finished; the two wanted interfaces are web and command-line + based - user privilege separation (at least two roles -- _student_ and _supervisor_) - logging in through a university authentication system (e.g. LDAP) - SIS (university information system) integration for fetching personal user data - safe environment in which the students' solutions are executed - support for multiple programming environments at once to avoid unacceptable - workload for administrator (maintain separate installations for many courses) + workload for administrator (maintain separate installation for every course) and high hardware occupation - advanced low-level evaluation flow configuration with high-level abstraction layer for ordinary configuration cases @@ -266,23 +268,23 @@ met. Most notably they are these ones: - easy implementation of different user interfaces - be ready for workload hundreds of students and tens of supervisors - automated installation of all components - -@todo: fill some nonfunctional requirements; -melo byt lepe popsano, ze je potreba oddelit UI od aplikacni logiky, aby si -nekdo mohl napsat jine UI, ze je potreba mit backend, ktery bude skalovat na -vykon a dovoli pridavani dalsich platforem atd. +- source code with permissive license allowing further development; this also + applies on used libraries and frameworks +- multi-platform worker supporting at least two major operating systems ### Conclusion -The survey shows that the system is used in many different ways, but the core -functionality is the same for all of them. When the system is ready it is likely -that there will be new ideas of how to use the system and thus the system must -be designed to be easily extendable, so everyone can develop their own feature. -This also means that widely used programming languages and techniques should be -used, so users can quickly understand the code and make changes. +The survey shows that there are a lot of different requirements and wishes for +the new system. When the system is ready it is likely that there will be new +ideas of how to use the system and thus the system must be designed to be easily +extendable, so everyone can develop their own feature. This also means that +widely used programming languages and techniques should be used, so users can +quickly understand the code and make changes. To find out the current state in the field of automatic grading systems we did a -short survey at universities, programming contests, and other available tools. +short market survey on the field of automatic grading systems at universities, +programming contests, and possibly other places where similat tools are +available. ## Related work