Merge branch 'master' of github.com:ReCodEx/GlobalWiki.wiki

.gitignore

@@ -0,0 +1,2 @@
+.vscode
+.markdownlint.json

Rewritten-docs.md

@@ -51,8 +51,7 @@ Notes:
   studenta
 -->
 
-Introduction
+# Introduction
-============
 
 Generally, there are many different ways and opinions on how to teach people
 something new. However, most people agree that a hands-on experience is one of
@@ -65,8 +64,8 @@ University education system is one of the areas where this knowledge can be
 applied. In computer programming, there are several requirements such as the
 code being syntactically correct, efficient and easy to read, maintain and
 extend. Correctness and efficiency can be tested automatically to help teachers
-save time for their research, but checking for bad design, habits and mistakes
+save time for their research, but reviewing bad design, bad coding habits and 
-is really hard to automate and requires manpower.
+logical mistakes is really hard to automate and requires manpower.
 
 Checking programs written by students takes a lot of time and requires a lot of
 mechanical, repetitive work. The first idea of an automatic evaluation system
@@ -75,70 +74,73 @@ which evaluated code in Algol submitted on punch cards. In following years, many
 similar products were written.
 
 There are two basic ways of automatically evaluating code -- statically (check
-the code without running it; safe, but not much precise) or dynamically (run the
+the code without running it; safe, but not very precise) or dynamically (run the
 code on testing inputs with checking the outputs against reference ones; needs
 sandboxing, but provides good real world experience).
 
-This project focuses on the machine-controlled part of source code evaluation.
+<!--
-First, problems of present software at our university were discussed and similar
+  *Simon*: I am not very sure about the formulation of 'our university' - shouldn't
-projects at other educational institutions were examined. With acquired
+  we rather say 'The Charles University in Prague' instead?
-knowledge from such projects in production, we set up goals for the new
+-->
-evaluation system, designed the architecture and implemented a fully operational
-solution. The system is now ready for production testing at our university.
 
+This project focuses on the machine-controlled part of source code evaluation.
+First, the problems of the software used at our university previously were 
+discussed and similar projects at other educational institutions were examined. 
+With acquired knowledge from such projects in production, we set up goals for
+the new evaluation system, designed the architecture and implemented a fully 
+operational solution. The system is now ready for production testing
+at our university.
 
 ## Assignment
 
 The major goal of this project is to create a grading application that will be
-used for programming classes at the Faculty of Mathematics and Physics, Charles
+used for programming classes at the Faculty of Mathematics and Physics of the Charles
-University. However, the application should be designed in a modular fashion so 
+University in Prague. However, the application should be designed in a modular fashion so
-that it can be easily extended to make other ways of using it possible.
+that it can be easily extended or modified to make other ways of using it possible.
 
 The project has a great starting point -- there is an old grading system
-currently used at our university (CodEx), so its mistakes and weaknesses can be
+currently used at the university (CodEx), so its flaws and weaknesses can be
-adressed. Furthermore, many teachers are willing to use and test the new system.
+addressed. Furthermore, many teachers are willing to use and test the new system.
 Following requirements were collected both from our personal experience with
 CodEx and from teachers' requests.
 
-**Basic grading system requirements:**
+### Basic grading system requirements:
 
-These are features that are necessary for any system for evaluation of 
+These are the features which are necessary for any system for evaluation of
-programming homework assignments used in a university programming course. 
+programming coding assignments used in any university programming course:
 
-<!---
+- students can use an intuitive user interface for interaction with the system,
-@todo maybe group the requirements by role (student might want to do XYZ...)
+  mainly for viewing assigned exercises, uploading their own solutions to the assignments,
-- it's ok as is (unless requested differently)
+  and viewing the results of the solutions after an automatic evaluation is finished
--->
+- teachers can create exercises including textual description, sample inputs and correct
-
-- creating exercises including textual description, sample inputs and correct
   reference outputs (for example "sum all numbers from given file and write the
   result to the standard output")
-- assigning the exercise to a group of users with some additional properties set
+- teachers can assigning an existing exercise to their class with some specific
-  (deadlines, etc.)
+  properties set (deadlines, etc.)
-- user interface for interaction with the system, mainly for showing assigned
+- teachers can specify their scale of points which will be awarted to the students
-  exercises, uploading solution sources and presenting evaluated results
+  depending on the correctness of his/her solution (expressed in percentage points)
-- safe environment to execute student solutions withing prescribed time and
+- teachers can view all of the solutions their students submitted and also the
-  memory limits and check corectness of outputs
+  results of the evaluations and they can override the automatically assigned points
-- assigning points to users depending of correctness of his/her solution
+  to the solutions manually
-- user management with support of roles (at least two -- _student_ and
+- teachers can see the statistics of their classes and individual students
+  of these claseese
+- administrators can depend on a safe environment in which the students' solutions
+  will be executed
+- administrators can manage users with support of roles (at least two -- _student_ and
   _supervisor_)
-- administrative interface for manual checking of solutions, overriding
-  automatically assigned amount of points and viewing of overall statistics 
-  about users
 
 CodEx satisfies all these requirements and a few more that originate from the
 way courses are organized at our university -- for example, users have roles
 (_student_, _supervisor_ and _administrator_) that determine their capabilities
-in the system and students are divided into groups that correspond to lab 
+in the system and students are divided into groups that correspond to lab groups.
-groups. 
 
 However, further requirements arose during the ten year long lifetime of the old
 system. There are not many ways to improve it from the perspective of a student,
 but a lot of feature requests came from administrators and supervisors.
-Collected ideas were mostly gathered from meetings with faculty staff involved
+The ideas were mostly gathered from meetings with faculty staff involved
 with the current system.
 
-**Requested features for the new system:**
+### Requested features for the new system:
 
 - logging in through a university authentication system (e.g. LDAP)
 - support for multiple programming environments at once to avoid unacceptable
@@ -150,45 +152,42 @@ with the current system.
 - comments, comments, comments (exercises, tests, solutions, ...)
 - edit student solution and privately resubmit it
 - resubmit solution with saving all results (including temporary ones)
-- mark one student solution as accepted (used for grading this assignment)
+- mark one student's solution as accepted (used for grading this assignment)
-- web and command-line submit tool
+- web and command-line submission tool
-- SIS (university information system) integration for fetching personal user 
+- SIS (university information system) integration for fetching personal user data
-  data
 - plagiarism detection
 - advanced low-level evaluation flow configuration with high-level abstraction
   layer for ordinary configuration cases
 - use of modern technologies with state-of-the-art compilers
 
 The survey shows that the system is used in many different ways, but the core
-functionality is the same for all of them. When the system is ready, it is
+functionality is the same for all of them. When the system is ready it is
-likely that new ideas are figured out, thus the system must be designed to be
+likely that there will be new ideas of how to use the system and thus the system
-easily extendable, so everyone can develop his dream feature. This also means,
+must be designed to be easily extendable, so everyone can develop their own feature.
-that widely used programming languages and techniques should be used, so users
+This also means that widely used programming languages and techniques should be used,
-can quickly understand the code and make changes.
+so users can quickly understand the code and make changes.
-
-To find out current state in the field of automatic grading systems, let's do a
-short survey at universities, programming contests or online tools.
 
+To find out the current state in the field of automatic grading systems we did a
+short survey at universities, programming contests, and other available tools.
 
 ## Related work
 
-First of all, some code evaluating projects were found and examined. It is not
+This is not a complete list of available evaluators, but only a few projects
-a complete list of such evaluators, but just a few projects which are used
+which are used these days and can be an inspiration for our project. Each project from the
-these days and can be an inspiration for our project. Each project from the
 list has a brief description and some key features mentioned.
 
 ### CodEx
 
-There already is a grading solution at MFF UK, which was implemented in 2006 by
+Currently used grading solution at the Faculty of Mathematics and Physics of
-group of students. Its name is [CodEx -- The Code
+the Charles University in Prague which was implemented in 2006 by a group
-Examiner](http://codex.ms.mff.cuni.cz/project/) and it has been used with some
+of students. It is called [CodEx -- The Code Examiner](http://codex.ms.mff.cuni.cz/project/)
-improvements since then. The original plan was to use the system only for basic
+and it has been used with some improvements since then. The original plan was
-programming courses, but there is demand for adapting it for many different
+to use the system only for basic programming courses, but there was a demand
-subjects.
+for adapting it for many different subjects.
 
 CodEx is based on dynamic analysis. It features a web-based interface, where
-supervisors assign exercises to their students and the students have a time
+supervisors can assign exercises to their students and the students have a time
-window to submit the solution. Each solution is compiled and run in sandbox
+window to submit their solutions. Each solution is compiled and run in sandbox
 (MO-Eval). The metrics which are checked are: corectness of the output, time
 and memory limits. It supports programs written in C, C++, C#, Java, Pascal,
 Python and Haskell.
@@ -200,7 +199,7 @@ several drawbacks. The main ones are:
 - **web interface** -- The web interface is simple and fully functional. But
   rapid development in web technologies opens new horizons of how web interface
   can be made.
-- **web api** -- CodEx offers a very limited XML API based on outdated
+- **web API** -- CodEx offers a very limited XML API based on outdated
   technologies that is not sufficient for users who would like to create custom
   interfaces such as a command line tool or mobile application.
 - **sandboxing** -- MO-Eval sandbox is based on principle of monitoring system
@@ -221,17 +220,16 @@ several drawbacks. The main ones are:
 
 ### Progtest
 
-[Progtest](https://progtest.fit.cvut.cz/) is private project from FIT ČVUT in 
+[Progtest](https://progtest.fit.cvut.cz/) is private project of [FIT ČVUT](https://fit.cvut.cz)
-Prague. As far as we know it is used for C/C++, Bash programming and 
+in Prague. As far as we know it is used for C/C++, Bash programming and knowledge-based quizzes.
-knowledge-based quizzes. There are several bonus points and penalties and also a 
+There are several bonus points and penalties and also a few hints what is failing in the submitted
-few hints what is failing in submitted solution. It is very strict on source 
+solution. It is very strict on source code quality, for example `-pedantic` option of GCC,
-code quality, for example `-pedantic` option of GCC, Valgrind for memory leaks 
+Valgrind for memory leaks or array boundaries checks via `mudflap` library.
-or array boundaries checks via `mudflap` library.
 
 ### Codility
 
-[Codility](https://codility.com/) is web based solution primary targeted to
+[Codility](https://codility.com/) is a web based solution primary targeted to
-company recruiters. It is commercial product of SaaS type supporting 16
+company recruiters. It is a commercial product available as a SaaS and it supports 16
 programming languages. The
 [UI](http://1.bp.blogspot.com/-_isqWtuEvvY/U8_SbkUMP-I/AAAAAAAAAL0/Hup_amNYU2s/s1600/cui.png)
 of Codility is [opensource](https://github.com/Codility/cui), the rest of
@@ -242,12 +240,12 @@ captured progress of writing code for each user.
 
 [CMS](http://cms-dev.github.io/index.html) is an opensource distributed system
 for running and organizing programming contests. It is written in Python and
-contain several modules. CMS supports C/C++, Pascal, Python, PHP and Java. 
+contains several modules. CMS supports C/C++, Pascal, Python, PHP, and Java
-PostgreSQL is a single point of failure, all modules heavily depend on database 
+programming languages. PostgreSQL is a single point of failure, all modules
-connection. Task evaluation can be only three step pipeline -- compilation, 
+heavily depend on the database connection. Task evaluation can be only a three
-execution, evaluation. Execution is performed in 
+step pipeline -- compilation, execution, evaluation. Execution is performed in
-[Isolate](https://github.com/ioi/isolate), sandbox written by consultant of our 
+[Isolate](https://github.com/ioi/isolate), sandbox written by the consultant
-project, Mgr. Martin Mareš, Ph.D.
+of our project, Mgr. Martin Mareš, Ph.D.
 
 ### MOE
 
@@ -270,14 +268,13 @@ recruiters and also some universities.
 
 ## ReCodEx goals
 
-From the survey above it is clear, that none of the existing systems is capable
+None of the existing systems we came across is capable of all the required features
-of all the features collected for the new system. No grading system is designed
+of the new system. There is no grading system which is designed to support a complicated
-to support complicated evaluation pipeline, so this part is unexplored field and
+evaluation pipeline, so this part is an unexplored field and has to be designed with caution.
-has to be designed with caution. Also, no project is modern and extendable in a
+Also, no project is modern and extensible so it could be used as a base for ReCodEx.
-way that it can be used as a base for ReCodEx. After considering all these
+After considering all these facts, it was clear that a new system has to be written
-facts, it is clear that the new system has to be written from scratch. This
+from scratch. This implies, that only a subset of all the features will be implemented
-implies, that only subset of all features will be implemented in the first
+in the first version, the other in the following ones.
-version, the others following later.
 
 Gathered features are categorized based on priorities for the whole system. The
 highest priority has main functionality similar to current CodEx. It is a base
@@ -294,41 +291,40 @@ side) and command-line submit tool. Plagiarism detection is not likely to be
 part of any release in near future unless someone other makes the engine. The
 detection problem is too hard to be solved as part of this project.
 
-The new project is **ReCodEx -- ReCodEx Code Examiner**. The name should point
+We named the project as **ReCodEx -- ReCodEx Code Examiner**. The name should point
-to CodEx, previous evaluation solution, but also reflect new approach to solve
+to the old CodEx, but also reflect the new approach to solve issues.
-issues. **Re** as part of the name means redesigned, rewritten, renewed or
+**Re** as part of the name means redesigned, rewritten, renewed, or restarted.
-restarted.
 
-At this point there is a clear idea how the new system will be used and what are
+At this point there is a clear idea how the new system will be used and what are the
 major enhancements for future releases. With this in mind, the overall
 architecture can be sketched. From the previous research, we set up several
-goals, which a new system should have. They mostly reflect drawbacks of current
+goals, which the new system should have. They mostly reflect drawbacks of the current
-version of CodEx and reasonable wishes of university users. Most notable
+version of CodEx and some reasonable wishes of university users. Most notable
 features are following:
 
-- modern HTML5 web frontend written in Javascript using a suitable framework
+- modern HTML5 web frontend written in JavaScript using a suitable framework
-- REST API implemented in PHP, communicating with database, backend and file
+- REST API implemented in PHP, communicating with database, evaluation backend and a file
   server
-- backend is implemented as distributed system on top of message queue framework
+- evaluation backend implemented as a distributed system on top of a message queue framework
   (ZeroMQ) with master-worker architecture
-- worker with basic support of Windows environment (without sandbox, no general
+<!-- @todo: WTF is worker??? The concept has not been introduced yet! -->
+- worker with basic support of the Windows environment (without sandbox, no general
   purpose suitable tool available yet)
-- evaluation procedure configured in YAML file, compound of small tasks
+- evaluation procedure configured in a YAML file, compound of small tasks
-  connected into arbitrary oriented acyclic graph
+  connected into an arbitrary oriented acyclic graph
 
 ### Intended usage
 
-Whole system is intended to help both supervisors and students. To achieve this,
+The whole system is intended to help both teachers (supervisors) and students.
-it is crucial to keep in mind typical usage scenarios of the system and try to
+To achieve this, it is crucial to keep in mind typical usage scenarios of the
-make these typical tasks as simple as possible. To synchronize visions of
+system and try to make these typical tasks as simple as possible.
-readers, basic concepts are recapitulated.
 
-First of all, the system has database of users. Each user has assigned a role,
+The system has a database of users. Each user has a role assigned,
-which correspond to his/her privileges. User can be logged in via local
+which correspond to his/her privileges. User can be logged in via
-authentication service or university system. There are groups of users, which
+email and password or using the university system. There are groups of users, which
-corresponds to lectured courses. Groups can be hierarchically ordered to reflect
+corresponds to the lectured courses. Groups can be hierarchically ordered to reflect
-additional metadata like academic year. For example, reasonable group hierarchy
+additional metadata such as the academic year. For example, a reasonable group hierarchy
-is like this:
+can look like this:
 
 ```
 Summer term 2016
@@ -341,32 +337,34 @@ Summer term 2016
 
 ```
 
-In this example, student users are part of the leaf groups, higher groups are
+In this example, students are members of the leaf groups, the higher level groups
-just for keeping related groups together. The hierarchy tree can be modified and
+are just for keeping the related groups together. The hierarchy tree can be modified and
-altered to fit specific needs for each organization, even the flat structure is
+altered to fit specific needs of the university or any other organization, even the
-possible.
+flat structure (i.e., no hierarchy) is possible.
 
-One user can be part of multiple groups and also one group can have multiple
+One user can be part of multiple groups and also one group can of course have multiple
-users. Each user in a group has a role which defines its capabilities.
+users. Each user in a group has also a specific role for the given group.
-Priviledged user can assign a new exercise in his/her group, change assignment
+Priviledged user (supervisor) can assign a new exercise in his/her group, change assignment
-details, view results of other users and manually change them. Normal user can
+details, view results of other users and manually change them. Normal user (student) can
 join a group, get list of assigned exercises, view assignment detail, submit
-his/her solution and of course view the results.
+his/her solution and view the results of the evaluation.
 
 Database of exercises (algorithmic problems) is another part of the project.
-Each exercise consists of text in multiple language variants, evaluation
+Each exercise consists of a text in multiple language variants, an evaluation
-configuration and set of inputs and reference outputs. Exercises are created by
+configuration and a set of inputs and reference outputs. Exercises are created by
-instructed priviledged users. Assigning exercise to a group means choose one of
+instructed priviledged users. Assigning an exercise to a group means to choose
-the exercises in the list and specify additional data. Assignment has a
+one of the available exercises and specifying additional properties. An assignment
-deadline, maximum amount of points and configuration for calculating the final
+has a deadline (optionally a second deadline), a maximum amount of points,
-amount, number of tries and supported runtimes (programming languages) including
+a configuration for calculating the final score, a maximum number of submissions,
-specific time and memory limits for sandboxed tasks.
+and a list of supported runtime environemnts (e.g., programming languages) including
+specific time and memory limits for the sandboxed tasks.
 
 #### Exercise evaluation chain
 
-The most important part of the application is evaluating exercises for solutions
+The most important part of the system is the evaluation of the solutions
-submitted by users. For imaginary system architecture _UI_, _API_, _Broker_ and
+submitted by the users for their assigned exercises.
-_Worker_ this goes as follows.
+
+~~For imaginary system architecture _UI_, _API_, _Broker_ and _Worker_ this goes as follows.~~
 
 First thing users have to do is to submit their solutions to _UI_ which provides
 interface to upload files and then submit them. _UI_ sends a request to _API_
@@ -400,8 +398,7 @@ includes overview which part succeeded and which failed (optionally with reason
 like "memory limit exceeded") and amount of awarded points.
 
 
-Analysis
+# Analysis
-========
 
 ## Solution concepts analysis
 
@@ -426,9 +423,9 @@ The ReCodEx project is divided into two logical parts – the *Backend*
 and the *Frontend* – which interact which each other and which cover the
 whole area of code examination. Both of these logical parts are
 independent of each other in the sense of being installed on separate
-machines on different locations and that one of the parts can be
+machines at different locations and that one of the parts can be
-replaced with different implementation and as long as the communication
+replaced with a different implementation and as long as the communication
-protocols are preserved, the system will continue to work as expected.
+protocols are preserved, the system will continue working as expected.
 
 *Backend* is the part which is responsible solely for the process of
 evaluation a solution of an exercise. Each evaluation of a solution is
@@ -441,7 +438,7 @@ environment, specific version of a compiler or the job must be evaluated
 on a processor with a specific number of cores. The backend
 infrastructure decides whether it will accept a job or decline it based
 on the specified requirements. In case it accepts the job, it will be
-placed in a queue and processed as soon as possible. The backend
+placed in a queue and it will be processed as soon as possible. The backend
 publishes the progress of processing of the queued jobs and the results
 of the evaluations can be queried after the job processing is finished.
 The backend produces a log of the evaluation and scores the solution
@@ -649,8 +646,7 @@ cleaner completes machine specific caching system.
 
 
 
-The Backend
+# The Backend
-===========
 
 The backend is the part which is hidden to the user and which has only
 one purpose: evaluate user’s solutions of their assignments.
@@ -675,7 +671,7 @@ for the technical description of the components)
 
 ### Fileserver
 
-@todo: stores particular datas from frontend and backend, hashing, HTTP API
+@todo: stores particular data from frontend and backend, hashing, HTTP API
 
 ### Worker
 
@@ -996,7 +992,7 @@ of entities and relational database models), describe the logical
 grouping of entities and how they are related:
 
 -   user + settings + logins + ACL
--   instance + licences + groups + group membership
+-   instance + licenses + groups + group membership
 -   exercise + assignments + localized assignments + runtime
     environments + hardware groups
 -   submission + solution + reference solution + solution evaluation