System usage

Rewritten-docs.md

@@ -43,10 +43,10 @@ used for programming classes at the Faculty of Mathematics and Physics, Charles
 University. However, the application should be designed in a modular fashion so 
 that it can be easily extended to make other ways of using it possible.
 
-The project has a great starting point -- there is an old grading system 
+The project has a great starting point -- there is an old grading system
-currently used at our university, so its mistakes and weaknesses can be 
+currently used at our university (CodEx), so its mistakes and weaknesses can be
-adressed. Furthermore, many teachers are willing to use and test the new system. 
+adressed. Furthermore, many teachers are willing to use and test the new system.
-Following requirements were collected both from our personal experience with 
+Following requirements were collected both from our personal experience with
 CodEx and from teachers' requests.
 
 **Basic grading system requirements:**
@@ -54,7 +54,10 @@ CodEx and from teachers' requests.
 These are features that are necessary for any system for evaluation of 
 programming homework assignments used in a university programming course. 
 
+<!---
 @todo maybe group the requirements by role (student might want to do XYZ...)
+- it's ok as is (unless requested differently)
+-->
 
 - creating exercises including textual description, sample inputs and correct
   reference outputs (for example "sum all numbers from given file and write the
@@ -78,33 +81,24 @@ way courses are organized at our university -- for example, users have roles
 in the system and students are divided into groups that correspond to lab 
 groups. 
 
-However, further requirements arose during the ten year long lifetime of the old 
+However, further requirements arose during the ten year long lifetime of the old
-system. There are not many ways to improve it from the perspective of a student, 
+system. There are not many ways to improve it from the perspective of a student,
-but a lot of feature requests came from administrators and supervisors. 
+but a lot of feature requests came from administrators and supervisors.
-Collected ideas were mostly gathered from meetings with faculty staff involved 
+Collected ideas were mostly gathered from meetings with faculty staff involved
 with the current system.
 
-**Drawbacks of CodEx:**
-
-@todo merge this with the CodEx entry in "Related work"
-
-- there has to be a separate installation of the system for some courses (e.g. 
-  Java programming and .NET programming) that do not use the same sandbox as the 
-  majority of courses.
-
 **Requested features for the new system:**
 
 - logging in through a university authentication system (e.g. LDAP)
 - support for multiple programming environments at once to avoid unacceptable
-  workload for administrator and hardware occupation
+  workload for administrator (maintain separate installations for many courses)
-- elimination of the need to maintain separate installations for Java and C# 
+  and high hardware occupation
-  programming courses
 - localization (both UI and exercises)
 - Markdown support for exercise texts
 - tagging exercises and search by tags
 - comments, comments, comments (exercises, tests, solutions, ...)
 - edit student solution and privately resubmit it
-- resubmit solution with saving all (including temporary) results
+- resubmit solution with saving all results (including temporary ones)
 - mark one student solution as accepted (used for grading this assignment)
 - web and command-line submit tool
 - SIS (university information system) integration for fetching personal user 
@@ -115,11 +109,11 @@ with the current system.
 - use of modern technologies with state-of-the-art compilers
 
 The survey shows that the system is used in many different ways, but the core
-functionality is the same for all. When the system is ready, it is likely that
+functionality is the same for all of them. When the system is ready, it is
-new ideas are developed. Thus the system must be designed to be easily
+likely that new ideas are figured out, thus the system must be designed to be
-extendable, so everyone can develop his dream feature. This also means, that
+easily extendable, so everyone can develop his dream feature. This also means,
-widely used programming languages and techniques should be used, so users can
+that widely used programming languages and techniques should be used, so users
-quickly understand the code and make changes.
+can quickly understand the code and make changes.
 
 To find out current state in the field of automatic grading systems, let's do a
 short survey at universities, programming contests or online tools.
@@ -239,9 +233,12 @@ to CodEx, previous evaluation solution, but also reflect new approach to solve
 issues. **Re** as part of the name means redesigned, rewritten, renewed or
 restarted.
 
-From the previous research, we set up several goals, which a new system should
+At this point there is a clear idea how the new system will be used and what are
-have. They mostly reflect drawbacks of current version of CodEx and reasonable
+major enhancements for future releases. With this in mind, the overall
-wishes of university users. Most notable features are following:
+architecture can be sketched. From the previous research, we set up several
+goals, which a new system should have. They mostly reflect drawbacks of current
+version of CodEx and reasonable wishes of university users. Most notable
+features are following:
 
 - modern HTML5 web frontend written in Javascript using a suitable framework
 - REST API implemented in PHP, communicating with database, backend and file
@@ -253,9 +250,88 @@ wishes of university users. Most notable features are following:
 - evaluation procedure configured in YAML file, compound of small tasks
   connected into arbitrary oriented acyclic graph
 
-#### Usage design
+#### Intended usage
+
+Whole system is intended to help both supervisors and students. To achieve this,
+it is crucial to keep in mind typical usage scenarios of the system and try to
+make these typical tasks as simple as possible. To synchronize visions of
+readers, basic concepts are recapitulated.
+
+First of all, the system has database of users. Each user has assigned a role,
+which correspond to his/her privileges. User can be logged in via local
+authentication service or university system. There are groups of users, which
+corresponds to lectured courses. Groups can be hierarchically ordered to reflect
+additional metadata like academic year. For example, reasonable group hierarchy
+is like this:
+
+```
+Summer term 2016
+├── Language C# and .NET platform
+│   ├── Labs Monday 10:30
+│   └── Labs Thursday 9:00
+├── Programming I
+│   ├── Labs Monday 14:00
+ ...
+
+```
 
-@todo: describe detailed usage ... grading and this kind of stuff
+In this example, student users are part of the leaf groups, higher groups are
+just for keeping related groups together. The hierarchy tree can be modified and
+altered to fit specific needs for each organization, even the flat structure is
+possible.
+
+One user can be part of multiple groups and also one group can have multiple
+users. Each user in a group has a role which defines its capabilities.
+Priviledged user can assign a new exercise in his/her group, change assignment
+details, view results of other users and manually change them. Normal user can
+join a group, get list of assigned exercises, view assignment detail, submit
+his/her solution and of course view the results.
+
+Database of exercises (algorithmic problems) is another part of the project.
+Each exercise consists of text in multiple language variants, evaluation
+configuration and set of inputs and reference outputs. Exercises are created by
+instructed priviledged users. Assigning exercise to a group means choose one of
+the exercises in the list and specify additional data. Assignment has a
+deadline, maximum amount of points and configuration for calculating the final
+amount, number of tries and supported runtimes (programming languages) including
+specific time and memory limits for sandboxed tasks.
+
+##### Exercise evaluation chain
+
+The most important part of the application is evaluating exercises for solutions
+submitted by users. For imaginary system architecture _UI_, _API_, _Broker_ and
+_Worker_ this goes as follows.
+
+First thing users have to do is to submit their solutions to _UI_ which provides
+interface to upload files and then submit them. _UI_ sends a request to _API_
+that user wants to evaluate assignment with provided files.
+
+_API_ checks the assignment invariants (deadlines, count of submissions, ...)
+and stores submitted files. The runtime environment is automatically detected
+based on input files and suitable exercise configuration variant is chosen (one
+exercise can have multiple variants, for example C and Java languages). Matching
+exercise configuration is then send to _Broker_ alongside solution source files.
+
+_Broker_ has to find suitable _Worker_ for execution of this particular
+submission. This decission is made based on capabilities of each _Worker_ and
+job requirements. When a match is found, the job is held until the _Worker_ is
+jobless and can receive an evaluation request.
+
+_Worker_ gets evaluation request with source files and job configuration. The
+configuration is parsed into small tasks with simple piece of work. Evaluation
+itself goes in direction of tasks ordering. It is crucial to keep _Worker_
+machine secure and stable, so isolated sandboxed environment is used when
+dealing with unknown source code. When the execution is finished, results are
+uploaded back.
+
+_API_ is notified about finished job from _Broker_. The results are parsed and
+results of important tasks (comparing actual and expected results) saved into
+database. Also, points are calculated depending on solution correctness and
+assignment configuration.
+
+_UI_ then only displays results summary fetched from the _API_. Presented data
+includes overview which part succeeded and which failed (optionally with reason
+like "memory limit exceeded") and amount of awarded points.
 
 
 ### Solution concepts analysis
@@ -1159,4 +1235,7 @@ used.
           - hw-group-id: group1
             chdir: ${EVAL_DIR}
 ```
+<!---
+// vim: set textwidth=80:
+-->