System usage

Rewritten-docs.md

@@ -44,7 +44,7 @@ University. However, the application should be designed in a modular fashion so
 that it can be easily extended to make other ways of using it possible.
 
 The project has a great starting point -- there is an old grading system
-currently used at our university, so its mistakes and weaknesses can be 
+currently used at our university (CodEx), so its mistakes and weaknesses can be
 adressed. Furthermore, many teachers are willing to use and test the new system.
 Following requirements were collected both from our personal experience with
 CodEx and from teachers' requests.
@@ -54,7 +54,10 @@ CodEx and from teachers' requests.
 These are features that are necessary for any system for evaluation of 
 programming homework assignments used in a university programming course. 
 
+<!---
 @todo maybe group the requirements by role (student might want to do XYZ...)
+- it's ok as is (unless requested differently)
+-->
 
 - creating exercises including textual description, sample inputs and correct
   reference outputs (for example "sum all numbers from given file and write the
@@ -84,27 +87,18 @@ but a lot of feature requests came from administrators and supervisors.
 Collected ideas were mostly gathered from meetings with faculty staff involved
 with the current system.
 
-**Drawbacks of CodEx:**
-
-@todo merge this with the CodEx entry in "Related work"
-
-- there has to be a separate installation of the system for some courses (e.g. 
-  Java programming and .NET programming) that do not use the same sandbox as the 
-  majority of courses.
-
 **Requested features for the new system:**
 
 - logging in through a university authentication system (e.g. LDAP)
 - support for multiple programming environments at once to avoid unacceptable
-  workload for administrator and hardware occupation
+  workload for administrator (maintain separate installations for many courses)
-- elimination of the need to maintain separate installations for Java and C# 
+  and high hardware occupation
-  programming courses
 - localization (both UI and exercises)
 - Markdown support for exercise texts
 - tagging exercises and search by tags
 - comments, comments, comments (exercises, tests, solutions, ...)
 - edit student solution and privately resubmit it
-- resubmit solution with saving all (including temporary) results
+- resubmit solution with saving all results (including temporary ones)
 - mark one student solution as accepted (used for grading this assignment)
 - web and command-line submit tool
 - SIS (university information system) integration for fetching personal user 
@@ -115,11 +109,11 @@ with the current system.
 - use of modern technologies with state-of-the-art compilers
 
 The survey shows that the system is used in many different ways, but the core
-functionality is the same for all. When the system is ready, it is likely that
+functionality is the same for all of them. When the system is ready, it is
-new ideas are developed. Thus the system must be designed to be easily
+likely that new ideas are figured out, thus the system must be designed to be
-extendable, so everyone can develop his dream feature. This also means, that
+easily extendable, so everyone can develop his dream feature. This also means,
-widely used programming languages and techniques should be used, so users can
+that widely used programming languages and techniques should be used, so users
-quickly understand the code and make changes.
+can quickly understand the code and make changes.
 
 To find out current state in the field of automatic grading systems, let's do a
 short survey at universities, programming contests or online tools.
@@ -239,9 +233,12 @@ to CodEx, previous evaluation solution, but also reflect new approach to solve
 issues. **Re** as part of the name means redesigned, rewritten, renewed or
 restarted.
 
-From the previous research, we set up several goals, which a new system should
+At this point there is a clear idea how the new system will be used and what are
-have. They mostly reflect drawbacks of current version of CodEx and reasonable
+major enhancements for future releases. With this in mind, the overall
-wishes of university users. Most notable features are following:
+architecture can be sketched. From the previous research, we set up several
+goals, which a new system should have. They mostly reflect drawbacks of current
+version of CodEx and reasonable wishes of university users. Most notable
+features are following:
 
 - modern HTML5 web frontend written in Javascript using a suitable framework
 - REST API implemented in PHP, communicating with database, backend and file
@@ -253,9 +250,88 @@ wishes of university users. Most notable features are following:
 - evaluation procedure configured in YAML file, compound of small tasks
   connected into arbitrary oriented acyclic graph
 
-#### Usage design
+#### Intended usage
+
+Whole system is intended to help both supervisors and students. To achieve this,
+it is crucial to keep in mind typical usage scenarios of the system and try to
+make these typical tasks as simple as possible. To synchronize visions of
+readers, basic concepts are recapitulated.
+
+First of all, the system has database of users. Each user has assigned a role,
+which correspond to his/her privileges. User can be logged in via local
+authentication service or university system. There are groups of users, which
+corresponds to lectured courses. Groups can be hierarchically ordered to reflect
+additional metadata like academic year. For example, reasonable group hierarchy
+is like this:
+
+```
+Summer term 2016
+├── Language C# and .NET platform
+│   ├── Labs Monday 10:30
+│   └── Labs Thursday 9:00
+├── Programming I
+│   ├── Labs Monday 14:00
+ ...
+
+```
 
-@todo: describe detailed usage ... grading and this kind of stuff
+In this example, student users are part of the leaf groups, higher groups are
+just for keeping related groups together. The hierarchy tree can be modified and
+altered to fit specific needs for each organization, even the flat structure is
+possible.
+
+One user can be part of multiple groups and also one group can have multiple
+users. Each user in a group has a role which defines its capabilities.
+Priviledged user can assign a new exercise in his/her group, change assignment
+details, view results of other users and manually change them. Normal user can
+join a group, get list of assigned exercises, view assignment detail, submit
+his/her solution and of course view the results.
+
+Database of exercises (algorithmic problems) is another part of the project.
+Each exercise consists of text in multiple language variants, evaluation
+configuration and set of inputs and reference outputs. Exercises are created by
+instructed priviledged users. Assigning exercise to a group means choose one of
+the exercises in the list and specify additional data. Assignment has a
+deadline, maximum amount of points and configuration for calculating the final
+amount, number of tries and supported runtimes (programming languages) including
+specific time and memory limits for sandboxed tasks.
+
+##### Exercise evaluation chain
+
+The most important part of the application is evaluating exercises for solutions
+submitted by users. For imaginary system architecture _UI_, _API_, _Broker_ and
+_Worker_ this goes as follows.
+
+First thing users have to do is to submit their solutions to _UI_ which provides
+interface to upload files and then submit them. _UI_ sends a request to _API_
+that user wants to evaluate assignment with provided files.
+
+_API_ checks the assignment invariants (deadlines, count of submissions, ...)
+and stores submitted files. The runtime environment is automatically detected
+based on input files and suitable exercise configuration variant is chosen (one
+exercise can have multiple variants, for example C and Java languages). Matching
+exercise configuration is then send to _Broker_ alongside solution source files.
+
+_Broker_ has to find suitable _Worker_ for execution of this particular
+submission. This decission is made based on capabilities of each _Worker_ and
+job requirements. When a match is found, the job is held until the _Worker_ is
+jobless and can receive an evaluation request.
+
+_Worker_ gets evaluation request with source files and job configuration. The
+configuration is parsed into small tasks with simple piece of work. Evaluation
+itself goes in direction of tasks ordering. It is crucial to keep _Worker_
+machine secure and stable, so isolated sandboxed environment is used when
+dealing with unknown source code. When the execution is finished, results are
+uploaded back.
+
+_API_ is notified about finished job from _Broker_. The results are parsed and
+results of important tasks (comparing actual and expected results) saved into
+database. Also, points are calculated depending on solution correctness and
+assignment configuration.
+
+_UI_ then only displays results summary fetched from the _API_. Presented data
+includes overview which part succeeded and which failed (optionally with reason
+like "memory limit exceeded") and amount of awarded points.
 
 
 ### Solution concepts analysis
@@ -1159,4 +1235,7 @@ used.
           - hw-group-id: group1
             chdir: ${EVAL_DIR}
 ```
+<!---
+// vim: set textwidth=80:
+-->