From 94492aac2c9f90c3646319a625cda8bcf68ff528 Mon Sep 17 00:00:00 2001
From: Petr Stefan <ptr.stef@gmail.com>
Date: Wed, 14 Dec 2016 20:59:48 +0100
Subject: [PATCH] Job config tutorial

---
 Rewritten-docs.md | 145 ++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 145 insertions(+)

diff --git a/Rewritten-docs.md b/Rewritten-docs.md
index 3490a29..171de84 100644
--- a/Rewritten-docs.md
+++ b/Rewritten-docs.md
@@ -630,3 +630,148 @@ the awarded points will be overridden.
 
 @todo: Describe the comments thread behavior (public/private comments),
 who else can see the comments
+
+
+### Writing job configuration
+
+To run and evaluate an exercise the backend needs to know the steps how to do
+that. This is different for each environment (operation system, programming
+language, etc.), so each of the environments needs to have separate
+configuration.
+
+Backend understands powerful, but quite low level description of simple
+connected tasks written in YAML syntax. More about the syntax and general task
+overview can be found on [separate
+page](https://github.com/ReCodEx/wiki/wiki/Assignments).  One of the planned
+features was user friendly configuration editor, but due to tight deadline and
+team composition it did not make it to the first release.  However, writing
+configuration in the basic format will be always available and allows you to use
+the full expressive power of the system.
+
+This section walks through creation of job configuration for _hello world_
+exercise. The goal is to compile file _source.c_ and check if it prints string
+`Hello World!` to the standard output.
+
+The problem can be split into several tasks:
+
+- compile _source.c_ into _helloworld_ by `/usr/bin/gcc`
+- run _helloworld_ and save standard output into _out.txt_
+- fetch predefined output (suppose it is already uploaded to fileserver) with
+  hash `a0b65939670bc2c010f4d5d6a0b3e4e4590fb92b` to _reference.txt_
+- compare _out.txt_ and _reference.txt_ by `/usr/bin/diff`
+
+The absolute path of tools can be obtained from system administrator. However,
+`/usr/bin/gcc` is location, where the GCC binary is available almost everywhere,
+so location of some tools can be (professionally) guessed.
+
+First, write header of the job to the configuration file. 
+
+```{.yml}
+submission:
+    job-id: hello-word-job
+    language: c
+    file-collector: http://localhost:9999/exercises
+    hw-groups:
+        - group1
+```
+
+Basically it means, that the job _hello-world-job_ is for C language and needs
+to be run on workers with capabilities of _group1_ group. Reference files are
+downloaded from http://localhost:9999/exercises.
+
+Next the tasks have to be constructed under _tasks_ section. In this demo job,
+every task depends only on previous one. The first task has input file
+_source.c_ (if submitted by user) already available in working directory, so
+just call the GCC. Compilation is run in sandbox as any other external program
+and should have relaxed time and memory limits. In this scenarion, worker
+defaults are used. If compilation fails, whole job is immediately terminated
+(_fatal-failure_ bit set). Because _chdir_ and _bound-directories_ options in
+sandbox limits section are mostly common for all tasks, they can be set in
+worker configuration instead of job configuration (suppose this for following
+tasks). For configuration of workers please contact your administrator.
+
+```{.yml}
+- task-id: "compilation"
+  type: "initiation"
+  priority: 1
+  fatal-failure: true
+  cmd:
+      bin: "/usr/bin/gcc"
+      args:
+          - "source.c"
+          - "-o"
+          - "helloworld"
+  sandbox:
+      name: "isolate"
+      limits:
+          - hw-group-id: group1
+            chdir: ${EVAL_DIR}
+            bound-directories:
+                - src: ${SOURCE_DIR}
+                  dst: ${EVAL_DIR}
+                  mode: RW
+```
+
+The compiled program is executed with time and memory limit set and standard
+output redirected to a file. This task depends on _compilation_ task, because
+the program cannot be executed without being compiled first. It is important to
+mark this task with _execution_ type, so exceeded limits will be reported in
+frontend.
+
+```{.yml}
+- task-id: "execution_1"
+  test-id: "A"
+  type: "execution"
+  priority: 2
+  fatal-failure: false
+  dependencies:
+      - compilation
+  cmd:
+      bin: "helloworld"
+  sandbox:
+      name: "isolate"
+	  stdout: ${EVAL_DIR}/out.txt
+      limits:
+          - hw-group-id: group1
+            time: 0.5
+            memory: 8192
+```
+
+Fetch sample solution from fileserver. Base URL of fileserver is in job header,
+so only the name of required file (`sha1sum` in our case) is necessary.
+
+```{.yml}
+- task-id: "fetch_solution_1"
+  test-id: "A"
+  priority: 3
+  dependencies:
+      - execution
+  cmd:
+      bin: "fetch"
+      args:
+          - "a0b65939670bc2c010f4d5d6a0b3e4e4590fb92b"
+          - "${SOURCE_DIR}/reference.txt"
+```
+
+Comparison of results is quite straightforward. Important is to mark the task as
+_evaluation_ type, so the return code represents if the program is correct (0)
+or not (other). Worker default limits are used.
+
+```{.yml}
+- task-id: "judge_1"
+  test-id: "A"
+  type: "evaluation"
+  priority: 4
+  dependencies:
+      - fetch_solution_1
+  cmd:
+      bin: "/usr/bin/diff"
+      args:
+          - "out.txt"
+          - "reference.txt"
+  sandbox:
+      name: "isolate"
+      limits:
+          - hw-group-id: group1
+```
+