Job config tutorial

master
Petr Stefan 8 years ago
parent 121300e473
commit 94492aac2c

@ -630,3 +630,148 @@ the awarded points will be overridden.
@todo: Describe the comments thread behavior (public/private comments),
who else can see the comments
### Writing job configuration
To run and evaluate an exercise the backend needs to know the steps how to do
that. This is different for each environment (operation system, programming
language, etc.), so each of the environments needs to have separate
configuration.
Backend understands powerful, but quite low level description of simple
connected tasks written in YAML syntax. More about the syntax and general task
overview can be found on [separate
page](https://github.com/ReCodEx/wiki/wiki/Assignments). One of the planned
features was user friendly configuration editor, but due to tight deadline and
team composition it did not make it to the first release. However, writing
configuration in the basic format will be always available and allows you to use
the full expressive power of the system.
This section walks through creation of job configuration for _hello world_
exercise. The goal is to compile file _source.c_ and check if it prints string
`Hello World!` to the standard output.
The problem can be split into several tasks:
- compile _source.c_ into _helloworld_ by `/usr/bin/gcc`
- run _helloworld_ and save standard output into _out.txt_
- fetch predefined output (suppose it is already uploaded to fileserver) with
hash `a0b65939670bc2c010f4d5d6a0b3e4e4590fb92b` to _reference.txt_
- compare _out.txt_ and _reference.txt_ by `/usr/bin/diff`
The absolute path of tools can be obtained from system administrator. However,
`/usr/bin/gcc` is location, where the GCC binary is available almost everywhere,
so location of some tools can be (professionally) guessed.
First, write header of the job to the configuration file.
```{.yml}
submission:
job-id: hello-word-job
language: c
file-collector: http://localhost:9999/exercises
hw-groups:
- group1
```
Basically it means, that the job _hello-world-job_ is for C language and needs
to be run on workers with capabilities of _group1_ group. Reference files are
downloaded from http://localhost:9999/exercises.
Next the tasks have to be constructed under _tasks_ section. In this demo job,
every task depends only on previous one. The first task has input file
_source.c_ (if submitted by user) already available in working directory, so
just call the GCC. Compilation is run in sandbox as any other external program
and should have relaxed time and memory limits. In this scenarion, worker
defaults are used. If compilation fails, whole job is immediately terminated
(_fatal-failure_ bit set). Because _chdir_ and _bound-directories_ options in
sandbox limits section are mostly common for all tasks, they can be set in
worker configuration instead of job configuration (suppose this for following
tasks). For configuration of workers please contact your administrator.
```{.yml}
- task-id: "compilation"
type: "initiation"
priority: 1
fatal-failure: true
cmd:
bin: "/usr/bin/gcc"
args:
- "source.c"
- "-o"
- "helloworld"
sandbox:
name: "isolate"
limits:
- hw-group-id: group1
chdir: ${EVAL_DIR}
bound-directories:
- src: ${SOURCE_DIR}
dst: ${EVAL_DIR}
mode: RW
```
The compiled program is executed with time and memory limit set and standard
output redirected to a file. This task depends on _compilation_ task, because
the program cannot be executed without being compiled first. It is important to
mark this task with _execution_ type, so exceeded limits will be reported in
frontend.
```{.yml}
- task-id: "execution_1"
test-id: "A"
type: "execution"
priority: 2
fatal-failure: false
dependencies:
- compilation
cmd:
bin: "helloworld"
sandbox:
name: "isolate"
stdout: ${EVAL_DIR}/out.txt
limits:
- hw-group-id: group1
time: 0.5
memory: 8192
```
Fetch sample solution from fileserver. Base URL of fileserver is in job header,
so only the name of required file (`sha1sum` in our case) is necessary.
```{.yml}
- task-id: "fetch_solution_1"
test-id: "A"
priority: 3
dependencies:
- execution
cmd:
bin: "fetch"
args:
- "a0b65939670bc2c010f4d5d6a0b3e4e4590fb92b"
- "${SOURCE_DIR}/reference.txt"
```
Comparison of results is quite straightforward. Important is to mark the task as
_evaluation_ type, so the return code represents if the program is correct (0)
or not (other). Worker default limits are used.
```{.yml}
- task-id: "judge_1"
test-id: "A"
type: "evaluation"
priority: 4
dependencies:
- fetch_solution_1
cmd:
bin: "/usr/bin/diff"
args:
- "out.txt"
- "reference.txt"
sandbox:
name: "isolate"
limits:
- hw-group-id: group1
```

Loading…
Cancel
Save