From 7766d52e79a55ed092ff51ea17d16cb1c84c62af Mon Sep 17 00:00:00 2001 From: Martin Polanka Date: Sat, 24 Feb 2018 14:28:10 +0100 Subject: [PATCH] move writing job config to job config separate page --- User-documentation.md | 155 ------------------------------------------ 1 file changed, 155 deletions(-) diff --git a/User-documentation.md b/User-documentation.md index c6e7225..8c05fbb 100644 --- a/User-documentation.md +++ b/User-documentation.md @@ -692,161 +692,6 @@ testWeights: d: 100 ``` - -## Writing Job Configuration - -To run and evaluate an exercise the backend needs to know the steps how to do -that. This is different for each environment (operation system, programming -language, etc.), so each of the environments needs to have separate -configuration. - -Backend works with a powerful, but quite low level description of simple -connected tasks written in YAML syntax. More about the syntax and general task -overview can be found on [separate -page](https://github.com/ReCodEx/wiki/wiki/Assignments). One of the planned -features was user friendly configuration editor, but due to tight deadline and -team composition it did not make it to the first release. However, writing -configuration in the basic format will be always available and allows users to -use the full expressive power of the system. - -This section walks through creation of job configuration for _hello world_ -exercise. The goal is to compile file _source.c_ and check if it prints `Hello -World!` to the standard output. This is the only test case named **A**. - -The problem can be split into several tasks: - -- compile _source.c_ into _helloworld_ with `/usr/bin/gcc` -- run _helloworld_ and save standard output into _out.txt_ -- fetch predefined output (suppose it is already uploaded to fileserver) with - hash `a0b65939670bc2c010f4d5d6a0b3e4e4590fb92b` to _reference.txt_ -- compare _out.txt_ and _reference.txt_ by `/usr/bin/diff` - -The absolute path of tools can be obtained from system administrator. However, -`/usr/bin/gcc` is location, where the GCC binary is available almost everywhere, -so location of some tools can be (professionally) guessed. - -First, write header of the job to the configuration file. - -```{.yml} -submission: - job-id: hello-word-job - hw-groups: - - group1 -``` - -Basically it means, that the job _hello-world-job_ needs to be run on workers -that belong to the `group_1` hardware group . Reference files are downloaded -from the default location configured in API (such as -`http://localhost:9999/exercises`) if not stated explicitly otherwise. Job -execution log will not be saved to result archive. - -Next the tasks have to be constructed under _tasks_ section. In this demo job, -every task depends only on previous one. The first task has input file -_source.c_ (if submitted by user) already available in working directory, so -just call the GCC. Compilation is run in sandbox as any other external program -and should have relaxed time and memory limits. In this scenario, worker -defaults are used. If compilation fails, the whole job is immediately terminated -(because the _fatal-failure_ bit is set). Because _bound-directories_ option in -sandbox limits section is mostly shared between all tasks, it can be set in -worker configuration instead of job configuration (suppose this for following -tasks). For configuration of workers please contact your administrator. - -Please note that working directory inside sandbox is automatically bounded to -the directory with fetched user source codes and therefore you do not have to -bound it by yourself. Also note that directories inside sandbox can be bound to -different paths, so inside sandbox you have to use special paths. For working -directory inside sandbox you can use ${EVAL_DIR} variable. - -```{.yml} -- task-id: "compilation" - type: "initiation" - fatal-failure: true - cmd: - bin: "/usr/bin/gcc" - args: - - "source.c" - - "-o" - - "helloworld" - sandbox: - name: "isolate" - limits: - - hw-group-id: group1 -``` - -The compiled program is executed with time and memory limit set and the standard -output is redirected to a file. This task depends on _compilation_ task, because -the program cannot be executed without being compiled first. It is important to -mark this task with _execution_ type, so exceeded limits will be reported in -frontend. - -Time and memory limits set directly for a task have higher priority than worker -defaults. One important constraint is, that these limits cannot exceed limits -set by workers. Worker defaults are present as a safety measure so that a -malformed job configuration cannot block the worker forever. Worker default -limits should be reasonably high, like a gigabyte of memory and several hours of -execution time. For exact numbers please contact your administrator. - -It is important to know that if the output of a program (both standard and -error) is redirected to a file, the sandbox disk quotas apply to that file, as -well as the files created directly by the program. In case the outputs are -ignored, they are redirected to `/dev/null`, which means there is no limit on -the output length (as long as the printing fits in the time limit). - -```{.yml} -- task-id: "execution_1" - test-id: "A" - type: "execution" - dependencies: - - compilation - cmd: - bin: "helloworld" - sandbox: - name: "isolate" - stdout: ${EVAL_DIR}/out.txt - limits: - - hw-group-id: group1 - time: 0.5 - memory: 8192 -``` - -Fetch sample solution from file server. Base URL of file server is in the header -of the job configuration, so only the name of required file (its `sha1sum` in -our case) is necessary. - -```{.yml} -- task-id: "fetch_solution_1" - test-id: "A" - dependencies: - - execution - cmd: - bin: "fetch" - args: - - "a0b65939670bc2c010f4d5d6a0b3e4e4590fb92b" - - "${SOURCE_DIR}/reference.txt" -``` - -Comparison of results is quite straightforward. It is important to set the task -type to _evaluation_, so that the return code is set to 0 if the program is -correct and 1 otherwise. We do not set our own limits, so the default limits are -used. - -```{.yml} -- task-id: "judge_1" - test-id: "A" - type: "evaluation" - dependencies: - - fetch_solution_1 - cmd: - bin: "/usr/bin/diff" - args: - - "out.txt" - - "reference.txt" - sandbox: - name: "isolate" - limits: - - hw-group-id: group1 -``` -