diff --git a/Rewritten-docs.md b/Rewritten-docs.md index 3490a29..171de84 100644 --- a/Rewritten-docs.md +++ b/Rewritten-docs.md @@ -630,3 +630,148 @@ the awarded points will be overridden. @todo: Describe the comments thread behavior (public/private comments), who else can see the comments + + +### Writing job configuration + +To run and evaluate an exercise the backend needs to know the steps how to do +that. This is different for each environment (operation system, programming +language, etc.), so each of the environments needs to have separate +configuration. + +Backend understands powerful, but quite low level description of simple +connected tasks written in YAML syntax. More about the syntax and general task +overview can be found on [separate +page](https://github.com/ReCodEx/wiki/wiki/Assignments). One of the planned +features was user friendly configuration editor, but due to tight deadline and +team composition it did not make it to the first release. However, writing +configuration in the basic format will be always available and allows you to use +the full expressive power of the system. + +This section walks through creation of job configuration for _hello world_ +exercise. The goal is to compile file _source.c_ and check if it prints string +`Hello World!` to the standard output. + +The problem can be split into several tasks: + +- compile _source.c_ into _helloworld_ by `/usr/bin/gcc` +- run _helloworld_ and save standard output into _out.txt_ +- fetch predefined output (suppose it is already uploaded to fileserver) with + hash `a0b65939670bc2c010f4d5d6a0b3e4e4590fb92b` to _reference.txt_ +- compare _out.txt_ and _reference.txt_ by `/usr/bin/diff` + +The absolute path of tools can be obtained from system administrator. However, +`/usr/bin/gcc` is location, where the GCC binary is available almost everywhere, +so location of some tools can be (professionally) guessed. + +First, write header of the job to the configuration file. + +```{.yml} +submission: + job-id: hello-word-job + language: c + file-collector: http://localhost:9999/exercises + hw-groups: + - group1 +``` + +Basically it means, that the job _hello-world-job_ is for C language and needs +to be run on workers with capabilities of _group1_ group. Reference files are +downloaded from http://localhost:9999/exercises. + +Next the tasks have to be constructed under _tasks_ section. In this demo job, +every task depends only on previous one. The first task has input file +_source.c_ (if submitted by user) already available in working directory, so +just call the GCC. Compilation is run in sandbox as any other external program +and should have relaxed time and memory limits. In this scenarion, worker +defaults are used. If compilation fails, whole job is immediately terminated +(_fatal-failure_ bit set). Because _chdir_ and _bound-directories_ options in +sandbox limits section are mostly common for all tasks, they can be set in +worker configuration instead of job configuration (suppose this for following +tasks). For configuration of workers please contact your administrator. + +```{.yml} +- task-id: "compilation" + type: "initiation" + priority: 1 + fatal-failure: true + cmd: + bin: "/usr/bin/gcc" + args: + - "source.c" + - "-o" + - "helloworld" + sandbox: + name: "isolate" + limits: + - hw-group-id: group1 + chdir: ${EVAL_DIR} + bound-directories: + - src: ${SOURCE_DIR} + dst: ${EVAL_DIR} + mode: RW +``` + +The compiled program is executed with time and memory limit set and standard +output redirected to a file. This task depends on _compilation_ task, because +the program cannot be executed without being compiled first. It is important to +mark this task with _execution_ type, so exceeded limits will be reported in +frontend. + +```{.yml} +- task-id: "execution_1" + test-id: "A" + type: "execution" + priority: 2 + fatal-failure: false + dependencies: + - compilation + cmd: + bin: "helloworld" + sandbox: + name: "isolate" + stdout: ${EVAL_DIR}/out.txt + limits: + - hw-group-id: group1 + time: 0.5 + memory: 8192 +``` + +Fetch sample solution from fileserver. Base URL of fileserver is in job header, +so only the name of required file (`sha1sum` in our case) is necessary. + +```{.yml} +- task-id: "fetch_solution_1" + test-id: "A" + priority: 3 + dependencies: + - execution + cmd: + bin: "fetch" + args: + - "a0b65939670bc2c010f4d5d6a0b3e4e4590fb92b" + - "${SOURCE_DIR}/reference.txt" +``` + +Comparison of results is quite straightforward. Important is to mark the task as +_evaluation_ type, so the return code represents if the program is correct (0) +or not (other). Worker default limits are used. + +```{.yml} +- task-id: "judge_1" + test-id: "A" + type: "evaluation" + priority: 4 + dependencies: + - fetch_solution_1 + cmd: + bin: "/usr/bin/diff" + args: + - "out.txt" + - "reference.txt" + sandbox: + name: "isolate" + limits: + - hw-group-id: group1 +``` +