|
|
@ -2345,7 +2345,23 @@ Roles description:
|
|
|
|
|
|
|
|
|
|
|
|
## Writing score configuration
|
|
|
|
## Writing score configuration
|
|
|
|
|
|
|
|
|
|
|
|
@todo: introduction to scoring, mention that there can be more implementation but for now only simple is available
|
|
|
|
Important thing about assignment is how to assign points to particular
|
|
|
|
|
|
|
|
solutions. As mentioned previously whole job is composed of logical tests. All
|
|
|
|
|
|
|
|
of these tests have to contain one essential "evaluation" task. Evaluation task
|
|
|
|
|
|
|
|
should output one float number which can be further used for scoring of
|
|
|
|
|
|
|
|
particular tests.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Total resulting score of the students solution is then calculated according to a
|
|
|
|
|
|
|
|
supplied score config (described below) and using specified calculator. Total
|
|
|
|
|
|
|
|
score is also a float between 0 and 1. This number is then multiplied by the
|
|
|
|
|
|
|
|
maximum of points awarded for the assignment by the teacher assigning the
|
|
|
|
|
|
|
|
exercise -- not the exercise author.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
For now there is only one way how to write score configuration using only simple
|
|
|
|
|
|
|
|
score calculator. But the implementation in API is agile enough to handle
|
|
|
|
|
|
|
|
upcoming score calculators which might use some more complex scoring algorithms.
|
|
|
|
|
|
|
|
This also means that future calculators do not have to use YAML format as the
|
|
|
|
|
|
|
|
already defined one, but can use its own or different one.
|
|
|
|
|
|
|
|
|
|
|
|
### Simple score calculation
|
|
|
|
### Simple score calculation
|
|
|
|
|
|
|
|
|
|
|
@ -2488,6 +2504,10 @@ the output length (as long as the printing fits in the time limit).
|
|
|
|
limits:
|
|
|
|
limits:
|
|
|
|
- hw-group-id: group1
|
|
|
|
- hw-group-id: group1
|
|
|
|
chdir: ${EVAL_DIR}
|
|
|
|
chdir: ${EVAL_DIR}
|
|
|
|
|
|
|
|
bound-directories:
|
|
|
|
|
|
|
|
- src: ${SOURCE_DIR}
|
|
|
|
|
|
|
|
dst: ${EVAL_DIR}
|
|
|
|
|
|
|
|
mode: RW
|
|
|
|
time: 0.5
|
|
|
|
time: 0.5
|
|
|
|
memory: 8192
|
|
|
|
memory: 8192
|
|
|
|
```
|
|
|
|
```
|
|
|
@ -2528,7 +2548,11 @@ used.
|
|
|
|
name: "isolate"
|
|
|
|
name: "isolate"
|
|
|
|
limits:
|
|
|
|
limits:
|
|
|
|
- hw-group-id: group1
|
|
|
|
- hw-group-id: group1
|
|
|
|
chdir: ${EVAL_DIR}
|
|
|
|
chdir: ${EVAL_DIR}
|
|
|
|
|
|
|
|
bound-directories:
|
|
|
|
|
|
|
|
- src: ${SOURCE_DIR}
|
|
|
|
|
|
|
|
dst: ${EVAL_DIR}
|
|
|
|
|
|
|
|
mode: RW
|
|
|
|
```
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|