Added information about scoring

master
Marek Lisy 8 years ago
parent 899a8d0c4b
commit 10c1b76d4c

@ -256,6 +256,32 @@ results:
...
```
## Scoring
Every assignment consists of tasks. Only some tasks however are part of the evaluation. Those evaluated tasks are grouped into **tests**. Each task might be assigned a _test-id_ parameter, as described above. Every test must consist of at least two tasks: execution and evaluation by a judge. The former retrieves information about the execution such as elapsed time and memory consumed, the latter result with a score - float between 0 and 1.
Total resulting score of the assignment submission is then calculated according to a supplied score config (described below). Total score is also a float between 0 and 1. This number is then multiplied by the maximum of points awarded for the assignment by the teacher assigning the exercise - not the assignment author.
### Simple score calculation
At the first stage of development, simple score calculation is used. This will most probably be replaced by more advanced score calculation algorithm in near future.
Simple score calculation just looks at the score of each test. In the score config, author of the assignment must specify weights of each test. Resulting score is calculated as a sum of products of score and weight of each test divided by the sum of all weights. The algorithm in Python would look something like this:
```
sum = weightSum = 0
for t in tests:
sum += t.score * t.weight
weightSum += t.weight
score = sum / weightSum
```
Sample score config in YAML format:
```
testWeights:
a: 300 # test with id 'a' has a weight of 300
b: 200
c: 100
d: 100
```
### Logs
During execution tasks can use only one shared log. There is no use for multiple logs which will be used in all tasks, because of pretty small amount of information which is loged. Log is in default disabled and can be enabled in job configuration, then all logged actions in tasks will be visible here.

Loading…
Cancel
Save