a few words about judges

master
Teyras 9 years ago
parent d4f39620e7
commit a44a0c0a49

@ -15,6 +15,12 @@ one the expected output. Tests can be organized into groups for easier
organization and to facilitate more complex grading scenarios (e.g. "An organization and to facilitate more complex grading scenarios (e.g. "An
assignment passes only if it passes all tests in group X"). assignment passes only if it passes all tests in group X").
Tests are run by a special program called a *judge*, which compares the output
of the program to the expected output. By setting a judge, we can specify how
strict the testing is - for example, some assignments require the solution to
output exactly the same bytes as expected. Others permit any number of
whitespace characters between words of the output.
### Stages ### Stages
A stage is a logical unit of the testing process. It specifies how to do a step A stage is a logical unit of the testing process. It specifies how to do a step
@ -30,6 +36,8 @@ contained in its configuration:
- **Test input policy** - how to pass the test input to the program? - **Test input policy** - how to pass the test input to the program?
- redirect it to its standard input - redirect it to its standard input
- pass the name of the input file as an argument - pass the name of the input file as an argument
- **Judge** - which judge should be used to evaluate the solution's output?
Custom judges can be supplied with the assignment.
- **Error policy** (optional) - what should we do when a test fails? - **Error policy** (optional) - what should we do when a test fails?
- **interrupt** the stage (default) - **interrupt** the stage (default)
- **continue** with the next test - **continue** with the next test

Loading…
Cancel
Save