diff --git a/Assignments.md b/Assignments.md index 65d475e..3254dfd 100644 --- a/Assignments.md +++ b/Assignments.md @@ -15,6 +15,12 @@ one the expected output. Tests can be organized into groups for easier organization and to facilitate more complex grading scenarios (e.g. "An assignment passes only if it passes all tests in group X"). +Tests are run by a special program called a *judge*, which compares the output +of the program to the expected output. By setting a judge, we can specify how +strict the testing is - for example, some assignments require the solution to +output exactly the same bytes as expected. Others permit any number of +whitespace characters between words of the output. + ### Stages A stage is a logical unit of the testing process. It specifies how to do a step @@ -30,6 +36,8 @@ contained in its configuration: - **Test input policy** - how to pass the test input to the program? - redirect it to its standard input - pass the name of the input file as an argument +- **Judge** - which judge should be used to evaluate the solution's output? + Custom judges can be supplied with the assignment. - **Error policy** (optional) - what should we do when a test fails? - **interrupt** the stage (default) - **continue** with the next test