elaborate test runnin

master
Teyras 9 years ago
parent ccf81142ed
commit d4f39620e7

@ -10,6 +10,10 @@ build, run and test it. The testing process is divided into stages.
### Tests ### Tests
A test is a pair of files, where the first one specifies an input and the other
one the expected output. Tests can be organized into groups for easier
organization and to facilitate more complex grading scenarios (e.g. "An
assignment passes only if it passes all tests in group X").
### Stages ### Stages
@ -20,8 +24,12 @@ contained in its configuration:
- **Name** - a unique string identifier of the stage - **Name** - a unique string identifier of the stage
- **Build command** (optional) - used to prepare the submitted files for this - **Build command** (optional) - used to prepare the submitted files for this
test stage test stage
- **Test list** - specifies the tests to be run during this stage - **Test list** - specifies the tests (or test groups) to be run during this
stage
- **Test command** - used to run one specific test - **Test command** - used to run one specific test
- **Test input policy** - how to pass the test input to the program?
- redirect it to its standard input
- pass the name of the input file as an argument
- **Error policy** (optional) - what should we do when a test fails? - **Error policy** (optional) - what should we do when a test fails?
- **interrupt** the stage (default) - **interrupt** the stage (default)
- **continue** with the next test - **continue** with the next test

Loading…
Cancel
Save