From d4f39620e7c1406500697aaee23c0aee276474b1 Mon Sep 17 00:00:00 2001 From: Teyras Date: Fri, 27 Nov 2015 12:12:21 +0100 Subject: [PATCH] elaborate test runnin --- Assignments.md | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-) diff --git a/Assignments.md b/Assignments.md index 19b5824..65d475e 100644 --- a/Assignments.md +++ b/Assignments.md @@ -10,6 +10,10 @@ build, run and test it. The testing process is divided into stages. ### Tests +A test is a pair of files, where the first one specifies an input and the other +one the expected output. Tests can be organized into groups for easier +organization and to facilitate more complex grading scenarios (e.g. "An +assignment passes only if it passes all tests in group X"). ### Stages @@ -20,8 +24,12 @@ contained in its configuration: - **Name** - a unique string identifier of the stage - **Build command** (optional) - used to prepare the submitted files for this test stage -- **Test list** - specifies the tests to be run during this stage +- **Test list** - specifies the tests (or test groups) to be run during this + stage - **Test command** - used to run one specific test +- **Test input policy** - how to pass the test input to the program? + - redirect it to its standard input + - pass the name of the input file as an argument - **Error policy** (optional) - what should we do when a test fails? - **interrupt** the stage (default) - **continue** with the next test