some clarification

master
Teyras 9 years ago
parent 16e54bcb22
commit 206f681ca9

@ -24,8 +24,13 @@ whitespace characters between words of the output.
### Stages ### Stages
A stage is a logical unit of the testing process. It specifies how to do a step A stage is a logical unit of the testing process. It specifies how to do a step
in the build process and how to test if it behaves correctly. The following is in the build process and how to test if the student's submission behaves
contained in its configuration: correctly. After the evaluation, the worker outputs a log for every processed
stage that contains information such as which tests passed and how many
resources were used.
A stage's configuration contains the following (this doesn't yet map exactly to
a particular configuration file format):
- **Name** - a unique string identifier of the stage - **Name** - a unique string identifier of the stage
- **Build command** (optional) - used to prepare the submitted files for this - **Build command** (optional) - used to prepare the submitted files for this
@ -34,18 +39,23 @@ contained in its configuration:
stage stage
- **Test command** - used to run one specific test - **Test command** - used to run one specific test
- **Test input policy** - how to pass the test input to the program? - **Test input policy** - how to pass the test input to the program?
- redirect it to its standard input - **redirect** it to its standard input (default)
- pass the name of the input file as an argument - pass the path to an input file as an **argument**
- **Judge** - which judge should be used to evaluate the solution's output? - **Judge** - which judge should be used to evaluate the solution's output?
Custom judges can be supplied with the assignment. Custom judges can be supplied with the assignment.
- **Limits** - how much memory, time, etc. can be used when evaluating a test
- **Error policy** (optional) - what should we do when a test fails? - **Error policy** (optional) - what should we do when a test fails?
- **interrupt** the stage (default) - **interrupt** the evaluation (default)
- **continue** with the next test - **continue** with another **test**
- **jump** to another stage (TODO cycle detection?) - **continue** with another **group**
- **jump** to another stage
- **Success policy** (optional) - what to do when all tests pass? - **Success policy** (optional) - what to do when all tests pass?
- **jump** to another stage (the next one by default) - **jump** to another stage (the next one by default)
- **end** the evaluation, even if there are still unprocessed stages - **end** the evaluation, even if there are still unprocessed stages
When jumping between stages, it's only possible to jump forward, so that no
stage is evaluated multiple times.
## Case study ## Case study
We present some of the courses that might use ReCodEx to evaluate homework We present some of the courses that might use ReCodEx to evaluate homework
@ -54,8 +64,8 @@ of stages.
### Simple programming exercises ### Simple programming exercises
For example introductory programming courses such as Programming I or For example introductory programming courses such as Programming I or Java
Programming in Java. programming.
In the simplest case we only need one stage that builds the program and passes In the simplest case we only need one stage that builds the program and passes
the test inputs to its standard input. We will use the C language for this the test inputs to its standard input. We will use the C language for this

Loading…
Cancel
Save