some clarification

master
Teyras 9 years ago
parent 16e54bcb22
commit 206f681ca9

@ -24,8 +24,13 @@ whitespace characters between words of the output.
### Stages
A stage is a logical unit of the testing process. It specifies how to do a step
in the build process and how to test if it behaves correctly. The following is
contained in its configuration:
in the build process and how to test if the student's submission behaves
correctly. After the evaluation, the worker outputs a log for every processed
stage that contains information such as which tests passed and how many
resources were used.
A stage's configuration contains the following (this doesn't yet map exactly to
a particular configuration file format):
- **Name** - a unique string identifier of the stage
- **Build command** (optional) - used to prepare the submitted files for this
@ -34,18 +39,23 @@ contained in its configuration:
stage
- **Test command** - used to run one specific test
- **Test input policy** - how to pass the test input to the program?
- redirect it to its standard input
- pass the name of the input file as an argument
- **redirect** it to its standard input (default)
- pass the path to an input file as an **argument**
- **Judge** - which judge should be used to evaluate the solution's output?
Custom judges can be supplied with the assignment.
- **Limits** - how much memory, time, etc. can be used when evaluating a test
- **Error policy** (optional) - what should we do when a test fails?
- **interrupt** the stage (default)
- **continue** with the next test
- **jump** to another stage (TODO cycle detection?)
- **interrupt** the evaluation (default)
- **continue** with another **test**
- **continue** with another **group**
- **jump** to another stage
- **Success policy** (optional) - what to do when all tests pass?
- **jump** to another stage (the next one by default)
- **end** the evaluation, even if there are still unprocessed stages
When jumping between stages, it's only possible to jump forward, so that no
stage is evaluated multiple times.
## Case study
We present some of the courses that might use ReCodEx to evaluate homework
@ -54,8 +64,8 @@ of stages.
### Simple programming exercises
For example introductory programming courses such as Programming I or
Programming in Java.
For example introductory programming courses such as Programming I or Java
programming.
In the simplest case we only need one stage that builds the program and passes
the test inputs to its standard input. We will use the C language for this

Loading…
Cancel
Save