recodex-wiki/Exercise-Configuration.md

Exercise Configuration

In ReCodEx there are two configurations of exercise High Level Configuration (HiLC) and Low Level Configuration (LoLC). LoLC is used in backend, by workers for instance and should be general enough to create all kinds of worker tasks. On the other hand HiLC should be easy enough to be written or composed by normal application users, preferably in the form of graphical editation. But this configuration always has to be somehow stored, that is where this document comes in handy.

HiLC is divided in several parts which takes care of different things. There are ExerciseConfig, Pipelines, Limits and RuntimeConfig. From these components configuration of exercise is composed and on every submit new LoLC is compiled from it.

ExerciseConfig

Represents basic exercise configuration which connects all things together. For some reasons there two formats of this configuration, one which is saved in the database and the other one which is sent back to web application. Both formats are described bellow.

Frontend Format

Returned as JSON.

Mandatory items are bold, optional italic, description of items follows:

• ${implicit list of environments} - ...
  • name - ...
  • tests - ...
    • name - ...
    • pipelines - ...
      • name - ...
      • variables - ...
        • name - ...
        • type - ...
        • value - ...

Example:

[
  {
    "name":"default",
    "tests":[
      {
        "name":"Test 1",
        "pipelines":[
          {
            "name":"pipeline1",
            "variables":[
              {
                "name":"varA",
                "type":"string",
                "value":"valA"
              }
            ]
          }
        ]
      },
      {
        "name":"Test 2",
        "pipelines":[
          {
            "name":"pipeline2",
            "variables":[
              {
                "name":"varB",
                "type":"file",
                "value":"valB"
              }
            ]
          }
        ]
      }
    ]
  },
  {
    "name":"java8",
    "tests":[
      {
        "name":"Test 1",
        "pipelines":[
          {
            "name":"pipelineJava",
            "variables":[
              {
                "name":"varJava",
                "type":"string",
                "value":"valJava"
              }
            ]
          }
        ]
      },
      {
        "name":"Test 2",
        "pipelines":[
          {
            "name":"pipeline2",
            "variables":[
              {
                "name":"varB",
                "type":"file",
                "value":"valB"
              }
            ]
          }
        ]
      }
    ]
  },
  {
    "name":"cpp11",
    "tests":[
      {
        "name":"Test 1",
        "pipelines":[
          {
            "name":"pipeline1",
            "variables":[
              {
                "name":"varA",
                "type":"string",
                "value":"valA"
              }
            ]
          }
        ]
      },
      {
        "name":"Test 2",
        "pipelines":[
          {
            "name":"pipeline2",
            "variables":[
              {
                "name":"varCpp",
                "type":"file",
                "value":"valCpp"
              }
            ]
          }
        ]
      }
    ]
  }
]

Backend Format

Stored in yaml.

Mandatory items are bold, optional italic, description of items follows:

• environments - ...
• tests - ...
  • ${test identification} - ...
    • pipelines - ...
      • ${pipeline identification} - ...
        • variables - ...
          • ${variable identification} - ...
            • type - ...
            • value - ...
    • environments - ...
      • ${environment identification} - ...
        • pipelines - ...
          • ${pipeline identification} - ...
            • variables - ...
              • ${variable identification} - ...
                • type - ...
                • value - ...

Example:

environments:
  - java8
  - cpp11
tests:
  "Test 1":
    pipelines:
      "pipeline1":
        variables:
          varA:
            type: string
            value: valA
    environments:
      java8:
        pipelines:
          "pipelineJava":
            variables:
              varJava:
                type: string
                value: valJava
      cpp11: []
  "Test 2":
    pipelines:
      "pipeline2":
        variables:
          varB:
            type: file
            value: valB
    environments:
      cpp11:
        pipelines:
          "pipeline2":
            variables:
              varCpp:
                type: file
                value: valCpp

Pipeline

Pipelines are sent to clients in JSON format and are stored in API in corresponding YAML with the same structure.

Important features:

• Every port either have to have defined reference to variable or it has to be blank. Actual value (for example string) is not allowed in port. If variable name is declared in port it has to exist in variables table.
• Connection between ports have to always be one end to another end. It is not possible to have one output port which redirects variables to two input ports.

Mandatory items are bold, optional italic, description of items follows:

• variables - ...
  • ${implicit list of variables}
    • name - ...
    • type - ...
    • value - ...
• boxes - ...
  • ${implicit list of boxes}
    • name - ...
    • portsIn - ...
      • ${name of the port}
        • type - ...
        • value - ...
    • portsOut - ...
    • type - ...

Example:

[
   {
      "name":"source",
      "portsIn":[],
      "portsOut":[{ "source_file":[{"type":"file", "value":"source_file"}] }],
      "type":"data"
   },
   {
      "name":"test",
      "portsIn":[],
      "portsOut":[{
         "test_file":[{"type":"file", "value":"test_file"}],
         "expected_output":[{"type":"file", "value":"expected_output"}]
      }],
      "type":"data"
   },
   {
      "name":"compilation",
      "portsIn":[{ "input_file":[{"type":"file", "value":"source_file"}] }],
      "portsOut":[{ "output_file":[{"type":"file", "value":"binary_file"}] }],
      "type":"compilation"
   },
   {
      "name":"run",
      "portsIn":[{ "binary_file":[{"type":"file", "value":"binary_file"}] }],
      "portsOut":[{ "output_file":[{"type":"file", "value":"actual_output"}] }],
      "type":"execution"
   },
   {
      "name":"judge",
      "portsIn":[{
         "actual_output":[{"type":"file", "value":"actual_output"}],
         "expected_output":[{"type":"file", "value":"expected_output"}]
      }],
      "portsOut":[{ "score":[{"type":"file", "value":"score"}] }],
      "type":"evaluation"
   }
]

Limits

Mandatory items are bold, optional italic, description of items follows:

• ${box identification} - ...
  • wall-time - ...
  • memory - ...
  • parallel - ...

Example:

box-id-1:
  wall-time: 5
  memory: 50
  parallel: 500
box-id-2:
  wall-time: 6
  memory: 60

ExerciseEnvironmentConfig

Frontend Format

Mandatory items are bold, optional italic, description of items follows:

• ...

Example:

[
  {
    "runtimeEnvironmentId":"CRuntime",
    "variablesTable":[
      {
        "varA":{
          "type":"string",
          "value":"valA"
        }
      },
      {
        "varB":{
          "type":"file",
          "value":"valB"
        }
      }
    ]
  },
  {
    "runtimeEnvironmentId":"JavaRuntime",
    "variablesTable":[
      {
        "varA":{
          "type":"file",
          "value":"javaA"
        }
      },
      {
        "varB":{
          "type":"string",
          "value":"javaB"
        }
      }
    ]
  }
]

Backend Format

Mandatory items are bold, optional italic, description of items follows:

• ...

Example: