recodex-wiki/FAQ.md

This page will hold the answers for Frequently Asked Questions.

Please note that this FAQ also contains specific information that apply only to our pilot instance of ReCodEx at MFF-UK. If you have a specific problem with this instance, contact the administrator (recodex@mff.cuni.cz).

Browser Issues

First, make sure you are using a modern web browser, preferably Mozilla Firefox or Google Chrome. Although our objective is to support all modern browsers, rapid development prevents us from doing extensive testing across many platforms; hence, we are sticking with the aforementioned two browsers for now.

Make sure you refresh your browser cache (Ctrl+F5) before logging in, especially if a new version has been deployed. The refresh will prevent older pieces of the application from lingering in your cache. This is just a precaution.

Sometimes a component on a page does not refresh itself correctly. The application is a complex system of components and is still under development. If you make an action and do not see the results in a reasonable while, refresh the page.

How to register into ReCodEx and what method should I use?

First of all please note that you need to register before signing in (those are different steps).

ReCodEx provides multiple ways for a user to register and log in. Currently there are two options: a local ReCodEx account or an account bound to the CAS service. The preferred way of registration and authentication is using the CAS service and using a local account only in special circumstances. CAS authentication provides more features, such as automatic binding of users to SIS groups. At present, the CAS registration is the only option enabled.

I have successfully passed CAS authentication for the registration, but then I ran into an error

There are two most often errors. First one is that your LDAP data (in CAS service) do not contain email address (i.e., you will get The user attributes received from the CAS do not contain an email address error). In this case, the remedy should be simple enough. Just fill in you email address in CAS administration.

The second most frequent reason is that your account does not have appropriate affiliation attributes (The user attributes received from the CAS has no affiliation attributes that would allow registration in ReCodEx. Authenticated account does not belong to a student not to an employee of MFF.). In such case you should either attempt to set your LDAP records straight or (if you are not a regular student), ask the administrator to create you an account manually. We are planning a feature that will allow the teachers to send invitations to external users, but it will not be ready any time soon.

I registered using a CAS account but I also want to use a local one

User can create local account in the profile Settings page. In here if you are registered using external account and not local one, you should see Create local account button. After clicking on it local login without password will be created. Next thing you have to set your password to newly created local login.

I registered (had created) a local account but I want to use also CAS and its advantages

The system's login mechanism is prepared for this option. The only prerequisite is that your email address in the local account must be the same as the one in CAS. If this is the case, binding the accounts is quite easy, just log in using CAS and the system will automatically bind those two accounts. Also please note that when binding your local account your password will be cleared and if you want to use it further, you have to change it.

Exercise vs. Assignment

Exercise and assignment are two terms which are not the same and there are some differences. In general, exercises are templates for assignments. In more detail, exercises are algorithmic problems defined and configured by supervisors. Exercise contains test cases which executes solutions submitted by users and check their outputs against sample results. Configuration of an exercise also defines time and memory limits which the solution has to meet and scoring details which are used in the evaluation (assigning points for passed tests). Exercise itself cannot be solved by students and is displayed only to supervisors and administrators.

If supervisor assigns an exercise to a group of students, an assignment is created from the exercise. Assignment can define some further details like number of points which can students gain or deadlines which should be met. But most importantly, an assignment is a full copy of the exercise. That means the assignment will not be modified when the configuration of the exercise is changed. This behavior was chosen to avoid unnecessary broken assignments in case the exercise is being edited (and possibly gets broken) by its author.

When the assignment gets out of sync with its original exercise (because the exercise was edited), a informational panel is displayed to the supervisor on the assignment detail/edit page. This panel also hold a button, which allows the supervisor to overwrite current assignment data (text, configuration, limits, ...) with the current version of the exercise.

Runtime Environments

Exercises may be configured to support different runtime environments. Runtime environments defines the programming language (i.e., compiler) and necessary libraries (like Java RTE or Mono). Assignment may limit, which RTEs are visible, but cannot add additional ones. The actual runtime used for a submission is pre-selected based on submitted files. If the files does not identify RTE uniquely, the submitter may select the actual runtime.

Some runtimes have specific nuances. The details (useful both for students and for supervisors) are on a separate page.

For some particular languages ReCodEx provides mappings of exit codes from internal runners to messages. Full tables of codes can be found on separate page.

As a supervisor I want to see more detailed logs of user submissions

Before we start let us define debug vs. normal submissions. The results of a debug submission contain evaluation results, the execution log and the outputs of executed programs. And the results of a normal submission contain only the results needed for proper assignment of points (aka. evaluation results).

With this being said, all submissions made by students for an assignment are normal and even submissions of reference exercise solutions are normal. As a supervisor you should be able to do something called resubmitting of a user solution; this feature should be available on every page of a particular submission. On resubmitting, debug outputs and execution log are turned on by default and can be found in the results archive. Please note that resubmitting all solutions for an assignment is just a normal submission without debug information.

How to write Custom Judges

ReCodEx provides a few initial judges programs. They are mostly adopted from CodEx and installed automatically with the worker component. Judging programs have to meet some requirements. Basic ones are inspired by standard diff application -- two mandatory positional parameters which have to be the files for comparison and exit code reflecting if the result is correct (0) or wrong (1).

This interface lacks support for returning additional data by the judges, for example similarity of the two files calculated as the Levenshtein edit distance. To allow passing these additional values an extended judge interface can be implemented:

• Parameters: There are two mandatory positional parameters which have to be files for comparison, first one is expected output, second one result from tested program
• Results:
  • comparison OK
    • exitcode: 0
    • stdout: at the beginning of the output, there is a double value which should be quality percentage of the judged file, after that white character as a delimiter can be present and furthermore some text which will be accessible within web application
  • comparison BAD
    • exitcode: 1
    • stdout: might contain text which will be accessible within web application
  • error during execution
    • exitcode: 2
    • stderr: there should be description of error

The additional double value is saved to the results file and can be used for score calculation in the frontend. If just the basic judge is used, the values are 1.0 for exit code 0 and 0.0 for exit code 1.

See judge templates page for more details.

Runtime Environments/Compilers Versions

Current state on pilot server (MFF) workers:

• C/C++ (GCC v9.2)
• Go lang (v1.11.5)
• Haskell (GHC v8.6.1)
• Java (OpenJDK v13)
• JavaScript (Node.js v10.16.3)
• C# (Mono v5.20)
• Pascal (FPC v3.0.2)
• PHP (v7.2.22)
• Prolog (SWI v8.0.1)
• Python v3.6
• Rust (v 1.37.0)