recodex-wiki/User-documentation.md

Výběr z dokumentace - stručné příručky

• Příručka pro nové uživatele / Guide for new users
• Příručka pro studenty / Guide for students
• Příručka pro vedoucí (cvičící)
• Příručka pro adminy (garanty předmětu, přednášející)
• Příručka pro tvorbu úlohy

User Documentation - OUTDATED

If you are a ReCodEx user and you wish to contribute, please, let us know...

	Web-app	Release Date
Corresponds to	v1.0.0	25.11.2016

Users interact with the ReCodEx through the web application. It is required to use a modern web browser with good HTML5 and CSS3 support. Among others, cookies and local storage are used. Also a decent JavaScript runtime must be provided by the browser.

Supported and tested browsers are: Firefox 50+, Chrome 55+, Opera 42+ and Edge 13+. Mobile devices can have problems with internationalization and possibly lack support for some common features of desktop browsers. In this stage of development is not possible for us to fine tune the interface for major mobile browsers on all mobile platforms. However, it is confirmed to work with latest Google Chrome, Firefox and Gello browser. Also, it is confirmed to work with Safari browser on iOS 10.

Usage of the web application is divided into sections concerning particular user roles. Under these sections all possible use cases can be found. These sections are inclusive, so more privileged users need to read instructions for all less privileged users. Described roles are:

• Student
• Group supervisor
• Group administrator
• Instance administrator
• Superadministrator

Terminology

Instance -- Represents a university, company or some other organization unit which uses ReCodEx.

Group -- A group of students to which exercises are assigned by a supervisor. It should typically correspond with a real world lecture or lab group.

User -- A person that interacts with the system using the web interface (or an alternative client).

Student -- A user with least privileges who is subscribed to some groups and can submit solutions to exercise assignments.

Supervisor -- A person responsible for assigning exercises to a group and reviewing submissions.

Admin -- A person responsible for the maintenance of the system and fixing problems supervisors cannot solve.

Exercise -- An algorithmic problem that can be assigned to a group. They can be shared by the teachers using an exercise database in ReCodEx.

Assignment -- An exercise assigned to a group, possibly with modifications. In more detail it is full copy of exercise.

Runtime environment -- Runtime environment is unique combination of platform (OS) and programming language runtime/compiler in specific version. Runtime environments are managed by the administrators to reflect abilities of whole system.

Hardware group -- Hardware group is a set of workers with similar hardware. Its purpose is to group workers that are likely to run a program using the same amount of resources. Hardware groups are managed by the system administrators who have to keep them up-to-date.

General Basics

Description of general basics which are the same for all users of ReCodEx web application follows.

First Steps in ReCodEx

You can create an account by clicking the "Create account" menu item in the left sidebar. You can choose between two types of registration methods -- by creating a local account with a specific password, or pairing your new account with an existing CAS UK account.

If you decide to create a new local account using the "Create ReCodEx account” form, you will have to provide your details and choose a password for your account. Although ReCodEx allows using quite weak passwords, it is wise to use a bit stronger ones. The actual strength is shown in progress bar near the password field during registration. You will later sign in using your email address as your username and the password you select.

If you decide to use the CAS UK service, then ReCodEx will verify your CAS credentials and create a new account based on information stored there (name and email address). You can change your personal information later on the "Settings" page.

Regardless of the desired account type, an instance it will belong to must be selected. The instance will be most likely your university or other organization you are a member of.

To log in, go to the homepage of ReCodEx and in the left sidebar choose the menu item "Sign in". Then you must enter your credentials into one of the two forms -- if you selected a password during registration, then you should sign with your email and password in the first form called "Sign into ReCodEx". If you registered using the Charles University Authentication Service (CAS), you should put your student’s number and your CAS password into the second form called "Sign into ReCodEx using CAS UK".

There are several options you can edit in your user account:

• changing your personal information (i.e., name)
• changing your credentials (email and password)
• updating your preferences (source code viewer/editor settings, default language)

You can access the settings page through the "Settings" button right under your name in the left sidebar.

If you are not active in ReCodEx for a whole day, you will be logged out automatically. However, we recommend you sign out of the application after you finish your interaction with it. The logout button is placed in the top section of the left sidebar right under your name. You may need to expand the sidebar with a button next to the "ReCodEx” title (informally known as hamburger button), depending on your screen size.

Forgotten Password

If you cannot remember your password and you do not use CAS UK authentication, then you can reset your password. You will find a link saying "Cannot remember what your password was? Reset your password." under the sign in form. After you click this link, you will be asked to submit your registration email address. A message with a link containing a special token will be sent to you by e-mail -- we make sure that the person who requested password resetting is really you. When you visit the link, you will be able to enter a new password for your account. The token is valid only for a couple of minutes, so do not forget to reset the password as soon as possible, or you will have to request a new link with a valid token.

If you sign in through CAS UK, then please follow the instructions provided by the administrators of the service described on their website.

Dashboard

When you log into the system you should be redirected to your "Dashboard". On this page you can see some brief information about the groups you are member of. The information presented there varies with your role in the system -- further description of dashboard will be provided later on with according roles.

Student

Student is a default role for every newly registered user. This role has quite limited capabilities in ReCodEx. Generally, a student can only submit solutions of exercises in some particular groups. These groups should correspond to courses he/she attends.

On the "Dashboard" page there is "Groups you are student of" section where you can find list of your student groups. In first column of every row there is a brief panel describing concerning group. There is name of the group and percentage of gained points from course. If you have enough points to successfully complete the course then this panel has green background with tick sign. In the second column there is a list of assigned exercises with its deadlines. If you want to quickly get to the groups page you might want to use provided "Show group's detail" button.

Join Group

To be able to submit solutions you have to be a member of the right group. Each instance has its own group hierarchy, so you can choose only those within your instance. That is why a list of groups is available from under an instance link located in the sidebar. This link brings you to instance detail page.

In there you can see a description of the instance and most importantly in "Groups hierarchy" box there is a hierarchical list of all public groups in the instance. Please note that groups with plus sign are collapsible and can be further extended. When you find a group you would like to join, continue by clicking on "See group's page" link following with "Join group" link.

Note: Some groups can be marked as private and these groups are not visible in hierarchy and membership cannot be established by students themselves. Management of students in this type of groups is in the hands of supervisors.

On the group detail page there are multiple interesting things for you. The first one is a brief overview containing the information describing the group, there is a list of supervisors and also the hierarchy of the subgroups. The most important section is the "Student's dashboard" section. This section contains the list of assignments and the list of fellow students. If the supervisors of the group allowed students to see the statistic of their fellow students then there will also be the number of points each of the students has gained so far.

Start Solving Assignments

In the "Assignments" box on the group detail page there is a list of assigned exercises which students are supposed to solve. The assignments are displayed with their names and deadlines. There are possibly two deadlines, the first one means that till this datetime student will receive full amount of points in case of successful solution. Second deadline does not have to be set, but in case it is, the maximum number of points for successful solution between these two deadlines can be different.

An assignment link will lead you to assignment detail page where are presented all known details about assignment. There are of course both deadlines, limit of submissions which you can make and also full-range description of assignment, which can be localized. The localization can be on demand switched between all language variants in tab like box.

Further on the page you can find "Submitted solutions" box where is a list of submissions with links to result details. But most importantly there is a "Submit new solution" button on the assignment page which provides an interface to submit solution of the assignment.

After clicking on submit button, dialog window will show up. In here you can upload files representing your solution, you can even add some notes to mark the solution. Your supervisor can also access this note. After you successfully upload all files necessary for your solution, click the "Submit your solution" button and let ReCodEx evaluate the solution.

During the execution ReCodEx backend might send evaluation progress state to your browser which will be displayed in another dialog window. When the whole execution is finished then a "See the results" button will appear and you can look at the results of your solution.

View Results of Submission

On the results detail page there are a lot of information. Apart from assignment description, which is not connected to your results, there is also the solution submitter name (supervisor can submit a solution on your behalf), further there are files which were uploaded on submission and most importantly "Evaluation details" and "Test results" boxes.

Evaluation details contains overall results of your solution. There are information such as whether the solution was provided before deadlines, if the evaluation process successfully finished or if compilation succeeded. After that you can find a lot of values, most important one is the last, "Total score", consisting of your score, slash and the maximum number of points for this assignment. Interestingly the your score value can be higher than the maximum, which is caused by "Bonus points" item above. If your solution is nice and supervisor notices it, he/she can assign you additional points for effort. On the other hand, points can be also subtracted for bad coding habits or even cheating.

In test results box there is a table of all exercise tests results. Columns represents these information:

• test case overall result, symbol of yes/no option
• test case name
• percentage of correctness of this particular test
• evaluation status, if test was successfully executed or failed
• memory limit, if supervisor allowed it then percentual memory usage is displayed
• time limit, if supervisor allowed it then percentual time usage is displayed

A new feature of web application is "Comments and notes" box where you can communicate with your supervisors or just write random private notes to your submission. Adding a note is quite simple, you just write it to text field in the bottom of box and click on the "Send" button. The button with lock image underneath can switch visibility of newly created comments.

In case you think the ReCodEx evaluation of your solution is wrong, please use the comments system described above, or even better notify your supervisor by another channel (email). Unfortunately there is currently no notification mechanism for new comment messages.

Group Supervisor

Group supervisor is typically the lecturer of a course. A user in this role can modify group description and properties, assign exercises or manage list of students. Further permissions like managing subgroups or supervisors is available only for group administrators.

On "Dashboard" page you can find "Groups you supervise" section. Here there are boxes representing your groups with the list of students attending course and their points. Student names are clickable with redirection to the profile of the user where further information about his/hers assignments and solution can be found. To quickly jump onto groups page, use "Show group's detail" button at the bottom of the matching group box.

Manage Group

Locate group you supervise and you want to manage. All your supervised groups are available in sidebar under "Groups -- supervisor" collapsible menu. If you click on one of those you will be redirected to group detail page. In addition to basic group information you can also see "Supervisor's controls" section. In this section there are lists of current students and assignments.

As a supervisor of group you are able to see "Edit group settings" button at the top of the page. Following this link will take you to group editation page with form containing these fields:

• group name which is visible to other users
• external identification which may be used for pairing with entries in an information system
• description of group which will be available to users in instance (in Markdown)
• set if group is publicly visible (and joinable by students) or private
• options to set if students should be able see statistics of each other
• minimal points threshold which students have to gain to successfully complete the course

After filling all necessary fields the form can be sent by clicking on "Edit group" button and all changes will be applied.

For students management there are "Students" and "Add student" boxes. The first one is simple list of all students which are attending the course with the possibility of delete them from the group. That can be done by hitting "Leave group" button near particular user. The second box is for adding students to the group. There is a text field for typing name of the student and after clicking on the magnifier image or pressing enter key there will appear list of matched users. At this moment just click on the "Join group" button and student will be signed in to your group.

Assigning Exercises

Before assigning an exercise, you obviously have to know what exercises are available. A list of all exercises in the system can be found under "Exercises" link in sidebar. This page contains a table with exercises names, difficulties and names of the exercise authors. Further information about exercise is available by clicking on its name.

On the exercise details page are numerous information about it. There is a box with all possible localized descriptions and also a box with some additional information of exercise author, its difficulty, version, etc. There is also a description for supervisors by exercise author under "Exercise overview" option, where some important information can be found. And most notably there is an information about available programming languages for this exercise, under "Supported runtime environments" section.

If you decide that the exercise is suitable for one of your groups, look for the "Groups" box at the bottom of the page. There is a list of all groups you supervise with an "Assign" button which will assign the exercise to the selected group.

After clicking on the "Assign" button you should be redirected to assignment editation page. In there you can find two forms, one for editation of assignment meta information and the second one for setting exercise time and memory limits.

In meta information form you can fill these options:

• name of the assignment which will be visible in a group
• visibility (if an assignment is under construction then you can mark it as not visible and students will not see it)
• subform for localized descriptions (new localization can be added by clicking on "Add language variant" button, current one can be deleted with "Remove this language" button)
  • language of description from dropdown field (English, Czech, German)
  • description in selected language
• score configuration which will be used on students solution evaluation, you can find some very simple one already in here, description of score configuration can be found further in "Writing score configuration" chapter
• first submission deadline
• maximum points that can be gained before the first deadline; if you want to manage all points manually, set it to 0 and then use bonus points, which are described in the next subchapter
• second submission deadline, after that students still can submit exercises but they are given no points no points (must be after the first deadline)
• maximum points that can be gained between first deadline and second deadline
• submission count limit for the solutions of the students -- this limits the amount of attempts a student has at solving the problem
• visibility of memory and time ratios; if true students can see the percentage of used memory and time (with respect to the limit) for each test
• minimum percentage of points which each submission must gain to be considered correct (if it gets less, it will gain no points)
• whether the assignment is marked as bonus one and points from solving it are not included into group threshold limit (that means solving it can get you additional points over the limit)

The form has to be submitted with "Edit settings" button otherwise changes will not be saved.

The same editation page serves also for the purpose of assignment editation, not only creation. That is why on bottom of the page "Delete the assignment" box can be found. Clearly the button "Delete" in there can be used to unassign exercise from group.

The last unexplored area is the time and memory limits form. The whole form is situated in a box with tabs which are leading to particular runtime environments. If you wish not to use one of those, locate "Remove" button at the bottom of the box tab which will delete this environment from the assignment. Please note that this action is irreversible.

In general, every tab in environments box contains some basic information about runtime environment and another nested tabbed box. In there you can find all hardware groups which are available for the exercise and set limits for all test cases. The time limits have to be filled in seconds (float), memory limits are in bytes (int). If you are interested in some reference values to particular test case then you can take a peek on collapsible "Reference solutions' evaluations" items. If you are satisfied with changes you made to the limits, save the form with "Change limits" button right under environments box.

Management of The Solutions of The Students

One of the most important tasks of a group supervisor is checking student solutions. As automatic evaluation of them cannot catch all problems in the source code, it is advisable to do a brief manual review of the coding style of the student and reflect that in assignment bonus points.

On "Assignment detail" page there is an "View student results" button near top of the page (next to "Edit assignment settings" button). This will redirect you to a page where is a list of boxes, one box per student. Each student box contains a list of submissions for this assignment. The row structure of submission list is the same as the structure in the "Submitted solution" box. More information about every solution can be shown by clicking on "Show details" link on the end of solution row.

This page is the same as for students with one exception -- there is an additional collapsed box "Set bonus points". In unfolded state, there is an input field for one number (positive or negative integer) and confirmation button "Set bonus points". After filling intended amount of points and submitting the form, the data in "Evaluation details" box get immediately updated. To remove assigned bonus points, submit just the zero number. The bonus points are not additive, newer value overrides older values.

It is useful to give a feedback about the solution back to the user. For this you can use the "Comments and notes" box. Make sure that the messages are not private, so that the student can see them. More detailed description of this box can be nicely used the "Comments and notes" box. Make sure that the messages are not private, so the student can see them. More detailed description of this box is available in student part of user documentation.

One of the discussed concept was marking one solution as accepted. However, due to lack of frontend developers it is not yet prepared in user interface. We hope, it will be ready as soon as possible. The button for accepting a solution will be most probably also on this page.

Creating Exercises

Link to exercise creation can be found in exercises list which is accessible through "Exercises" link in sidebar. On the bottom of the exercises list page you can find "Add exercise" button which will redirect you to exercise editation page. In this moment exercise is already created so if you just leave this page exercise will stay in the database. This is also reason why exercise creation form is the same as the exercise editation form.

Exercise editation page is divided into three separate forms. First one is supposed to contain meta information about exercise, second one is used for uploading and management of supplementary files and third one manages runtime configuration in which exercise can be executed.

First form is located in "Edit exercise settings" and generally contains meta information needed by frontend which are somehow somewhere visible. In here you can define:

• exercise name which will be visible to other supervisors
• difficulty of exercise (easy, medium, hard)
• description which will be available only for visitors, may be used for further description of exercise (for example information about test cases and how they could be scored)
• private/public switch, if exercise is private then only you as author can see it, assign it or modify it
• subform containing localized descriptions of exercise, new one can be added with "Add language variant" button and current one deleted with "Remove this language"
  • language in which this particular description is in (Czech, English, German)
  • actual localized description of exercise

After all information is properly set form has to be submitted with "Edit settings" button.

Management of supplementary files can be found in "Supplementary files" box. Supplementary files are files which you can use further in job configurations which have to be provided in all runtime configurations. These files are uploaded directly to fileserver from where worker can download them and use during execution according to job configuration.

Files can be uploaded either by drag and drop mechanism or by standard "Add a file" button. In opened dialog window choose file which should be uploaded. All chosen files are immediately uploaded to server but to save supplementary files list you have to hit "Save supplementary files" button. All previously uploaded files are visible right under drag and drop area, please note that files are stored on fileserver and cannot be deleted after upload.

The last form on exercise editation page is runtime configurations editation form. Exercise can have multiple runtime configurations according to the number of programming languages in which it can be run. Every runtime configuration corresponds to one programming language because all of them has to have a bit different job configuration.

New runtime configuration can be added with "Add new runtime configuration" button this will spawn new tab in runtime configurations box. In here you can fill following:

• human readable identifier of runtime configuration
• runtime environment which corresponds to programming language
• job configuration in YAML, detailed description of job configuration can be found further in this chapter in "Writing job configuration" section

If you are done with changes to runtime configurations save form with "Change runtime configurations" button. If you want to delete some particular runtime just hit "Remove" button in the right tab, please note that after this operation runtime configurations form has to be again saved to apply changes.

All runtime configurations which were added to exercise will be visible to supervisors and all can be used in assignment, so please be sure that all of the languages and job configurations are working.

If you choose to delete exercise, at the bottom of the exercise editation page you can find "Delete the exercise" box where "Delete" button is located. By clicking on it exercise will be delete from the exercises list and will no longer be available.

Reference Solutions of An Exercise

Each exercise should have a set of reference solutions, which are used to tune time and memory limits of assignments. Values of used time and memory for each solution are displayed in yellow boxes under forms for setting assignment limits as described earlier.

However, there is currently no user interface to upload and evaluate reference solutions. It is possible to use direct REST API calls, but it is not much user friendly. If you are interested, please look at API documentation, notably sections Uploaded-Files and Reference-Exercise-Solutions. You need to upload the reference solution files, create a new reference solution and then evaluate the solution. After that, measured data will be available in the box at assignment editing page (setting limits section).

We are now working on a better user interface, which will be available soon. Then its description will be added here.

Group Administrator

Group administrator is the group supervisor with some additional permissions in particular group. Namely group administrator is capable of creating a subgroups in managed group and also adding and deleting supervisors. Administrator of the particular group can be only one person.

Creating Subgroups And Managing Supervisors

There is no special link which will get you to groups in which you are administrator. So you have to get there through "Groups - supervisor" link in sidebar and choose the right group detail page. If you are there you can see "Administrator controls" section, here you can either add supervisor to group or create new subgroup.

Form for creating a subgroup is present right on the group detail page in "Add subgroup" box. Group can be created with following options:

• name which will be visible in group hierarchy
• external identification, can be for instance ID of group from school system
• some brief description about group
• allow or deny users to see each others statistics from assignments

After filling all the information a group can be created by clicking on "Create new group" button. If creation is successful then the group is visible in "Groups hierarchy" box on the top of page. All information filled during creation can be later modified.

Adding a supervisor to a group is rather easy, on group detail page is an "Add supervisor" box which contains text field. In there you can type name or username of any user from system. After filling user name, click on the magnifier image or press the enter key and all suitable users are searched. If your chosen supervisor is in the updated list then just click on the "Make supervisor" button and new supervisor should be successfully set.

Also, existing supervisor can be removed from the group. On the group detail page there is "Supervisors" box in which all supervisors of the group are visible. If you are the group administrator, you can see there "Remove supervisor" buttons right next to supervisors names. After clicking on it some particular supervisor should not to be supervisor of the group anymore.

Instance Administrator

Instance administrator can be only one person per instance. In addition to previous roles this administrator should be able to modify the instance details, manage licences and take care of top level groups which belong to the instance.

Instance Management

List of all instances in the system can be found under "Instances" link in the sidebar. On that page there is a table of instances with their respective admins. If you are one of them, you can visit its page by clicking on the instance name. On the instance details page you can find a description of the instance, current groups hierarchy and a form for creating a new group.

If you want to change some of the instance settings, follow "Edit instance" link on the instance details page. This will take you to the instance editation page with corresponding form. In there you can fill following information:

• name of the instance which will be visible to every other user
• brief description of instance and for whom it is intended
• checkbox if instance is open or not which means public or private (hidden from potential users)

If you are done with your editation, save filled information by clicking on "Update instance" button.

If you go back to the instance details page you can find there a "Create new group" box which is able to add a group to the instance. This form is the same as the one for creating subgroup in already existing group so we can skip description of the form fields. After successful creation of the group it will appear in "Groups hierarchy" box at the top of the page.

Licences

On the instance details page, there is a box "Licences". On the first line, it shows it this instance has currently valid licence or not. Then, there are multiple lines with all licences assigned to this instance. Each line consists of a note, validity status (if it is valid or revoked by superadministrator) and the last date of licence validity.

A box "Add new licence" is used for creating new licences. Required fields are the note and the last day of validity. It is not possible to extend licence lifetime, a new one should be generated instead. It is possible to have more than one valid licence at a time. Currently there is no user interface for revoking licences, this is done manually by superadministrator. If an instance is to be disabled, all valid licences have to be revoked.

Superadministrator

Superadministrator is a user with the most privileges and as such superadmin should be quite a unique role. Ideally, there should be only one user of this kind, used with special caution and adequate security. With this stated it is obvious that superadmin can perform any action the API is capable of.

Management of Users

There are only a few user roles in ReCodEx. Basically there are only three: student, supervisor, and superadmin. Base role is student which is assigned to every registered user. Roles are stored in database alongside other information about user. One user always has only one role at the time. At first startup of ReCodEx, the administrator has to change the role for his/her account manually in the database. After that manual intervention into database should never be needed.

There is a little catch in groups and instances management. Groups can have admins and supervisors. This setting is valid only per one particular group and has to be separated from basic role system. This implies that supervisor in one group can be student in another and simultaneously have global supervisor role. Changing role from student to supervisor and back is done automatically when the new privileges are granted to the user, so managing roles by hand in database is not needed. Previously stated information can be applied to instances as well, but instances can only have admins.

Roles description:

• Student -- Default role which is used for newly created accounts. Student can join or leave public groups and submit solutions of assigned exercises.
• Supervisor -- Inherits all permissions from student role. Can manage groups to which he/she belongs to. Supervisor can also view and change groups details, manage assigned exercises, view students in group and their solutions for assigned exercises. On top of that supervisor can create/delete groups too, but only as subgroup of groups he/she belongs to.
• Superadmin -- Inherits all permissions from supervisor role. Most powerful user in ReCodEx who should be able to do access any functionality provided by the application.

Writing Score Configuration

An important thing about assignment is how to assign points to particular solutions. As mentioned previously, the whole job is composed of logical tests. All of these tests have to contain one essential "evaluation" task. Evaluation task should output one float number which can be further used for scoring of particular tests.

Total resulting score of the students solution is then calculated according to a supplied score config (described below) and using specified calculator. Total score is also a float between 0 and 1. This number is then multiplied by the maximum of points awarded for the assignment by the teacher assigning the exercise -- not the exercise author.

For now, there is only one way how to write score configuration using only simple score calculator. But the implementation in API is agile enough to handle upcoming score calculators which might use some more complex scoring algorithms. This also means that future calculators do not have to use the YAML format for configuration. In fact, the configuration can be a string in any format.

Simple Score Calculation

First implemented calculator is simple score calculator with test weights. This calculator just looks at the score of each test and put them together according to the test weights specified in assignment configuration. Resulting score is calculated as a sum of products of score and weight of each test divided by the sum of all weights. The algorithm in Python would look something like this:

sum = 0
weightSum = 0
for t in tests:
  sum += t.score * t.weight
  weightSum += t.weight
score = sum / weightSum

Sample score config in YAML format:

testWeights:
  a: 300   # test with id 'a' has a weight of 300
  b: 200
  c: 100
  d: 100