|
|
@ -256,42 +256,69 @@ addons (mostly administrative features).
|
|
|
|
|
|
|
|
|
|
|
|
#### End user requirements
|
|
|
|
#### End user requirements
|
|
|
|
|
|
|
|
|
|
|
|
- group hierarchy -- @todo: copy text from
|
|
|
|
- _group hierarchy_ -- creating an arbitrarily nested tree structure should be
|
|
|
|
[here](https://github.com/ReCodEx/wiki/wiki/Rewritten-docs/87e4bcd39a4fca3eadbb4748e9a3b6ced2bd7150#intended-usage)
|
|
|
|
supported to allow keeping related groups together, such as in the example
|
|
|
|
- there is a database of exercises; teachers can create exercises including
|
|
|
|
below. A group hierarchy also allows archiving data from past courses.
|
|
|
|
textual description, sample inputs and correct reference outputs (for example
|
|
|
|
|
|
|
|
"sum all numbers from given file and write the result to the standard output")
|
|
|
|
```
|
|
|
|
- teachers can specify way of computation grading points which will be awarded
|
|
|
|
Summer term 2016
|
|
|
|
to the students depending on the quality of his/her solution for each
|
|
|
|
|-- Language C# and .NET platform
|
|
|
|
assignment extra
|
|
|
|
| |-- Labs Monday 10:30
|
|
|
|
- teachers can view detailed data about their students (users of a their groups)
|
|
|
|
| `-- Labs Thursday 9:00
|
|
|
|
including all submitted solutions; also, each of the solution can be manually
|
|
|
|
|-- Programming I
|
|
|
|
reviewed, commented and assigned additional points (positive or negative)
|
|
|
|
| |-- Labs Monday 14:00
|
|
|
|
- one particular solution can be marked as accepted (used for grading this
|
|
|
|
...
|
|
|
|
assignment); otherwise, the solution with most points is used
|
|
|
|
```
|
|
|
|
- teacher can edit student solution and privately resubmit it; optionally saving
|
|
|
|
|
|
|
|
all results (including temporary ones)
|
|
|
|
- _a database of exercises_ -- teachers should be able to create exercises
|
|
|
|
- localization of all texts (UI and exercises)
|
|
|
|
including textual description, sample inputs and correct reference outputs
|
|
|
|
- Markdown support for creating exercise texts
|
|
|
|
(for example "sum all numbers from given file and write the result to the
|
|
|
|
- tagging exercises in database and search by these tags
|
|
|
|
standard output") and to browse this database
|
|
|
|
- comments of exercises, tests and solutions
|
|
|
|
- _customizable grading system_ -- teachers need to specify the way of
|
|
|
|
- plagiarism detection
|
|
|
|
computation of the final score, which will be awarded to the student's
|
|
|
|
|
|
|
|
submissions depending on their quality
|
|
|
|
|
|
|
|
- _viewing student details_ -- teachers should be able to view detailed data
|
|
|
|
|
|
|
|
about their students (members of their groups), including all submitted
|
|
|
|
|
|
|
|
solutions;
|
|
|
|
|
|
|
|
- _awarding additional points_ -- adding (or subtracting) points from the final
|
|
|
|
|
|
|
|
score of a submission by a supervisor must be supported
|
|
|
|
|
|
|
|
- _marking a solution as accepted_ -- the system should allow marking one
|
|
|
|
|
|
|
|
particular solution as accepted (used for grading the assignment) by the
|
|
|
|
|
|
|
|
supervisor
|
|
|
|
|
|
|
|
- _solution resubmission_ -- teachers should be able edit student's solutions
|
|
|
|
|
|
|
|
and privately resubmit them, optionally saving all results (including
|
|
|
|
|
|
|
|
temporary ones); this feature can be used to quickly fix errors in the
|
|
|
|
|
|
|
|
solution
|
|
|
|
|
|
|
|
- _localization_ -- all texts (UI and exercises) should be translatable
|
|
|
|
|
|
|
|
- _formatted exercise texts_ -- Markdown or another lightweight markup language
|
|
|
|
|
|
|
|
should be supported for formatting exercise texts
|
|
|
|
|
|
|
|
- _exercise tags_ -- the system should support tagging exercises searching by
|
|
|
|
|
|
|
|
these tags
|
|
|
|
|
|
|
|
- _comments_ -- adding both private and public comments to exercises, tests and
|
|
|
|
|
|
|
|
solutions should be supported
|
|
|
|
|
|
|
|
- _plagiarism detection_
|
|
|
|
|
|
|
|
|
|
|
|
#### Administrative requirements
|
|
|
|
#### Administrative requirements
|
|
|
|
|
|
|
|
|
|
|
|
- users can use an intuitive user interface for interaction with the system,
|
|
|
|
- _pluggable user interface_ -- the system should allow using an alternative
|
|
|
|
mainly for viewing assigned exercises, uploading their own solutions to the
|
|
|
|
user interface, such as a command line client; implementation of such clients
|
|
|
|
assignments, and viewing the results of the solutions after an automatic
|
|
|
|
should be as straightforward as possible
|
|
|
|
evaluation is finished; the two wanted interfaces are web and command-line
|
|
|
|
- _privilege separation_ -- there should be at least two roles -- _student_ and
|
|
|
|
based
|
|
|
|
_supervisor_. Cases when a student of a course is also a teacher of another
|
|
|
|
- user privilege separation (at least two roles -- _student_ and _supervisor_)
|
|
|
|
lab must be handled correctly
|
|
|
|
- logging in through a university authentication system (e.g. LDAP)
|
|
|
|
- _alternate authentication methods_ -- logging in through a university
|
|
|
|
- SIS (university information system) integration for fetching personal user
|
|
|
|
authentication system (e.g. LDAP) and potentially other services, such as
|
|
|
|
data
|
|
|
|
OAuth, should be supported
|
|
|
|
- safe environment in which the students' solutions are executed
|
|
|
|
- _querying SIS_ -- loading user data from the university information system
|
|
|
|
- support for multiple programming environments at once to avoid unacceptable
|
|
|
|
should be supported
|
|
|
|
workload for administrator (maintain separate installation for every course)
|
|
|
|
- _sandboxing_ -- there should be a safe environment in which the students'
|
|
|
|
and high hardware occupation
|
|
|
|
solutions are executed to prevent system failures due to malicious code being
|
|
|
|
|
|
|
|
submitted; the sandboxed environment should have the least possible impact on
|
|
|
|
|
|
|
|
measurement results (most importantly on measured times)
|
|
|
|
|
|
|
|
- _heterogenous worker pool_ -- there must be support for submission evaluation
|
|
|
|
|
|
|
|
in multiple programming environments in a single installation to avoid
|
|
|
|
|
|
|
|
unacceptable workload for the administrator (maintaining a separate
|
|
|
|
|
|
|
|
installation for every course) and high hardware occupation
|
|
|
|
- advanced low-level evaluation flow configuration with high-level abstraction
|
|
|
|
- advanced low-level evaluation flow configuration with high-level abstraction
|
|
|
|
layer for ordinary configuration cases; the configuration should be able to
|
|
|
|
layer for ordinary configuration cases; the configuration should be able to
|
|
|
|
express more complicated flows than just compiling a source code and running
|
|
|
|
express more complicated flows than just compiling a source code and running
|
|
|
@ -303,33 +330,36 @@ addons (mostly administrative features).
|
|
|
|
### Non-functional requirements
|
|
|
|
### Non-functional requirements
|
|
|
|
|
|
|
|
|
|
|
|
Non-functional requirements are requirements of technical character with no
|
|
|
|
Non-functional requirements are requirements of technical character with no
|
|
|
|
direct mapping to visible parts of system. In ideal word, users should not know
|
|
|
|
direct mapping to visible parts of the system. In an ideal world, users should
|
|
|
|
about these if they work properly, but would be at least annoyed if these
|
|
|
|
not know about these features if they work properly, but would be at least
|
|
|
|
requirements were not met. Most notably they are these ones:
|
|
|
|
annoyed if they did not.
|
|
|
|
|
|
|
|
|
|
|
|
- user interface of the system accessible on users' computers without
|
|
|
|
- _no installation_ -- the primary user interface of the system must be accessible
|
|
|
|
installation of any kind of additional software
|
|
|
|
on users' computers without the need to install any additional software
|
|
|
|
- easy implementation of different user interfaces
|
|
|
|
- _performace_ -- the system must be ready for at least hundreds of students and
|
|
|
|
- be ready for workload hundreds of students and tens of supervisors
|
|
|
|
tens of supervisors using it at once
|
|
|
|
- automated installation of all components
|
|
|
|
- _automated deployment_ -- all of the components of the system must be easy to
|
|
|
|
- source code with permissive license allowing further development; this also
|
|
|
|
deploy in an automated fashion
|
|
|
|
applies on used libraries and frameworks
|
|
|
|
- _open source licensing_ -- the source code should be released under a
|
|
|
|
- multi-platform worker supporting at least two major operating systems
|
|
|
|
permissive license allowing further development; this also applies to used
|
|
|
|
|
|
|
|
libraries and frameworks
|
|
|
|
|
|
|
|
- _multi-platform worker_ -- worker machines running Linux, Windows and
|
|
|
|
|
|
|
|
potentially other operating systems must be supported
|
|
|
|
|
|
|
|
|
|
|
|
### Conclusion
|
|
|
|
### Conclusion
|
|
|
|
|
|
|
|
|
|
|
|
The survey shows that there are a lot of different requirements and wishes for
|
|
|
|
The survey shows that there are a lot of different requirements and wishes for
|
|
|
|
the new system. When the system is ready it is likely that there will be new
|
|
|
|
the new system. When the system is ready, it is likely that there will be new
|
|
|
|
ideas of how to use the system and thus the system must be designed to be easily
|
|
|
|
ideas of how to use the system and thus the system must be designed to be easily
|
|
|
|
extendable, so everyone can develop their own feature. This also means that
|
|
|
|
extendable, so that these new ideas can be easily implemented, either by us or
|
|
|
|
widely used programming languages and techniques should be used, so users can
|
|
|
|
community members. This also means that widely used programming languages and
|
|
|
|
quickly understand the code and make changes.
|
|
|
|
techniques should be used, so that users can quickly understand the code and
|
|
|
|
|
|
|
|
make changes.
|
|
|
|
|
|
|
|
|
|
|
|
## Related work
|
|
|
|
## Related work
|
|
|
|
|
|
|
|
|
|
|
|
To find out the current state in the field of automatic grading systems we did a
|
|
|
|
To find out the current state in the field of automatic grading systems, we did
|
|
|
|
short market survey on the field of automatic grading systems at universities,
|
|
|
|
a short market survey on the field of automatic grading systems at universities,
|
|
|
|
programming contests, and possibly other places where similar tools are
|
|
|
|
programming contests, and possibly other places where similar tools are
|
|
|
|
available.
|
|
|
|
available.
|
|
|
|
|
|
|
|
|
|
|
|