|
|
@ -1,7 +1,7 @@
|
|
|
|
Introduction
|
|
|
|
Introduction
|
|
|
|
============
|
|
|
|
============
|
|
|
|
|
|
|
|
|
|
|
|
Generally, there are a lot of different ways and opinions on how to teach people
|
|
|
|
Generally, there are many different ways and opinions on how to teach people
|
|
|
|
something new. However, most people agree that a hands-on experience is one of
|
|
|
|
something new. However, most people agree that a hands-on experience is one of
|
|
|
|
the best ways to make the human brain remember a new skill. Learning must be
|
|
|
|
the best ways to make the human brain remember a new skill. Learning must be
|
|
|
|
entertaining and interactive, with fast and frequent feedback. Some kinds of
|
|
|
|
entertaining and interactive, with fast and frequent feedback. Some kinds of
|
|
|
@ -38,22 +38,28 @@ Analysis
|
|
|
|
|
|
|
|
|
|
|
|
### Assignment
|
|
|
|
### Assignment
|
|
|
|
|
|
|
|
|
|
|
|
The major goal of this project is to create grading application that will be
|
|
|
|
The major goal of this project is to create a grading application that will be
|
|
|
|
used for programming classes at the Faculty of Mathematics and Physics, Charles
|
|
|
|
used for programming classes at the Faculty of Mathematics and Physics, Charles
|
|
|
|
University. However, the application should have modular design and be easily
|
|
|
|
University. However, the application should be designed in a modular fashion so
|
|
|
|
extendable, so other kinds of usage will be possible.
|
|
|
|
that it can be easily extended to make other ways of using it possible.
|
|
|
|
|
|
|
|
|
|
|
|
This project have a great starting point -- there is an old grading system used
|
|
|
|
The project has a great starting point -- there is an old grading system
|
|
|
|
these days at out university, so its mistakes and weaknesses can be adressed, and
|
|
|
|
currently used at our university, so its mistakes and weaknesses can be
|
|
|
|
a lot of teachers willing to use the new system. The ideas were collected both
|
|
|
|
adressed. Furthermore, many teachers are willing to use and test the new system.
|
|
|
|
from our personal experience with old system and from teachers' requests.
|
|
|
|
Following requirements were collected both from our personal experience with
|
|
|
|
|
|
|
|
CodEx and from teachers' requests.
|
|
|
|
|
|
|
|
|
|
|
|
**Common grading system features:**
|
|
|
|
**Basic grading system requirements:**
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
These are features that are necessary for any system for evaluation of
|
|
|
|
|
|
|
|
programming homework assignments used in a university programming course.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
@todo maybe group the requirements by role (student might want to do XYZ...)
|
|
|
|
|
|
|
|
|
|
|
|
- creating exercises including textual description, sample inputs and correct
|
|
|
|
- creating exercises including textual description, sample inputs and correct
|
|
|
|
reference outputs (for example "sum all numbers from given file and write the
|
|
|
|
reference outputs (for example "sum all numbers from given file and write the
|
|
|
|
result to the standard output")
|
|
|
|
result to the standard output")
|
|
|
|
- assign the exercise to a group of users with some additional properties set
|
|
|
|
- assigning the exercise to a group of users with some additional properties set
|
|
|
|
(deadlines, etc.)
|
|
|
|
(deadlines, etc.)
|
|
|
|
- user interface for interaction with the system, mainly for showing assigned
|
|
|
|
- user interface for interaction with the system, mainly for showing assigned
|
|
|
|
exercises, uploading solution sources and presenting evaluated results
|
|
|
|
exercises, uploading solution sources and presenting evaluated results
|
|
|
@ -63,23 +69,36 @@ from our personal experience with old system and from teachers' requests.
|
|
|
|
- user management with support of roles (at least two -- _student_ and
|
|
|
|
- user management with support of roles (at least two -- _student_ and
|
|
|
|
_supervisor_)
|
|
|
|
_supervisor_)
|
|
|
|
- administrative interface for manual checking of solutions, overriding
|
|
|
|
- administrative interface for manual checking of solutions, overriding
|
|
|
|
automatically assigned amount of points and view overall statistics about
|
|
|
|
automatically assigned amount of points and viewing of overall statistics
|
|
|
|
users
|
|
|
|
about users
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
CodEx satisfies all these requirements and a few more that originate from the
|
|
|
|
|
|
|
|
way courses are organized at our university -- for example, users have roles
|
|
|
|
|
|
|
|
(_student_, _supervisor_ and _administrator_) that determine their capabilities
|
|
|
|
|
|
|
|
in the system and students are divided into groups that correspond to lab
|
|
|
|
|
|
|
|
groups.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
However, further requirements arose during the ten year long lifetime of the old
|
|
|
|
|
|
|
|
system. There are not many ways to improve it from the perspective of a student,
|
|
|
|
|
|
|
|
but a lot of feature requests came from administrators and supervisors.
|
|
|
|
|
|
|
|
Collected ideas were mostly gathered from meetings with faculty staff involved
|
|
|
|
|
|
|
|
with the current system.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
**Drawbacks of CodEx:**
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
@todo merge this with the CodEx entry in "Related work"
|
|
|
|
|
|
|
|
|
|
|
|
Schools and educating institutions needs a bit specific setup. System
|
|
|
|
- there has to be a separate installation of the system for some courses (e.g.
|
|
|
|
configuration needs to reflect structure of courses and user hierarchy
|
|
|
|
Java programming and .NET programming) that do not use the same sandbox as the
|
|
|
|
(_student_, _supervisor_ and _administrator_). These requirement are satisfied
|
|
|
|
majority of courses.
|
|
|
|
with currently used system at the university, but there are some new ones, which
|
|
|
|
|
|
|
|
escalated during long production lifetime of old system. From student
|
|
|
|
|
|
|
|
perspective there are not many thing to improve, but a lot of them come from
|
|
|
|
|
|
|
|
administrator and supervisors. Collected ideas are mainly from meeting with
|
|
|
|
|
|
|
|
faculty staff involved in current system.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
**Wanted features for new system:**
|
|
|
|
**Requested features for the new system:**
|
|
|
|
|
|
|
|
|
|
|
|
- login through university authentication system
|
|
|
|
- logging in through a university authentication system (e.g. LDAP)
|
|
|
|
- support for multiple programming environments at once to avoid unacceptable
|
|
|
|
- support for multiple programming environments at once to avoid unacceptable
|
|
|
|
workload for administrator and hardware occupation
|
|
|
|
workload for administrator and hardware occupation
|
|
|
|
|
|
|
|
- elimination of the need to maintain separate installations for Java and C#
|
|
|
|
|
|
|
|
programming courses
|
|
|
|
- localization (both UI and exercises)
|
|
|
|
- localization (both UI and exercises)
|
|
|
|
- Markdown support for exercise texts
|
|
|
|
- Markdown support for exercise texts
|
|
|
|
- tagging exercises and search by tags
|
|
|
|
- tagging exercises and search by tags
|
|
|
@ -88,7 +107,8 @@ faculty staff involved in current system.
|
|
|
|
- resubmit solution with saving all (including temporary) results
|
|
|
|
- resubmit solution with saving all (including temporary) results
|
|
|
|
- mark one student solution as accepted (used for grading this assignment)
|
|
|
|
- mark one student solution as accepted (used for grading this assignment)
|
|
|
|
- web and command-line submit tool
|
|
|
|
- web and command-line submit tool
|
|
|
|
- SIS integration for fetching personal user data
|
|
|
|
- SIS (university information system) integration for fetching personal user
|
|
|
|
|
|
|
|
data
|
|
|
|
- plagiarism detection
|
|
|
|
- plagiarism detection
|
|
|
|
- advanced low-level evaluation flow configuration with high-level abstraction
|
|
|
|
- advanced low-level evaluation flow configuration with high-level abstraction
|
|
|
|
layer for ordinary configuration cases
|
|
|
|
layer for ordinary configuration cases
|
|
|
@ -105,7 +125,7 @@ To find out current state in the field of automatic grading systems, let's do a
|
|
|
|
short survey at universities, programming contests or online tools.
|
|
|
|
short survey at universities, programming contests or online tools.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
### Related projects
|
|
|
|
### Related work
|
|
|
|
|
|
|
|
|
|
|
|
First of all, some code evaluating projects were found and examined. It is not
|
|
|
|
First of all, some code evaluating projects were found and examined. It is not
|
|
|
|
a complete list of such evaluators, but just a few projects which are used
|
|
|
|
a complete list of such evaluators, but just a few projects which are used
|
|
|
|