LEdoian
/
recodex-wiki
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Remove useless stuff

Browse Source
master
Petr Stefan 8 years ago
parent 05683c18fb
commit d7c928fa47
1 changed files with 38 additions and 320 deletions
Show Stats Download Patch File Download Diff File

358
Rewritten-docs.md
Unescape Escape View File

@ -2571,7 +2571,9 @@ used.
# The Backend
# Implementation
## The backend
The backend is the part which is hidden to the user and which has only
one purpose: evaluate user’s solutions of their assignments.
@ -2583,8 +2585,6 @@ one purpose: evaluate user’s solutions of their assignments.
@todo: describe how the backend receives the inputs and how it
communicates the results
## Components
Whole backend is not just one service/component, it is quite complex system on its own.
@todo: describe the inner parts of the Backend (and refer to the Wiki
@ -2594,6 +2594,11 @@ for the technical description of the components)
@todo: gets stuff done, single point of failure and center point of ReCodEx universe
@todo: what to mention:
- job scheduling, worker queues
- API notification using curl, authentication using HTTP Basic Auth
- asynchronous resending progress messages
### Fileserver
@todo: stores particular data from frontend and backend, hashing, HTTP API
@ -2601,219 +2606,16 @@ for the technical description of the components)
### Worker
@todo: describe a bit of internal structure in general
- two threads
- number of ZeroMQ sockets, using it also for internal communication
- how sandboxes are fitted into worker, unix syscalls, #ifndef
- libcurl for fetchning, why not to use some object binding
- working with local filesystem, directory structure
- hardware groups in detail
@todo: describe how jobs are generally executed
### Monitor
@todo: not necessary component which can be omitted, proxy-like service
## Backend internal communication
@todo: internal backend communication, what communicates with what and why
The Frontend
============
The frontend is the part which is visible to the user of ReCodEx and
which holds the state of the system – the user accounts, their roles in
the system, the database of exercises, the assignments of these
exercises to groups of users (i.e., students), and the solutions and
evaluations of them.
Frontend is split into three parts:
- the server-side REST API (“API”) which holds the business logic and
keeps the state of the system consistent
- the relational database (“DB”) which persists the state of the
system
- the client side application (“client”) which simplifies access to
the API for the common users
The centerpiece of this architecture is the API. This component receives
requests from the users and from the Backend, validates them and
modifies the state of the system and persists this modified state in the
DB.
We have created a web application which can communicate with the API
server and present the information received from the server to the user
in a convenient way. The client can be though any application, which can
send HTTP requests and receive the HTTP responses. Users can use general
applications like [cURL](https://github.com/curl/curl/),
[Postman](https://www.getpostman.com/), or create their own specific
client for ReCodEx API.
Frontend capabilities
---------------------
@todo: describe what the frontend is capable of and how it really works,
what are the limitations and how it can be extended
Terminology
-----------
This project was created for the needs of a university and this fact is
reflected into the terminology used throughout the Frontend. A list of
important terms’ definitions follows to make the meaning unambiguous.
### User and user roles
*User* is a person who uses the application. User is granted access to
the application once he or she creates an account directly through the
API or the web application. There are several types of user accounts
depending on the set of permissions – a so called “role” – they have
been granted. Each user receives only the most basic set of permissions
after he or she creates an account and this role can be changed only by
the administrators of the service:
- *Student* is the most basic role. Student can become member of a
group and submit his solutions to his assignments.
- *Supervisor* can be entitled to manage a group of students.
Supervisor can assign exercises to the students who are members of
his groups and review their solutions submitted to
these assignments.
- *Super-admin* is a user with unlimited rights. This user can perform
any action in the system.
There are two implicit changes of roles:
- Once a *student* is added to a group as its supervisor, his role is
upgraded to a *supervisor* role.
- Once a *supervisor* is removed from the lasts group where he is a
supervisor then his role is downgraded to a *student* role.
These mechanisms do not prevent a single user being a supervisor of one
group and student of a different group as supervisors’ permissions are
superset of students’ permissions.
### Login
*Login* is a set of user’s credentials he must submit to verify he can
be allowed to access the system as a specific user. We distinguish two
types of logins: local and external.
- *Local login* is user’s email address and a password he chooses
during registration.
- *External login* is a mapping of a user profile to an account of
some authentication service (e.g., [CAS](https://ldap1.cuni.cz/)).
### Instance
*An instance* of ReCodEx is in fact just a set of groups and user
accounts. An instance should correspond to a real entity as a
university, a high-school, an IT company or an HR agency. This approach
enables the system to be shared by multiple independent organizations
without interfering with each other.
Usage of the system by the users of an instance can be limited by
possessing a valid licence. It is up to the administrators of the system
to determine the conditions under which they will assign licences to the
instances.
### Group
*Group* corresponds to a school class or some other unit which gathers
users who will be assigned the same set exercises. Each group can have
multiple supervisors who can manage the students and the list of
assignments.
Groups can form a tree hierarchy of arbitrary depth. This is inspired by the
hierarchy of school classes belonging to the same subject over several school
years. For example, there can be a top level group for a programming class that
contains subgroups for every school year. These groups can then by divided into
actual student groups with respect to lab attendance. Supervisors can create
subgroups of their groups and further manage these subgroups.
### Exercise
*An exercise* consists of textual assignment of a task and a definition
of how a solution to this exercise should be processed and evaluated in
a specific runtime environment (i.e., how to compile a submitted source
code and how to test the correctness of the program). It is a template
which can be instantiated as an *assignment* by a supervisor of a group.
### Assignment
An assignment is an instance of an *exercise* assigned to a specific
*group*. An assignment can modify the text of the task assignment and it
has some additional information which is specific to the group (e.g., a
deadline, the number of points gained for a correct solution, additional
hints for the students in the assignment). The text of the assignment
can be edited and supervisors can translate the assignment into another
language.
### Solution
*A solution* is a set of files which a user submits to a given
*assignment*.
### Submission
*A submission* corresponds to a *solution* being evaluated by the
Backend. A single *solution* can be submitted repeatedly (e.g., when the
Backend encounters an error or when the supervisor changes the assignment).
### Evaluation
*An evaluation* is the processed report received from the Backend after
a *submission* is processed. Evaluation contains points given to the
user based on the quality of his solution measured by the Backend and
the settings of the assignment. Supervisors can review the evaluation
and add bonus points (both positive and negative) if the student
deserves some.
### Runtime environment
*A runtime environment* defines the used programming language or tools
which are needed to process and evaluate a solution. Examples of a
runtime environment can be:
- *Linux + GCC*
- *Linux + Mono*
- *Windows + .NET 4*
- *Bison + Yacc*
### Limits
A correct *solution* of an *assignment* has to pass all specified tests (mostly
checks that it yields the correct output for various inputs) and typically must
also be effective in some sense. The Backend measures the time and memory
consumption of the solution while running. This consumption of resources can be
*limited* and the solution will receive fewer points if it exceeds the given
limits in some test cases defined by the *exercise*.
User management
---------------
@todo: roles and their rights, adding/removing different users, how the
role of a specific user changes
Instances and hierarchy of groups
---------------------------------
@todo: What is an instance, how to create one, what are the licences and
how do they work. Why can the groups form hierarchies and what are the
benefits – what it means to be an admin of a group, hierarchy of roles
in the group hierarchy.
Exercises database
------------------
@todo: How the exercises are stored, accessed, who can edit what
### Creating a new exercise
@todo Localized assignments, default settings
### Runtime environments and hardware groups
@todo read this later and see if it still makes sense
#### Runtime environments
ReCodEx is designed to utilize a rather diverse set of workers -- there can be
differences in many aspects, such as the actual hardware running the worker
@ -2837,69 +2639,28 @@ However, limits can differ between runtime environments -- formally speaking,
limits are a function of three arguments: an assignment, a hardware group and a
runtime environment.
### Reference solutions
@todo: how to add one, how to evaluate it
The task of determining appropriate resource limits for exercises is difficult
to do correctly. To aid exercise authors and group supervisors, ReCodEx supports
assigning reference solutions to exercises. Those are example programs that
should cover the main approaches to the implementation. For example, searching
for an integer in an ordered array can be done with a linear search, or better,
using a binary search.
Reference solutions can be evaluated on demand, using a selected hardware group.
The evaluation results are stored and can be used later to determine limits. In
our example problem, we could configure the limits so that the linear
search-based program doesn't finish in time on larger inputs, but a binary
search does.
Note that separate reference solutions should be supplied for all supported
runtime environments.
### Exercise assignments
@todo: Creating instances of an exercise for a specific group of users,
capabilities of settings. Editing limits according to the reference
solution.
Evaluation process
------------------
@todo: How the evaluation process works on the Frontend side.
### Uploading files and file storage
@todo: One by one upload endpoint. Explain different types of the
Uploaded files.
### Automatic detection of the runtime environment
@todo: Users must submit correctly named files – assuming the RTE from
the extensions.
REST API implementation
-----------------------
@todo: What is the REST API, what are the basic principles – GET, POST,
Headers, JSON.
### Monitor
### Authentication and authorization scopes
@todo: not necessary component which can be omitted, proxy-like service
@todo: How authentication works – signed JWT, headers, expiration,
refreshing. Token scopes usage.
### Cleaner
### HTTP requests handling
@todo: if it is something what to say here
@todo: Router and routes with specific HTTP methods, preflight, required
headers
## The frontend
### HTTP responses format
### REST API
@todo: Describe the JSON structure convention of success and error
responses
@todo: what to mention
- basic - GET, POST, JSON, Header, ...
- endpoint structure, Swager UI
- handling requests, preflight, checking roles with annotation
- Uploading files and file storage - one by one upload endpoint. Explain
different types of the Uploaded files.
- Automatic detection of the runtime environment - users must submit
correctly named files, assuming the RTE from the extensions
### Used technologies
#### Used technologies
@todo: PHP7 – how it is used for typehints, Nette framework – how it is
used for routing, Presenters actions endpoints, exceptions and
@ -2909,7 +2670,7 @@ problem with the extension and how we reported it and how to treat it in
the future when the bug is solved. Relational database – we use MariaDB,
Doctine enables us to switch the engine to a different engine if needed
### Data model
#### Data model
@todo: Describe the code-first approach using the Doctrine entities, how
the entities map onto the database schema (refer to the attached schemas
@ -2928,65 +2689,22 @@ grouping of entities and how they are related:
@todo: Tell the user about the generated API reference and how the
Swagger UI can be used to access the API directly.
Web Application
---------------
@todo: What is the purpose of the web application and how it interacts
with the REST API.
### Web application
### Used technologies
@todo: what to mention:
- used libraries, JSX, ...
- usage in user doc
- server side rendering
- maybe more ...
@todo: Briefly introduce the used technologies like React, Redux and the
build process. For further details refer to the GitHub wiki
### How to use the application
@todo: Describe the user documentation and the FAQ page.
Backend-Frontend communication protocol
=======================================
## Communication protocol
@todo: describe the exact methods and respective commands for the
communication
Initiation of a job evaluation
------------------------------
@todo: How does the Frontend initiate the evaluation and how the Backend
can accept it or decline it
Job processing progress monitoring
----------------------------------
When evaluating a job the worker sends progress messages on predefined points of
evaluation chain. The sending place can be on very beginning of the job, when
submit archive is downloaded or at the end of each simple task with its state
(completed, failed, skipped). These messages are sent to broker through existing
ZeroMQ connection. Detailed format of messages can be found on [communication
page](https://github.com/ReCodEx/wiki/wiki/Overall-architecture#commands-from-worker-to-broker).
Broker only resends received progress messages to the monitor component via
ZeroMQ socket. The output message format is the same as the input format.
Monitor parses received messages to JSON format, which is easy to work with in
JavaScript inside web application. All messages are cached (one queue per job)
and can be obtained multiple times through WebSocket communication channel. The
cache is cleared 5 minutes after receiving last message.
Publishing of the results
-------------------------
After job finish the worker packs results directory into single archive and
uploads it to the fileserver through HTTP protocol. The target URL is obtained
from API in headers on job initiation. Then "job done" notification request is
performed to API via broker. Special submissions (reference or asynchronous
submissions) are loaded immediately, other types are loaded on-demand on first
results request.
Loading results means fetching archive from fileserver, parsing the main YAML
file generated by worker and saving data to the database. Also, points are
assigned by score calculator.
<!---
// vim: set formatoptions=tqn flp+=\\\|^\\*\\s* textwidth=80 colorcolumn=+1:

Write Preview
Loading…
Cancel
Save