From 7aeabd226e986bd9378d0ef3edea6bf4a2ab0cb8 Mon Sep 17 00:00:00 2001 From: Teyras Date: Mon, 30 Jan 2017 15:59:43 +0100 Subject: [PATCH] part of database implementation chapter --- Rewritten-docs.md | 168 ++++++++++++++++++++++++++++++++++++++++------ 1 file changed, 149 insertions(+), 19 deletions(-) diff --git a/Rewritten-docs.md b/Rewritten-docs.md index fae2b51..9e1f6cb 100644 --- a/Rewritten-docs.md +++ b/Rewritten-docs.md @@ -1612,7 +1612,7 @@ The Nette framework is an MVP (Model, View, Presenter) framework. It has many tools for creating complex websites and we need only a subset of them or we use different libraries which suite our purposes better: -- **Model** - the model layer is implemented using the Doctrine 2 ORM insead of +- **Model** - the model layer is implemented using the Doctrine 2 ORM instead of Nette Database - **View** - the whole view layer of the Nette framework (e.g., the Latte engine used for HTML template rendering) is unnecessary since we will return all the @@ -3078,6 +3078,10 @@ in the configuration file of the cleaner. ## REST API +The REST API is a PHP application run in an HTTP server. Its purpose is +providing controlled access to the evaluation backend and storing the state of +the application. + @todo: what to mention - basic - GET, POST, JSON, Header, ... - endpoint structure, Swager UI @@ -3089,27 +3093,153 @@ in the configuration file of the cleaner. ### Used Technologies -@todo: PHP7 – how it is used for typehints, Nette framework – how it is -used for routing, Presenters actions endpoints, exceptions and -ErrorPresenter, Doctrine 2 – database abstraction, entities and -repositories + conventions, Communication over ZMQ – describe the -problem with the extension and how we reported it and how to treat it in -the future when the bug is solved. Relational database – we use MariaDB, -Doctine enables us to switch the engine to a different engine if needed +We chose to use PHP in version 7.0, which was the most recent version at the +time of starting the project. The most notable new feature is optional static +typing of function parameters and return values. We use this as much as possible +to enable easy static analysis with tools like PHPStan. Using static analysis +leads to less error-prone code that does not need as many tests as code that +uses duck typing and relies on automatic type conversions. We aim to keep our +codebase compatible with new releases of PHP. + +To speed up the development and to make it easier to follow best practices, we +decided to use the Nette framework. The framework itself is focused on creating +applications that render HTML output, but a lot of its features can be used in a +REST application, too. + +Doctrine 2 ORM is used to provide a layer of abstraction over storing objects in +a database. This framework also makes it possible to change the database server. +The current implementation uses MariaDB, an open-source fork of MySQL. + +To communicate with the evaluation backend, we need to use ZeroMQ. This +functionality is provided by the `php_zmq` plugin that is shipped with most PHP +distributions. ### Data model -@todo: Describe the code-first approach using the Doctrine entities, how -the entities map onto the database schema (refer to the attached schemas -of entities and relational database models), describe the logical -grouping of entities and how they are related: - -- user + settings + logins + ACL -- instance + licences + groups + group membership -- exercise + assignments + localized assignments + runtime - environments + hardware groups -- submission + solution + reference solution + solution evaluation -- comment threads + comments +We decided to use a code-first approach when designing our data model. This +approach is greatly aided by the Doctrine 2 ORM framework, which works with +entities -- PHP classes for which we specify which attributes should be +persisted in a database. The database schema is generated from the entity +classes. This way, the exact details of how our data is stored is a secondary +concern for us and we can focus on the implementation of the business logic +instead. + +The rest of this section is a description of our data model and how it relates +to the real world. All entities are stored in the `App\Model\Entity` namespace. +There are repository classes that are used to work with entities without calling +the Doctrine `EntityManager` directly. These are in the `App\Model\Repository` +namespace. + +#### User Account Management + +The `User` entity class contains data about users registered in ReCodEx. To +allow extending the system with additional authentication methods, login details +are stored in separate entities. There is the `Login` entity class which +contains a user name and password for our internal authentication system, and +the `ExternalLogin` entity class, which contains an identifier for an external login +service such as LDAP. Currently, each user can only have a single authentication +method (account type). The entity with login information is created along with +the `User` entity when a user signs up. If a user requests a password reset, a +`ForgottenPassword` entity is created for the request. + +A user needs a way to adjust settings such as their preferred language or theme. +This is the purpose of the `UserSettings` entity class. Each possible option has +its own attribute (database column). Current supported options are `darkTheme`, +`defaultLanguage` and `vimMode` + +Every user has a role in the system. The basic ones are student, supervisor and +administrator, but new roles can be created by adding `Role` entities. Roles can +have permissions associated with them. These associations are represented by +`Permission` entitites. Each permission consists of a role, resource, action and +an `isAllowed` flag. If the `isAllowed` flag is set to true, the permission is +positive (lets the role access the resource), and if it is false, it denies +access. The `Resource` entity contains just a string identifier of a resource +(e.g., group, user, exercise). Action is another string that describes what the +permission allows or denies for the role and resource (e.g., edit, delete, +view). + +The `Role` entity can be associated with a parent entity. If this is the case, +the role inherits all the permissions of its parent. + +All actions done by a user are logged using the `UserAction` entity for +debugging purposes. + +#### Instances and Groups + +Users of ReCodEx are divided into groups that correspond to school lab groups +for a single course. Each group has a textual name and description. It can have +a parent group so that it is possible to create tree hierarchies of groups. + +Group membership is realized using the `GroupMembership` entity class. It is a +joining entity for the `Group` and `User` entities, but it also contains +additional information, most importantly `type`, which helps to distinguish +students from group supervisors. + +Groups are organized into instances. Every `Instance` entity corresponds to an +organization that uses the ReCodEx installation, for example a university or a +company that organizes programming workshops. Every user and group belong to +exactly one instance (users choose an instance when they create their account). + +Every instance can be associated with multiple `Licence` entities. Licences are +used to determine whether an instance can be currently used (access to those +without a valid instance will be denied). They can correspond to billing periods +if needed. + +#### Exercises + +The `Exercise` entity class is used to represent exercises -- programming tasks +that can be assigned to student groups. It contains data that does not relate to +this "concrete" assignment, such as the name, version and a private description. + +Some exercise descriptions need to be translated into multiple languages. +Because of this, the `Exercise` entity is associated with the +`LocalizedAssignment` entity, one for each translation of the text. + +An exercise can support multiple programming runtime environments. These +environments are represented by `RuntimeEnvironment` entities. Apart from a name +and description, they contain details of the language and operating system that +is being used. There is also a list of extensions that is used for detecting +which environment should be used for student submissions. + +`RuntimeEnvironment` entities are not linked directly to exercises. Instead, +the `Exercise` entity has an M:N relation with the `SolutionRuntimeConfig`, +which is associated with `RuntimeEnvironment`. It also contains a path to a job +configuration file template that will be used to create a job configuration file +for the worker that processes solutions of the exercise. + +Resource limits are stored outside the database, in the job configuration file +template. + +#### Reference Solutions + +To make setting resource limits objectively possible for a potentially diverse +set of worker machines, there should be multiple reference solutions for every +exercise in all supported languages that can be used to measure resource usage +of different approaches to the problem on various hardware and platforms. + +Reference solutions are contained in `ReferenceSolution` entities. These +entities can have multiple `ReferenceSolutionEvaluation` entities associated +with them that link to evaluation results (`SolutionEvaluation` entity). Details +of this structure will be described in the section about student solutions. + +#### Assignments + +The `Assignment` entity is created from an `Exercise` entity when an exercise is +assigned to a group. Most details of the exercise can be overwritten (see the +reference documentation for a detailed overview). Additional information such as +deadlines or point values for individual tests is also configured for the +assignment and not for an exercise. + +Assignments can also have their own `LocalizedAssignment` entities. If the +assignment texts are not changed, they are shared between the exercise and its +assignment. + +Runtime configurations can be also changed for the assignment. This way, a +supervisor can for example alter the resource limits for the tests. + +#### Student Solutions + +#### Comment threads ### Request Handling