13 KiB
Web API
Description
The Web API provides a controlled access to the evaluation backend. It also enables the use of different user frontends such as default web application, mobile applications, commandline tools and possibly others. The communication goes as HTTP(S) requests in predefined format, nowadays mostly known as REST format. Results from the API are in plain text in JSON format to be easily parsed in various languages (notably JavaScript).
This component must be publicly visible in the internet, so it is important to care about security and follow our recommendations. Security and user access restriction are among our primary concerns, so proper roles with permission separation are introduced and maintained. Also some additional checks are made directly in the code, so that a user cannot access information which are out of their authorization.
Architecture
Web API is written in PHP using Nette framework. This framework provides useful components like Tracy for logging and showing errors, Tester for productive unit testing or Latte templating engine. Nette is modern, widely used and great performing software with active developers and user community. Nette can help eliminate security holes, simplify debugging and make coding easier with numerous plugins and extensions. Also, it is published under permissive BSD license.
API architecture consists of several parts:
- router -- component handling mapping from URL addresses to methods in presenter classes (called endpoints)
- presenters -- classes containing one method per endpoint responsible for fetching and parsing request arguments and performing desired actions
- entities -- classes persisted using a database with an ORM framework
- repositories -- common operations on entities of one type, mostly finding entity by identifier or persisting changes to the database
- helpers -- set of classes solving more complicated internal logic, used from presenters to keep them reasonably small
Each presenter method has several annotations. They are used for generating REST API documentation in Swagger, specifying request type and its parameters and specifying one level of access restrictions. Also, there is simple description of the endpoint.
For specifying the request type (GET, POST, DELETE) are used annotations with exactly these names without parameters. To describe request parameters is used @Param annotation with following arguments:
- type -- the type of argument, one of post of query
- name -- name of the argument (the key)
- validation -- validation of the value, see Nette validation rules
- msg -- description for users about the values this parameter can contain
- required -- specifies if this option is mandatory (
true, default) or optional (false) - description -- description for documentation of the API
Another annotation is @LoggedIn which takes no arguments. It can be placed before whole class or before a method, so requests from unauthorized users are forbidden. Permissions can be granted or prohibitted by @UserIsAllowed annotation. This one is only per method and takes one argument in format key="value". The value specifies which action (value) of a resource (key) user need to be allowed to perform this request. Example how an annotated endpoint can look like:
/**
* Create a user account
* @POST
* @LoggedIn
* @UserIsAllowed(users="create")
* @Param(type="post", name="email", validation="email",
* description="An email that will serve as a login name")
* @Param(type="post", name="name", validation="string:2..",
* description="First name")
* @Param(type="post", name="password", validation="string:1..",
* msg="Password cannot be empty.",
* description="A password for authentication")
*/
public function actionCreateAccount() {
...
}
Authorization of users is based on validating his access token, which is obtained by users on successful login. This token has predefined validity period and has to be renewed before expiration to stay logged in. The token is sent to the API in HTTP Authorization header as value in format Bearer <token>. Requests with invalid or malformed token are treated as anonymous access with all its restrictions.
The Doctrine ORM framework is used as an object persistence layer. It provides simple to use annotations to specify columns of database tables including types, indexes and also it is possible to make mapping between entities. For detailed info refer to official documentation.
The API is capable of sending email messages. They can inform administrator about errors and users about submission evaluation or temporary link to change forgotten password. The Nette Mail extension provides nice interface for sending messages through external SMTP server (preferred) or builtin PHP function mail. It is important to set up the mailserver properly to ensure message delivery to the clients. The messages are rendered in HTML format via simple Latte templates.
Installation
The web API requires a PHP runtime version at least 7. Which one depends on actual configuration, there is a choice between mod_php inside Apache, php-fpm with Apache or Nginx proxy or running it as standalone uWSGI script. Common thing is, that there are some PHP extensions, that have to be installed on the system. Namely ZeroMQ binding (php-zmq package or similar), MySQL module (php-mysqlnd package) and ldap extension module for CAS authentication (php-ldap package). Make sure that the extensions are loaded in your php.ini file (/etc/php.ini or files in /etc/php.d/).
The API depends on some other projects and libraries. For managing them is used Composer. It can be installed from system repositories or downloaded from the website, where are also detailed instructions. Composer reads composer.json file in project root and install dependencies to vendor/ subdirectory. To do that, run:
$ composer install
Configuration and usage
The API can be configured in config.neon and config.local.neon files in app/config directory. The first file is predefined by authors and should not be modified. The second one is not present and could be created from template config.local.neon.example in the config directory. Local configuration have higher precedence, so it will override default values from config.neon.
Configurable items
Description of configurable items. All timeouts are in milliseconds if not stated otherwise.
- accessManager -- configuration of access token in JWT standard. Do not modify unless you really know what are you doing.
- fileServer -- connection to fileserver
- address -- URI of fileserver
- auth -- username and password for HTTP basic authentication
- timeouts -- connection timeout for establishing new connection and request timeout for completing one request
- broker -- connection to broker
- address -- URI of broker
- auth -- username and password for broker callback authentication back to API
- timeouts -- ack timeout for first response that broker receives the message, send timeout how long try to send new job to the broker and result timeout how long to wait for confirmation if job can be processed or not
- monitor -- connection to monitor
- address -- URI of monitor
- CAS -- CAS external authentication
- serviceId -- visible identifier of this service
- ldapConnection -- parameters for connecting to LDAP, hostname, base_dn, port, security and bindName
- fields -- names of LDAP keys for informations as email, firstName and lastName
- emails -- common configuration for sending email (addresses and template variables)
- apiUrl -- base URL of api server including port (for referencing pictures in messages)
- footerUrl -- link in the message footer
- siteName -- name of frontend (ReCodEx, or KSP for unique instance for KSP course)
- githubUrl -- URL to GitHub repository of this project
- from -- sending email address
- failures -- admin messages on errors
- emails -- additional info for sending mails, to is admin mail address, from is source address, subjectPrefix is prefix of mail subject
- forgottenPassword -- user messages for changing passwords
- redirectUrl -- URL of web application where the password can be changed
- tokenExpiration -- expiration timeout of temporary token (in seconds)
- emails -- additional info for sending mails, from is source address and subjectPrefix is prefix of mail subject
- mail -- configuration of sending mails
- smtp -- using SMTP server, have to be "true"
- host -- address of the server
- port -- sending port (common values are 25, 465, 587)
- username -- login to the server
- password -- password to the server
- secure -- security, values are empty for no security, "ssl" or "tls"
- context -- additional parameters, depending on used mail engine. For examle self-signed certificates can be allowed as verify_peer and verify_peer_name to false and allow_self_signed to true under ssl key (see example).
Outside the parameters section of configuration is configuration for Doctrine. It is ORM framework which maps PHP objects (entities) into database tables and rows. The configuration is simple, required items are only user, password and host with dbname, i.e. address of database computer (mostly localhost) with name of ReCodEx database.
Example local configuration file
parameters:
accessManager:
leeway: 60
issuer: https://recodex.projekty.ms.mff.cuni.cz
audience: https://recodex.projekty.ms.mff.cuni.cz
expiration: 86400 # 24 hours in seconds
usedAlgorithm: HS256
allowedAlgorithms:
- HS256
verificationKey: "recodex-123"
fileServer:
address: http://127.0.0.1:9999
auth:
username: "user"
password: "pass"
timeouts:
connection: 500
broker:
address: tcp://127.0.0.1:9658
auth:
username: "user"
password: "pass"
timeouts:
ack: 100
send: 5000
result: 1000
monitor:
address: wss://recodex.projekty.ms.mff.cuni.cz:4443/ws
CAS:
serviceId: "cas-uk"
ldapConnection:
hostname: "ldap.cuni.cz"
base_dn: "ou=people,dc=cuni,dc=cz"
port: 389
security: SSL
bindName: "cunipersonalid"
fields:
email: "mail"
firstName: "givenName"
lastName: "sn"
emails:
apiUrl: https://recodex.projekty.ms.mff.cuni.cz:4000
footerUrl: https://recodex.projekty.ms.mff.cuni.cz
siteName: "ReCodEx"
githubUrl: https://github.com/ReCodEx
from: "ReCodEx <noreply@example.com>"
failures:
emails:
to: "Admin Name <admin@example.com>"
from: %emails.from%
subjectPrefix: "ReCodEx Failure Report - "
forgottenPassword:
redirectUrl: "https://recodex.projekty.ms.mff.cuni.cz/
forgotten-password/change"
tokenExpiration: 600 # 10 minues
emails:
from: %emails.from%
subjectPrefix: "ReCodEx Forgotten Password Request - "
mail:
smtp: true
host: "smtp.ps.stdin.cz"
port: 587
username: "user"
password: "pass"
secure: "tls"
context:
ssl:
verify_peer: false
verify_peer_name: false
allow_self_signed: true
doctrine:
user: "user"
password: "pass"
host: localhost
dbname: "recodex-api"
Database preparation
When the API is installed and configured (doctrine section is sufficient here) the database schema can be generated. There is a prepared command to do that from command line:
$ php www/index.php orm:schema-tool:update --force
With API comes some initial values, for example default user roles with proper permissions. To fill your database with these values there is another command line command:
$ php www/index.php db:fill
Check the outputs of both commands for errors. If there are any, try to clean temporary API cache in temp/cache/ directory and repeat the action.
Webserver configuration
The simplest way to get started is to start the built-in PHP server in the root directory of your project:
$ php -S localhost:4000 -t www
Then visit http://localhost:4000 in your browser to see the welcome page of API project.
For Apache or Nginx, setup a virtual host to point to the www/ directory of the project and you should be ready to go. It is critical that whole app/, log/ and temp/ directories are not accessible directly via a web browser (see security warning). Also it is highly recommended to set up a HTTPS certificate for public access to the API.
Troubleshooting
In case of any issues first remove the Nette cache directory temp/cache/ and try again. This solves most of the errors. If it does not help, examine API logs from log/ directory of the API source or logs of your webserver.