recodex-wiki/Web-API.md

# Web API

## Description

The Web API provides a controlled access to the evaluation backend. It also 
enables the use of different user frontends such as default web application, 
mobile applications, commandline tools and possibly others. The communication 
goes as HTTP(S) requests in predefined format, nowadays mostly known as 
[REST](https://en.wikipedia.org/wiki/Representational_state_transfer) format. 
Results from the API are in plain text in JSON format to be easily parsed in 
various languages (notably JavaScript).

This component must be publicly visible on the internet, so it is important to 
care about security and follow our recommendations. Security and user access 
restriction are among our primary concerns, so proper roles with permission 
separation are introduced and maintained. Also some additional checks are made 
directly in the code, so that a user cannot access information which are out of 
their authorization.

## Architecture

Web API is written in PHP using [Nette framework](https://nette.org/en/). This framework provides useful components like _Tracy_ for logging and showing errors, _Tester_ for productive unit testing or _Latte_ templating engine. Nette is modern, widely used and great performing software with active developers and user community. Nette can help eliminate security holes, simplify debugging and make coding easier with numerous plugins and extensions. Also, it is published under permissive BSD license.

API architecture consists of several parts:

- **router** -- component handling mapping from URL addresses to methods in presenter classes (called endpoints)
- **presenters** -- classes containing one method per endpoint responsible for 
  fetching and parsing request arguments and performing desired actions
- **entities** -- classes persisted using a database with an ORM framework
- **repositories** -- common operations on entities of one type, mostly finding entity by identifier or persisting changes to the database
- **helpers** -- set of classes solving more complicated internal logic, used from presenters to keep them reasonably small

Each presenter method has several annotations. They are used for generating REST API documentation in [Swagger](http://swagger.io/), specifying request type and its parameters and specifying one level of access restrictions. Also, there is simple description of the endpoint.

For specifying the request type (_GET_, _POST_, _DELETE_) annotations with exactly these names without any parameters are used. To describe request parameters `@Param` annotation is used with following arguments:

- type -- the type of argument, one of _post_ of _query_
- name -- name of the argument (the key)
- validation -- validation of the value, see [Nette validation rules](https://doc.nette.org/en/2.4/validators#toc-rules)
- msg -- description for users about the values this parameter can contain
- required -- specifies if this option is mandatory (`true`, default) or optional (`false`)
- description -- description for documentation of the API

Another annotation is `@LoggedIn` which takes no arguments. It can be placed before a whole class or before a method, so requests from unauthorized users are forbidden. Permissions can be granted or prohibitted by `@UserIsAllowed` annotation. This one is only per method and takes one argument in `key="value"` format. The value specifies which action (_value_) of a resource (_key_) the user needs to be allowed to perform this request. An example of how an annotated endpoint can look like:

```{.php}
/**
 * Create a user account
 * @POST
 * @LoggedIn
 * @UserIsAllowed(users="create")
 * @Param(type="post", name="email", validation="email",
 *    description="An email that will serve as a login name")
 * @Param(type="post", name="name", validation="string:2..",
 *    description="First name")
 * @Param(type="post", name="password", validation="string:1..",
 *    msg="Password cannot be empty.",
 *    description="A password for authentication")
 */
public function actionCreateAccount() {
    ...
}
```


The [Doctrine](https://github.com/Kdyby/Doctrine) 
[ORM](https://en.wikipedia.org/wiki/Object-relational_mapping) framework is used 
as an object persistence layer. It provides simple to use annotations to specify 
columns of database tables including types, indexes and also it is possible to 
make mapping between entities. For detailed info refer to [official 
documentation](http://docs.doctrine-project.org/projects/doctrine-orm/en/latest/).

The API is capable of sending email messages. They can inform an administrator about errors and users about submission evaluation or a temporary link to change forgotten password. The [Nette Mail](https://doc.nette.org/en/2.4/mailing) extension provides nice interface for sending messages through external SMTP server (**preferred**) or builtin PHP function `mail`. It is important to set up the mailserver properly to ensure message delivery to the clients. The messages are rendered in HTML format via simple _Latte_ templates.

### Authentication

Instead of relying on PHP sessions, we decided to use an authentication flow 
based on JWT tokens (RFC 7519). On successful login, the user is issued an 
access token that they have to send with subsequent requests using the HTTP 
Authorization header (`Authorization: Bearer <token>`). The token has a limited 
validity period and has to be renewed periodically using a dedicated API 
endpoint. 

To implement this behavior in Nette framework, a new `IUserStorage` 
implementation was created, along with an `IIdentity` and authenticators for 
both our internal login service and CAS.

An advantage of this approach is being able control the authentication process
completely instead of just receiving session data through a global variable.

## Installation

The web API requires a PHP runtime version at least 7. Which one depends on actual configuration, there is a choice between _mod_php_ inside Apache, _php-fpm_ with Apache or Nginx proxy or running it as standalone uWSGI script. It is common that there are some PHP extensions, that have to be installed on the system. Namely ZeroMQ binding (`php-zmq` package or similar), MySQL module (`php-mysqlnd` package) and ldap extension module for CAS authentication (`php-ldap` package). Make sure that the extensions are loaded in your `php.ini` file (`/etc/php.ini` or files in `/etc/php.d/`).

The API depends on some other projects and libraries. For managing them [Composer](https://getcomposer.org/) is used. It can be installed from system repositories or downloaded from the website, where detailed instructions are as well. Composer reads `composer.json` file in the project root and installs dependencies to the `vendor/` subdirectory. To do that, run:
```
$ composer install
```

## Configuration and usage

The API can be configured in `config.neon` and `config.local.neon` files in `app/config` directory. The first file is predefined by authors and should not be modified. The second one is not present and could be created by copying `config.local.neon.example` template in the config directory. Local configuration have higher precedence, so it will override default values from `config.neon`.

### Configurable items

Description of configurable items. All timeouts are in milliseconds if not stated otherwise.

- accessManager -- configuration of access token in [JWT standard](https://www.rfc-editor.org/rfc/rfc7519.txt). Do **not** modify unless you really know what are you doing.
- fileServer -- connection to fileserver
	- address -- URI of fileserver
	- auth -- _username_ and _password_ for HTTP basic authentication
	- timeouts -- _connection_ timeout for establishing new connection and _request_ timeout for completing one request
- broker -- connection to broker
	- address -- URI of broker
	- auth -- _username_ and _password_ for broker callback authentication back to API
	- timeouts -- _ack_ timeout for first response that broker receives the message, _send_ timeout how long try to send new job to the broker and _result_ timeout how long to wait for confirmation if job can be processed or not
- monitor -- connection to monitor
	- address -- URI of monitor
- CAS -- CAS external authentication
	- serviceId -- visible identifier of this service
	- ldapConnection -- parameters for connecting to LDAP, _hostname_, _base_dn_, _port_, _security_ and _bindName_
	- fields -- names of LDAP keys for informations as _email_, _firstName_ and _lastName_
- emails -- common configuration for sending email (addresses and template variables)
	- apiUrl -- base URL of api server including port (for referencing pictures in messages)
	- footerUrl -- link in the message footer
	- siteName -- name of frontend (ReCodEx, or KSP for unique instance for KSP course)
	- githubUrl -- URL to GitHub repository of this project
	- from -- sending email address
- failures -- admin messages on errors
	- emails -- additional info for sending mails, _to_ is admin mail address, _from_ is source address, _subjectPrefix_ is prefix of mail subject
- forgottenPassword -- user messages for changing passwords
	- redirectUrl -- URL of web application where the password can be changed
	- tokenExpiration -- expiration timeout of temporary token (in seconds)
	- emails -- additional info for sending mails, _from_ is source address and _subjectPrefix_ is prefix of mail subject
- mail -- configuration of sending mails
	- smtp -- using SMTP server, have to be "true"
	- host -- address of the server
	- port -- sending port (common values are 25, 465, 587)
	- username -- login to the server
	- password -- password to the server
	- secure -- security, values are empty for no security, "ssl" or "tls"
	- context -- additional parameters, depending on used mail engine. For examle self-signed certificates can be allowed as _verify_peer_ and _verify_peer_name_ to false and _allow_self_signed_ to true under _ssl_ key (see example).

Outside the parameters section of configuration is configuration for Doctrine. It is ORM framework which maps PHP objects (entities) into database tables and rows. The configuration is simple, required items are only _user_, _password_ and _host_ with _dbname_, i.e. address of database computer (mostly localhost) with name of ReCodEx database.

### Example local configuration file

```{.yml}
parameters:
  accessManager:
    leeway: 60
    issuer: https://recodex.projekty.ms.mff.cuni.cz
    audience: https://recodex.projekty.ms.mff.cuni.cz
    expiration: 86400  # 24 hours in seconds
    usedAlgorithm: HS256
    allowedAlgorithms:
      - HS256
    verificationKey: "recodex-123"
  fileServer:
    address: http://127.0.0.1:9999
    auth:
      username: "user"
      password: "pass"
    timeouts:
      connection: 500
  broker:
    address: tcp://127.0.0.1:9658
    auth:
      username: "user"
      password: "pass"
    timeouts:
      ack: 100
      send: 5000
      result: 1000
  monitor:
    address: wss://recodex.projekty.ms.mff.cuni.cz:4443/ws
  CAS:
    serviceId: "cas-uk"
    ldapConnection:
      hostname: "ldap.cuni.cz"
      base_dn: "ou=people,dc=cuni,dc=cz"
      port: 389
      security: SSL
      bindName: "cunipersonalid"
    fields:
      email: "mail"
      firstName: "givenName"
      lastName: "sn"
  emails:
    apiUrl: https://recodex.projekty.ms.mff.cuni.cz:4000
    footerUrl: https://recodex.projekty.ms.mff.cuni.cz
    siteName: "ReCodEx"
    githubUrl: https://github.com/ReCodEx
    from: "ReCodEx <noreply@example.com>"
  failures:
    emails:
      to: "Admin Name <admin@example.com>"
      from: %emails.from%
      subjectPrefix: "ReCodEx Failure Report - "
  forgottenPassword:
    redirectUrl: "https://recodex.projekty.ms.mff.cuni.cz/
                  forgotten-password/change"
    tokenExpiration: 600 # 10 minues
    emails:
      from: %emails.from%
      subjectPrefix: "ReCodEx Forgotten Password Request - "
  mail:
    smtp: true
    host: "smtp.ps.stdin.cz"
    port: 587
    username: "user"
    password: "pass"
    secure: "tls"
    context:
      ssl:
        verify_peer: false
        verify_peer_name: false
        allow_self_signed: true
doctrine:
  user: "user"
  password: "pass"
  host: localhost
  dbname: "recodex-api"
```

### Database preparation

When the API is installed and configured (_doctrine_ section is sufficient here) the database schema can be generated. There is a prepared command to do that from command line:

```
$ php www/index.php orm:schema-tool:update --force
```

With API comes some initial values, for example default user roles with proper permissions. To fill your database with these values there is another command line command:

```
$ php www/index.php db:fill
```

Check the outputs of both commands for errors. If there are any, try to clean temporary API cache in `temp/cache/` directory and repeat the action.


### Webserver configuration

The simplest way to get started is to start the built-in PHP server in the root directory of your project:

```
$ php -S localhost:4000 -t www
```

Then visit `http://localhost:4000` in your browser to see the welcome page of API project.

For Apache or Nginx, setup a virtual host to point to the `www/` directory of the project and you should be ready to go. It is **critical** that whole `app/`, `log/` and `temp/` directories are not accessible directly via a web browser (see [security warning](https://nette.org/security-warning)). Also it is **highly recommended** to set up a HTTPS certificate for public access to the API.

### Troubleshooting

In case of any issues first remove the Nette cache directory `temp/cache/` and try again. This solves most of the errors. If it does not help, examine API logs from `log/` directory of the API source or logs of your webserver.