Hermes documentation

Start here

Welcome to the official documentation for Hermes. This is a full searchable general documentation for Hermes. We also advise you checking out the Advanced section and Tutorials & How Tos section on the main Hermes-core documentation page.

About Hermes

Hermes — a framework for cryptographically assured access control and data security. It is a unified key/encryption management scheme that provides a number of security guarantees through access compartmentalization and distribution of CRUD permissions to data blocks cryptographically.

Hermes-core is a proof of concept implementation for Hermes — a practical data security scheme with the reference implementation, which enables distributed sharing and collaboration, enforcing access control cryptographically, while the maximum possible number of security guarantees for the protected data is preserved even in the case when one or more parts of the system are compromised.

Hermes in a nutshell

Hermes-core is a proof of concept for Hermes – a practical data security scheme with the reference implementation, which enables distributed sharing and collaboration, enforcing access control cryptographically, while the maximum possible number of security guarantees for the protected data is preserved even in the case when one or more parts of the system are compromised.

Hermes is an end-to-end encryption methodology for manipulating structured records with crypto-controlled CRUD rights. Hermes-core is a proof-of-concept implementation of Hermes as a standalone library, which can be easily embedded into a networked or local application.

Hermes maps privileges — read, write, update, delete — to cryptographic keys, in arbitrary granularity towards protected object’s structure. Hermes does that via cryptographic process, so no attack on Hermes’s execution flow can affect the protected data - in the worst case, it will lead to nothing worse than a denial of service.

Hermes relies on stacking several cryptographic processes (symmetric and asymmetric encryption without interactive key exchange, MACs) and managing keys/data structures/encryption scopes. It uses battle-proven cryptographic algorithms and their implementations and is built using special encryption abstraction library Themis, which allows swapping problematic crypto-primitives or even whole algorithm families without changing a line of high-level code.

Access is managed via private keys, sharing, and permission set assignments. These procedures rely on the use of Public Key Infrastructure. A bare-bones version of such infrastructure is aimed at easy integration using a simple flat file storage because Hermes is aimed at integration with any data and trust sources.

How Hermes works

Hermes operates with data that is subdivided into records (also named "blocks" in the Hermes-core implementation), which represent the hierarchy of recordsets and groups of recordsets (collections) all contained in the Data store (see illustration and explanation below). Each blob of data (record) is encrypted using a symmetric key, from which a set of hashes is generated. Possession of a symmetric key by a user allows reading and carrying out other processes on hashes (including with writing data).

Hermes-core is a basic building block for the hierarchic infrastructure of Hermes - in Hermes-core a record (block) is the smallest indivisible data entity that is not combined into higher level hierarchical data structures.

There are 3 storage entities in Hermes that constitute the Server side:

  • Data store contains the hierarchy of encrypted objects.
  • Credential store stores keys and hashes, asymmetrically encrypted in such a way that can only be decrypted by authorised user’s private key. Those can contain access control key which grants READ permission and Update Tag which allows performing WRITE operations.
  • Keystore contains the symmetric keys (for READ and UPDATE), with as many copies of these keys as there are users authorised to access the record, where every copy is wrapped (asymmetrically encrypted) with a public credential of the respective authorised user. If the permissions to READ and to WRITE extend to not just records, but to the list of records, they turn into permissions to DELETE/ADD elements. The 4th entity of Hermes is Client:

Client (or clients) is the active entity in the Hermes architecture, the one that actually produces or consumes the data. The Client only possesses the keypair that allows decrypting the asymmetrically encrypted data from the Server. The READ permissions are always checked on Client. The absence of the key for performing READ operations will not allow the Client to decrypt the downloaded piece of data. The WRITE permissions are checked both on Client and Server so they cannot “fool” each other.

Glossary of common Hermes' terminology

This is a table of common terms used to describe the processes and notions of Hermes (and, consequently, Hermes-core).

Term Definition
Record Atomic (granular) data unit within the Hermes’ scheme.
Recordset A document that consists of all records with appropriate Update tags.
Update tag (UT) Message authentication code of a record. A UT is used for the UPDATE authorisation process. Belongs to the Data store.
Access control key (ACK) Symmetric key that allows performing operations on data. Belongs to the Keystore.
Access control policy (ACP) A combination of access rules set for the specified records.
READ ACK (READ token) ACK for performing READ function on the data. Belongs to the Keystore.
UPDATE ACK (UPDATE token) Symmetric key for performing UPDATE function on the data. Belongs to the Keystore.
Key encryption key (KEK) Cryptographic key (symmetric or asymmetric) used for encrypting other cryptographic keys in the Hermes’ scheme. Belongs to the Keystore.

Quickstart

To see Hermes-core working, you need to perform the following actions.

Installing or building Hermes-core

You can build it manually from source, or install from the available package manager.

Running examples

  • Usage examples describe how examples work and what are possible usages for Hermes-core.

Checking high-level tutorials

Consider checking full tutorials to understand how to add and update blocks, grant READ and UPDATE access right for users, revoke access rights.

  • C tutorial, where both Hermes and client app are written in C.
  • Python tutorial, where Hermes app is C-based, but client code runs on Python.
  • Go tutorial, where Hermes app is C-based, but client code runs on Go.

The next steps

As your next steps towards exploring Hermes-core, we advise you to:

Hermes usage ideas

This documentation piece can be also subtitled as "Oh the Places You'll Go". The protection provided by Hermes enables developers to implement control mechanisms that limit the classic CRUD set of permissions on any arbitrary chunk of protected data (including with implementing arbitrary status flags, i.e. execution). It is done cryptographically, so no tampering with protected records changes their access rights.

When used correctly, Hermes allows slicing a single data structure into smaller parts and allowing different parties to access various parts of that data structure with different permissions. This significantly limits the attack surface and threat model on the data. What’s more important, no significant alteration of the infrastructure is necessary. The range of tools used by Hermes was built for complementing the existing relationship models, not for coming up with new ones. Built by expert cryptographers and software engineers, it combines strong security guarantees with an API that is easy to use.

What's so unique about Hermes

  • Access control list (ACL) in Hermes is cryptographic.
  • Hermes binds over (almost) any data structure.
  • Hermes works with any network/server/client layout (to Hermes it all is just data).

Hermes represents a practical scheme that takes into account a number of important requirements:

  • Presence of only strong end-to-end encryption between the entities;

  • Provision of enforced security simultaneously with access policy distribution to data, compared to algorithmic ways and operational guidelines;

  • Processing of sensitive data in plain text only in user’s context and storage of sensitive data on Server-side entities in encrypted form only.

When at least some of the aforementioned requirements are fulfilled, it significantly increases the security level of the system.

Some scenarios for using Hermes

A typical use-case for Hermes: an online medical service where the patients securely interact with the staff of multiple medical institutions. For example, a patient’s records contain a number of results for medical examinations. The patient’s personal physician needs to be able to access all of them, while the employees of the medical institutions need to be able to access only the latest results to adjust their activities towards the patient. Leaving access control management to a medical service opens up a risk of an accidental privilege misplacement or an intentional data leak. Being able to manage the access permissions only from trusted endpoints enables provable security for collaboration of such datasets.

Enforcing access control cryptographically
The access control distribution in Hermes is based on possession of cryptographic key and is much stronger than traditional operation guidelines and algorithmic mechanisms.

Provision of controlled secure data sharing in an environment with limited trust to server
Hermes allows using servers with limited trust level. A server is only used for storage of encrypted sensitive data.

Why Hermes does it better

  1. Sensitive data is only ever processed in plain text within the client execution environment. Server-side components only operate with encrypted sensitive data;
  2. Compromisation of a single server-side component causes only limited damage;
  3. Hermes only uses end-to-end encryption with mutual authentication between the entities;
  4. Hermes only uses well-examined, extremely secure cryptographic algorithms.

Strong cryptographic base

Hermes (and correspondingly, Hermes-core) is built on top of Themis, which is a high-level cryptographic services library that allows you to protect data at rest (in your database, files or wherever you store them in your application) and data in motion (e.g. travelling between client and server, server and server, application and application, etc.).

Understanding Hermes

Hermes model

Note: The following descriptions are aimed at helping you understand the general concept of Hermes.

The main goal of Hermes is to cryptographically enforce the data access control policy and to minimise the risks of compromisation of a single Hermes’ component (or a subset of such components). Such enforcements are achieved by:

  • reducing the risk of potential damage from a security breach by distributing the components of Hermes;
  • limiting/blocking the exposure to sensitive information for each component and in this way securing the perimeter of a potential security breach;
  • relying on strong cryptography for proper access control to sensitive information rather than on organisational operational guidelines and proper software implementation and operation;
  • using end-to-end encryption to protect communication channels between the components.

Hermes process

Hermes achieves these security goals by defining cryptographic processes for: - creating new records; - reading records; - updating records; - deleting records; - granting access rights/permissions; - revoking access rights/permissions;

in such a way that Hermes-generated structure can be accessed only through a Hermes Client with corresponding keys (belonging to the users that have corresponding permissions).

By using these processes through high-level API and implementing certain wrappers for Hermes-core abstractions and our databases/PKIs/etc., consumer applications can secure their data using Hermes-based access control.

Hermes entities

Typical implementations of Hermes include the following entities:

0) Your application that provides means for generating and displaying the data.

1) Client(s) (system users/services - referred to as users in the documentation): these are active entities in the architecture – the only ones that produce or consume the data. The users can be both real users or system processes, communicating with Hermes infrastructure via pre-defined API after presenting the keys.

2) Data store: a physical or logical unit for storage and distribution of secure data – for example, a relational database, a collection of documents, a filesystem. It is assumed that all protected data can be divided into pieces (or records). There are no restrictions imposed by Hermes on how to divide the data into records, it is up to the system user/service that creates the data to define and express this policy.

Hermes provides the ability to set individual access policy on a per-record basis. Data store only manipulates the encrypted data and does not have keys to decrypt it. However, the Data store execution environment is responsible for authorising record updates. This is done through storage and verification of a special Update Tag for each record.

Note: The smallest data entity recognised by Hermes-core is a record (sometimes called "block") that is not further divided into records. In Hermes, recordsets are divided into records or combined into groups/collections of various hierarchical levels.

3) Keystore: a physical or logical unit for storing data access control keys. It implements per-record data access control policy by storing and providing record READ/UPDATE keys. Keys are never processed in plain text and the Keystore itself doesn’t have any credentials that would allow obtaining the keys’ values. Instead, for each record, the Keystore has one or more copies of READ and UPDATE keys, where each copy is wrapped (encrypted with asymmetric ECDH-based scheme) with a public credential of the respective authorised client. It is assumed that UPDATE access implies READ access. The Keystore never denies key delivery to a respective user and always fetches the user’s key if it’s available. It’s expected that the user will unwrap the key using their private credentials before processing the actual data. The Keystore is populated by users upon distributing access control policy for the data they produce.

4) Credential store: a trusted source of user/service public credentials (and possible Keystore / Data store public credentials). Users use Credential store to wrap ACKs for specific users while distributing access control policy for their data. It’s expected that other entities communicate with the Credential store via authenticated channels.

You can read about Data store, Keystore, and Credential store in more details in Abstract entities.

The model of security for Hermes is described in Security model.

For a deeper understanding of the concepts behind Hermes, please read the scientific document on Hermes - "Hermes – a framework for cryptographically assured access control and data security".

Abstract entities (backend)

Hermes-core is divided into the 2 core components:

  • Backend or Server-side (Server/storage), which is outlined in this document,
  • Frontend or Client-side (Client), which is outlined in a separate block.

Storage

Storage entity must have 3 separate parts (the parts can be allocated in virtually any way, but an active compartmentalisation on the database's side adds an extra layer of security):

  1. Credential store,
  2. Data store,
  3. Keystore.

Credential store
Credential store is a trusted source of user/service public credentials (and possible key/data store public credentials). Users use the Credential store to wrap ACKs for specific users while distributing access control policy for their data. It’s expected that other entities communicate with the Credential store via authenticated channels.

Credential store is used for retrieving public keys by the user identifier (user ID). It can be as simple as a function or a table that maps public keys to user IDs to a fully-blown local certificate authority.

Hermes-core does not provide you with Credential store, instead, it suggests integrating with any PKI you've got (i.e. PKI can be a read-only set of keys).

Data store
Data store is the main global storage entity for storing sensitive information. In Hermes, each record (block) is stored in an appropriate recordset (document), which can be represented by the following fields:

  1. document ID
  2. block ID
  3. private metadata
  4. MAC
  5. data

Keystore
Keystore is used for storing access control keys/tokens (per user and block):

  1. block ID - ID of the data block for which the token is designated,
  2. user ID - ID of the user for which the token is designated,
  3. owner ID - ID of the user who issued the token,
  4. key/token type (read or update) - type of the permissions granted with the token,
  5. key/token - the actual key/token.

Each access control key ("token") is stored as a combination of separate tokens: block ID + user ID + token + owner ID. To be able to use the token, a user with the user ID needs his/her private key and a public user key with the owner ID.

Storage interfaces

The storage interfaces for Hermes-core are described in detail in the Hermes-core Server (Building app - Server) article in the documentation.

Transport

The components of Hermes-core need to have some kind of communication established between them: between Client and Server, between the Server components. This is done with the help of Transport.

The communication channel between the components should be encrypted and mutually authenticated. This can be done using the Secure Session from our Themis cryptographic library with our wrapper over raw transport. Since Secure Session is a stream-oriented protocol, the channel will be also protected from replay attacks and request reordering. Alternatively, you may choose to leave the communication channel unencrypted (which is not recommended for the reasons outlined above) or use a secure transport method of your choice (i.e. TLS, etc.).

Architecture

This section is dedicated to describing the architecture of Hermes-core components. Hermes-core uses a typical Client-Server interaction.

Client

The Client-side is represented by the user-facing app or service. The main Client-side components are implemented by mid_hermes_t class (can be found in mid_hermes/mid_hermes.h and mid_hermes/client/hermes_client.c) and carry out all the operations necessary for the functioning of Hermes-core. Moreover, the hermes-core repository includes wrappers for implementing Hermes-based applications with Python and Go.

Read in more details what actions are possible for client app in Hermes-core Client section.

Server

By default, the Server-side part of Hermes-core is represented by 3 physically independent standalone servers: Data store, Keystore, and Credential store. For more abstraction, Hermes-core doesn't include thread-dependent mechanisms. This is done because we assume that a Hermes-based application may be created using any modern programming language, most of which include more comfortable and portable thread control mechanisms than plain POSIX threads.

Read in more details what actions are possible for server app in Building app Server section.

Client-server communication

For the reasons of abstraction, Hermes-core doesn't include predefined communication mechanisms. Instead, Hermes-core contains an interface hm_rpc_transport_t (see include/hermes/rpc/transport.h), which includes the following function definitions:

typedef uint32_t(*hm_rpc_transport_send_t)(void *transport, const uint8_t *buffer, const size_t buffer_length);

typedef uint32_t(*hm_rpc_transport_recv_t)(void *transport, uint8_t *buffer, size_t buffer_length);

typedef uint32_t(*hm_rpc_transport_get_remote_id_t)(void *transport, uint8_t **id, size_t *id_length);

typedef struct hm_rpc_transport_type {
    hm_rpc_transport_send_t send;
    hm_rpc_transport_recv_t recv;
    hm_rpc_transport_get_remote_id_t get_remote_id;
    void *user_data;
} hm_rpc_transport_t;

It also includes a wrapper for transport implementations that uses Themis' Secure Session (see include/hermes/secure_transport/transport.h). This is done to provide a way to use another (non-Secure Sesion) means of secure transport (i.e. SSL/TLS).

  • Communication
    Implementation of this interface is enough for Hermes-core to function correctly.

  • Remote Procedure Call
    For abstraction and simplicity, all the components of Hermes-core use internal remote procedure call protocol.

Don't want to implement your own transport layer? Check Building app Transport section.

API description and processes

Hermes-core API is a convenient library, which exposes the high-level functions of Hermes-core to consumer applications.

Internally, it is a small wrapper around the Hermes-core remote APIs: it serialises requests from the consumer applications and forwards them to Hermes-core via communication channels. Since the requests may contain sensitive data, the communication channel between Hermes API and Hermes-core should be encrypted and mutually authenticated. This can be done using the Secure Session from our Themis cryptographic library with our wrapper over raw transport. Since Secure Session is a stream-oriented protocol, the channel will be also protected from replay attacks and request reordering. Alternatively, you may choose to leave the communication channel unencrypted (which is not recommended for the reasons outlined above) or use a secure transport method of your choice (i.e. TLS, etc.).

Hermes-core operates on blocks that are stored in the Data store's database. Operations on blocks in Hermes-core come down to 4 classic data manipulation algorithms (CRUD) and their rights management:

  • Document creation — CREATE block with special content;
  • Reading document structure — READ block;
  • Adding a block to a document — UPDATE block;
  • Deleting block from a document — UPDATE block;
  • Updating the document structure — UPDATE block;
  • Deleting a block — DELETE block;
  • Revoking access — REVOKE READ/UPDATE permissions from users who have those permissions by other users who have those permissions;
  • Rotating data and keys — rotation means re-encrypting the data using new keys.

The distribution of CRUD permissions to data blocks in Hermes-core is carried out cryptographically and described in details below.

Processes

You can find this code in include/hermes/mid_hermes/mid_hermes.h.

CREATE (and grant access)

The CREATE process for a Hermes-core block consists of two phases: 1. Encrypting blocks of the data and putting it into an appropriate place of the Data store; 2. Distributing access control policy for that specific block.

Create block

hermes_status_t mid_hermes_create_block(
        mid_hermes_t *mid_hermes,
        uint8_t **id, size_t *id_length,
        const uint8_t *block, const size_t block_length,
        const uint8_t *meta, const size_t meta_length);

Read

READ access — a permission for a user to READ a block, decrypting it with a READ key. It’s assumed that the user is in the possession of the private part of its credential before performing the READ operation.

To READ a block, user performs the following operations:

  • downloads and unwraps READ key from the associated Keystore;
  • downloads and decrypts the encrypted block from the Data store using READ key.

User does not download (nor is permitted to download) the block’s update tag, which is considered to be a sensitive information with respect to the user with READ access.

Read block

hermes_status_t mid_hermes_read_block(
        mid_hermes_t *mid_hermes,
        const uint8_t *block_id, const size_t block_id_length,
        uint8_t **block, size_t *block_length,
        uint8_t **meta, size_t *meta_length);

Grant read access

hermes_status_t mid_hermes_grant_read_access(
        mid_hermes_t *mid_hermes,
        const uint8_t *block_id, const size_t bloc_id_length,
        const uint8_t *user_id, const size_t user_id_length);

Update

To UPDATE a block, user performs the following operations:

  • downloads and unwraps wrapped READ and UPDATE keys from the associated Keystore;
  • downloads and decrypts the encrypted block from the Data store using READ key;
  • calculates current update tag for the current block using UPDATE key;
  • processes the block;
  • calculates a new update tag for the updated block using UPDATE key;
  • encrypts the updated block with READ key using READ key;
  • uploads 1) current update tag, 2) new update tag and 3) encrypted updated block to the Data store.

Upon receiving the abovementioned data, the Data store will verify that the user performing the UPDATE possesses a valid UPDATE key by comparing the current update tag received from the user with the stored update tag associated with the block. If they are equal, the Data store will overwrite the stored encrypted block with the received encrypted updated block and the stored update tag with a new update tag. This way the Data store never processes the blocks in plain text.

Update block

hermes_status_t mid_hermes_update_block(
        mid_hermes_t *mid_hermes,
        const uint8_t *block_id, const size_t block_id_length,
        const uint8_t *block, const size_t block_length,
        const uint8_t *meta, const size_t meta_length);

Grant update access

hermes_status_t mid_hermes_grant_update_access(
        mid_hermes_t *mid_hermes,
        const uint8_t *block_id, const size_t bloc_id_length,
        const uint8_t *user_id, const size_t user_id_length);

Delete

Deleting blocks is a particular simplified case of block UPDATE, where updated block is NULL, so there is no need for the updater to calculate and send encrypted updated block.

Delete block

hermes_status_t mid_hermes_delete_block(
        mid_hermes_t *mid_hermes, 
        const uint8_t *block_id, const size_t block_id_length);

Revoke access

Abstractly speaking, if user A (with READ/UPDATE permissions) wants to revoke READ (READ/UPDATE) permissions from user B to some block, A should reconstruct current access control policy to this block, then revoke all the READ (READ/UPDATE) permissions for each users who have certain permissions, and finally set a new access control policy to block, based on the former policy.

Revoke read access

hermes_status_t mid_hermes_deny_read_access(
        mid_hermes_t *mid_hermes,
        const uint8_t *block_id, const size_t bloc_id_length,
        const uint8_t *user_id, const size_t user_id_length);

Revoke update access

hermes_status_t mid_hermes_deny_update_access(
        mid_hermes_t *mid_hermes,
        const uint8_t *block_id, const size_t bloc_id_length,
        const uint8_t *user_id, const size_t user_id_length);

Rotate data and keys

The process of rotating data and keys generates new READ and UPDATE keys for a block, re-encrypts a block using new keys, and replaces the old keys with the new ones (i.e. those users who only had READ permissions will get a new READ key, and the users who had READ and UPDATE permissions, will get new READ and UPDATE keys).

Rotate block

hermes_status_t mid_hermes_rotate_block(
        mid_hermes_t *mid_hermes,
        const uint8_t *block_id, const size_t block_id_length);

Please see the Hermes scientific paper - Hermes – a framework for cryptographically assured access control and data security and Hermes' implementation paper Implementing Hermes-based Security Systems for more details and illustrations for the processes described above.

Security model

Note: This security model describes the security model of Hermes. As Hermes-core is a proof-of-concept implementation of Hermes, everything described below holds similarly true for Hermes-core.

Hermes cryptographically enforces the data access control policy and minimises compromisation risks of a single component. Such enforcements are achieved by:

  • reducing the risk of potential damage from a security breach thanks to (physical) distribution of the components of Hermes;
  • limiting/blocking the exposure to sensitive information for each component and in this way securing the perimeter of a potential security breach;
  • relying on strong cryptography for proper access control over sensitive information rather than on organisational operational guidelines and proper software implementation/operation;
  • using end-to-end encryption to protect communication channels between the components.

Threat model

Hermes operates in a very restrictive threat model. It is assumed that: - each system entity can be compromised; - protected data can be partially leaked; - a passive/active attacker may be present in communication channels; - Data store can be dumped; - data object model can be leaked.

Some possible risks are out of the scope of Hermes, i.e. if authorised users deliberately leak sensitive information via out-of-band channels or when the basic execution environment (hardware and/or operating system) has been compromised.

Trust model

  • Users are trusted to properly manage the data they have access to. No security solution can cope if the users publish the data outside of the Hermes-protected infrastructure (or another security system).
  • Users are also trusted to further distribute the access rights to the data (the rights that they’ve received from other authorised users) to other users (that don't yet hold the permissions/rights to the data in question). This means that if the user has level-A access rights, he/she may transfer these access rights of level-A (or lower, but never higher than the level they possess themselves) to another user.
    Hermes is able to enforce the existing access control policy, but neither Hermes nor the user granting rights can generate or provide a new access policy or grant more permissions than initially requested. A policy of traitor tracing has to be assigned on a higher-level system.
  • It’s assumed that the execution environment of the components of Hermes operates properly: only the Hermes' software is running within the address space of each components’ process, and each instruction is handled according to the appropriate CPU manual.
  • It’s also assumed that the operating system for Hermes is properly managed from the perspective of security - that it has proper access control mechanisms and policies, that it is regularly patched for security vulnerabilities, and that it provides proper compartmentalization and separation of processes.

Security guarantees

1) Compromisation of a single entity from the system causes only limited damage; 2) All the sensitive information only appears in plain text in the system user/service’s context exclusively; 3) Data is protected in granular form (per record/document); 4) All communications are protected with end-to-end encryption. 5) Data store imports/stores/exports data in encrypted form only; 6) Keystore imports/stores/exports ACKs in encrypted form only; 7) Credential store imports/stores/exports only system user/service’s public credentials; 8) Each data record is protected with a unique key; 9) Each data record may have its own flexible access control policy.

Additional security reading

For extensive additional information on the security model of Hermes, possible threats, attacks, and mitigations, please see:
- [theoretical paper on Hermes - Hermes – a framework for cryptographically assured access control and data security and

Using Hermes

Hermes is a cryptography-based method of providing protected data storage and sharing that allows enforcing cryptographically checked CRUD permissions to data blocks and doesn't let server that's running Hermes do anything worse than DoS. In other words, Hermes enables collaboration and distributed data sharing through enforcing access control with the help of cryptographic methods. This section is a brief explanation of how you can achieve this.

To use Hermes-core, 4 components need to be implemented: 3 storage entities and a transport interface. The storage entities require databases (generic or specialised) and some “driver” that will act as a point of communication with Hermes-core.

Databases

  1. Data database — data storage entity, which implements the hm_ds_db_t interface (see include/hermes/data_store/db.h).
  2. Credential database — data storage entity, which implements the hm_cs_db_t interface (see include/hermes/credential_store/db.h).
  3. Keystore database — data storage entity, which implements the hm_ks_db_t interface (see include/hermes/key_store/db.h).

Transport

The transport interface is basically the API. Transport(s) is the transport_t interface (see include/hermes/rpc/transport.h). In fact, each instance of communication between the components of the system can be implemented with the help of different transports (i.e. sockets). Transport interfaces primarily need to be able to carry out the receive and send functionality.

Assembling Hermes-core

After implementing the 4 components mentioned above, the assembly of parts of Hermes-core follows. These are the entities you’ll need to implement:

On Client: 1. Client. It consists of mid_hermes (with all corresponding dependencies) and transport.

On Server:
1. Credential store. It includes Credential database, transport, and Credential store server (see include/hermes/credential_store/server.h).
2. Data store. It includes Data database, transport, and Data store server (see include/hermes/data_store/server.h).
3. Keystore. It includes Keystore database, transport, and Keystore server (see include/hermes/key_store/server.h).

Installing Hermes from repository

To see Hermes-core working, you need to perform the following actions.

Install or build Hermes-core

Hermes-core is available for the following versions of operating systems:
*.deb: - Debian: Wheezy, Jessie, Stretch; - Ubuntu: Trusty Tahr, Xenial Xerus, Yakkety Yak, Zesty Zapus.

*.rpm: - CentOS: 7.

Installing for Debian / Ubuntu

1. Import the public key used by Cossack Labs to sign packages:

wget -qO - https://pkgs.cossacklabs.com/gpg | sudo apt-key add -

Note: If you wish to validate key fingerprint, it is: 29CF C579 AD90 8838 3E37 A8FA CE53 BCCA C8FF FACB.

2. You may need to install the apt-transport-https package before proceeding:

sudo apt-get install apt-transport-https

3. Add Cossack Labs repository to your sources.list. You should add a line that specifies your OS name and the release name:

deb https://pkgs.cossacklabs.com/stable/$OS $RELEASE main
  • $OS should be debian or ubuntu.
  • $RELEASE should be one of Debian or Ubuntu release names. You can determine this by running lsb_release -cs, if you have lsb_release installed.

We currently build packages for the following OS and RELEASE combinations:

  • Debian "Wheezy" (Debian 7)
  • Debian "Jessie" (Debian 8)
  • Debian "Stretch" (Debian 9)
  • Ubuntu Trusty Tahr (Ubuntu 14.04)
  • Ubuntu Xenial Xerus (Ubuntu 16.04)
  • Ubuntu Yakkety Yak (Ubuntu 16.10)
  • Ubuntu Zesty Zapus (Ubuntu 17.04)

For example, if you are running Debian 8 "Jessie", run:

echo "deb https://pkgs.cossacklabs.com/stable/debian jessie main" | \
  sudo tee /etc/apt/sources.list.d/cossacklabs.list

4. Reload local package database:

sudo apt-get update

5. Install the package

sudo apt-get install libhermes-core

If you want to run examples or tests, please install dev packages as well:

sudo apt-get install libhermes-core libhermes-core-dev libthemis-dev

Installing for CentOS / RHEL / OEL

Note, we only build RPM packages for x86_64.

1. Import the public key used by Cossack Labs to sign packages:

sudo rpm --import https://pkgs.cossacklabs.com/gpg

Note: If you wish to validate key fingerprint, it is: 29CF C579 AD90 8838 3E37 A8FA CE53 BCCA C8FF FACB.

2. Create a Yum repository file for Cossack Labs package repository:

wget -qO - https://pkgs.cossacklabs.com/stable/centos/cossacklabs.repo | \
  sudo tee /etc/yum.repos.d/cossacklabs.repo

3. Install the package:

sudo yum install libhermes-core

If you want to run examples or tests, please install dev packages as well:

sudo yum install libhermes-core libhermes-core-devel libthemis-devel

What to do after the installation

Consider checking full tutorials to understand how to add and update blocks, grant READ and UPDATE access right for users and revoke access rights.

  • Usage examples describe how examples work and what are possible usages for Hermes-core.
  • C tutorial, where both service and client app are written in C.
  • Python tutorial, where service side is C-based, but client code runs on Python.
  • Go tutorial, where service side is C-based, but client code runs on Go.

Building Hermes

You need to start with installing the libraries and utilities that we're going to need.

For Debian the command is:

sudo apt-get update && sudo apt-get install build-essential libssl-dev git 

We need build-essential for building binary libraries and libssl-dev as backend for Themis.

Let's download and install Themis into your system:

git clone https://github.com/cossacklabs/themis
cd themis
make && sudo make install
cd ..

Now you should download and install Hermes-core:

git clone https://github.com/cossacklabs/hermes-core
cd hermes-core
make && sudo make install

Verify Hermes-core

To verify Hermes-core, build and run the tests:

make test

You're going to see a similar output on success:

== Entering suite #1, "rpc test" ==

--> 3 check(s), 3 ok, 0 failed (0.00%)

==> 3 check(s) in 1 suite(s) finished after 0.00 second(s),
    3 succeeded, 0 failed (0.00%)

[SUCCESS]

build/tests/credential_store_test
== Entering suite #1, "credential_store test" ==

--> 97 check(s), 97 ok, 0 failed (0.00%)

==> 97 check(s) in 1 suite(s) finished after 1.00 second(s),
    97 succeeded, 0 failed (0.00%)

[SUCCESS]

build/tests/data_store_test
== Entering suite #1, "data_store test" ==

--> 28 check(s), 28 ok, 0 failed (0.00%)

==> 28 check(s) in 1 suite(s) finished after 1.00 second(s),
    28 succeeded, 0 failed (0.00%)

[SUCCESS]

build/tests/key_store_test
== Entering suite #1, "key_store test" ==

--> 26 check(s), 26 ok, 0 failed (0.00%)

==> 26 check(s) in 1 suite(s) finished after 1.00 second(s),
    26 succeeded, 0 failed (0.00%)

[SUCCESS]

build/tests/client_test
== Entering suite #1, "client test" ==

--> 93 check(s), 93 ok, 0 failed (0.00%)

==> 93 check(s) in 1 suite(s) finished after 1.00 second(s),
    93 succeeded, 0 failed (0.00%)

[SUCCESS]

build/tests/mid_hermes_test

== Entering suite #1, "mid hermes test" ==

--> 1492 check(s), 1492 ok, 0 failed (0.00%)

==> 1492 check(s) in 1 suite(s) finished after 7.00 second(s),
    1492 succeeded, 0 failed (0.00%)

[SUCCESS]


build/tests/mid_hermes_ll_test
== Entering suite #1, "mid hermes ll test" ==

--> 28 check(s), 28 ok, 0 failed (0.00%)

==> 28 check(s) in 1 suite(s) finished after 0.00 second(s),
    28 succeeded, 0 failed (0.00%)

[SUCCESS]

Congratulations, you've successfully built Hermes-core.

Building example apps

The Hermes-core application architecture requires the four main system components: 1. Credential Store, 2. Data Store, 3. Keystore, 4. Client.

Hermes-core repository currently includes usage examples in C, Python, and Go for Client side and an example in C for Credential/Data/Keystore.

The examples in Hermes-core use the file system as a storage entity (backend) for Credential store, Keystore, and Data store. The examples use simple TCP/IP socket communication as transport layer.

Building examples

The examples can be found in docs/examples/ folder of repository, and they consist of: 1. credential_store_service — Credential store (in C). 2. data_store_service — Data store (in C). 3. key_store_service — Keystore (in C). 4. client — Hermes client (in C, Python, and Go).

Running examples require you to have Hermes-core library installed on your system.

You can build it manually from source, or install from available package manager.

Download Hermes-core

Note: Hermes-core should be already installed on your system for you to be able to build examples.

Download Hermes-core repo if you haven't done so already:

git clone https://github.com/cossacklabs/hermes-core
cd hermes-core
make

Build examples

...and compile examples:

make examples

The command above will build C, Python, and Go examples in the corresponding folders.

Your hermes-core/docs folder structure will look similar to this one:

hermes-core/
L-- docs/examples/
    +-- c/
    ¦   +-- key_gen/
    ¦   L-- mid_hermes/
    ¦       +-- client
    ¦       +-- credential_store_service/
    ¦       +-- data_store_service/
    ¦       L-- key_store_service/
    +-- python/
    ¦   L-- hermes_client.py
    L-- go/
        L-- hermes_client.go

Using example apps

Working with examples consists of the following steps:

  • generate keypairs for several users (user1, user2, user3);
  • create a file;
  • run services: credential_store_service, data_store_service and key_store_service;
  • run the Client: using the keys belonging to user1, encrypt and add a new file to the database;
  • grant READ/UPDATE permissions to the file to other users;
  • add new files, update file content;
  • revoke permissions;
  • remove files.

Each of these operations illustrates the usage of the Hermes-core and the actions it can perform.

Consider checking out the full tutorials to gain a better understanding of how to add and update blocks, grant READ and UPDATE access rights for users and revoke access rights.

Local CLI example

Building and running local CLI example

This is an example of using low-level functions of Hermes-core in order to implement all of Hermes-core's Server parts (Data store, Credential store, and Keystore) through a single app, which also serves as a Client entity.

The main Hermes-core client class is mid_hermes_t which can be found in include/hermes/mid_hermes/mid_hermes.h. We recommend using only this class for Hermes-core applications. However, in some cases, the usage of low-level mid_hermes might be more comfortable.

Low-level mid_hermes

The low-level interface of mid_hermes includes the following 4 classes:

  1. mid_hermes_ll_block_t - main data block type
  2. mid_hermes_ll_buffer_t - {data, size} type
  3. mid_hermes_ll_user_t - user type
  4. mid_hermes_ll_tocken_t - read/update token type

Note: The use of low-level interface requires all the communication, data, keys, and token storages to be controlled by the user. Use it very carefully.

Hermes-core repository includes an usage example for low-level mid_hermes in docs/examples/c/mid_hermes_low_level directory. This example doesn't use any communication and assumes that all of the parts of Hermes are on a local machine.

Building low-level example

Installing Hermes-core

First of all, install Hermes-core with dev packages using Installation guide, or compile Hermes-core from source.

Running example

You can try Hermes-core using a local example that contains all the storage entities Hermes-core needs for correct work. This local example can be found in docs/examples/c/mid_hermes_low_level folder of the repository.

  • Before using this local example you need to register some "users" in Hermes.
    To register a user with id "USER", type this into the command line:
cd docs/examples/c/mid_hermes_low_level
make
mkdir -p ./db/credential_store
../key_gen/key_pair_gen USER.priv ./db/credential_store/$(echo -n USER | base64)
  • Create first file file.1block:
echo "smth" > file1.block
  • Add the block file1.block with meta "first block in HERMES" to Hermes-core as a USER, using the following command:
./hermes_client_ll add_block USER $(cat USER.priv | base64) file1.block "first block in Hermes"
  • Now you can see some new files in the Hermes-core storages:
./db/data_store/ZmlsZTEuYmxvY2s=/data                - encrypted block
./db/data_store/ZmlsZTEuYmxvY2s=/mac                 - UPDATE TAG
./db/data_store/ZmlsZTEuYmxvY2s=/meta                - block meta
./db/key_store/ZmlsZTEuYmxvY2s=/VVNFUg==/r/token     - read token
./db/key_store/ZmlsZTEuYmxvY2s=/VVNFUg==/r/owner     - read token owner
./db/key_store/ZmlsZTEuYmxvY2s=/VVNFUg==/w/token     - update token
./db/key_store/ZmlsZTEuYmxvY2s=/VVNFUg==/w/owner     - update token owner

Where:

ZmlsZTEuYmxvY2s= - base64 encoded string "file1.block"

VVNFUg== - base64 encoded string "USER".

  • Find the list of all the other commands supported by the example using the following command:
docs/examples/c/mid_hermes_low_level/hermes_client_ll --help
usage: hermes_client_ll <command> <user id> <base64 encoded user private key> <name of file for proceed> <meta> <for user>.
           <command>                         - executes the command to be performed by the client, see below;
           <user id>                         - user identifier (user needs to be registered in Credential store);
           <base64 encoded user private key> - base64 encoded private key of the user;
           <name of file to be processed>    - filename of the file to be processed (file name is used as block ID in Hermes);
           <meta>                            - some data associated with a file that is stored in the database in plaintext;
           <for user>                        - identifier of the user for which permissions are provided/revoked from (this information is needed by some commands).

commands:
           add_block - add <name of file to be processed> block with <meta> to Hermes system
           read_block - read <name of file to be processed> block with <meta> from Hermes system
           update_block - update <name of file to be processed> block with <meta> in Hermes system
           delete_block - delete <name of file to be processed> block from Hermes system
           grant_read - grant read access for <for user> to <name of file to be processed> block in Hermes system
           grant_update - grant update access for <for user> to <name of file to be processed> block in Hermes system
           revoke_read - deny read access for <for user> to <name of file to be processed> block in Hermes system
           revoke_update - deny update access for <for user> to <name of file to be processed> block in Hermes system

Congratulations! Hermes-core is working.

Please also see the Tutorials section for a step-by-step explanation of how to build your own Hermes client-server app.

Creating your own Hermes-based app

It is highly advisable to familiarise yourself with the scientific Hermes paper and Hermes Implementation paper to deeply understand the core components and processes of Hermes, before getting down to the creating a Hermes-based app.

Applied Theory

Building Hermes-based app starts with understanding the architecture of target solution: do the clients and data stores reside on different servers? Should the app perform remote or local operations? What can be trusted and what cannot? Depending on these factors, different layouts should be chosen:

  • Distributed, client-server;
  • Integrated, client-server, pieces aligned in a custom way;
  • Compact, single-node.

For example, let's take one typical use-case for Hermes — an online medical service where the patients securely interact with the staff of multiple medical institutions. Let's say, we have patient's records that contain a number of medical examination results, and the patient’s personal physician needs to be able to access all of them, while the employees of the medical institutions need to be able to access only the latest results to adjust their activities towards the patient.

Our goal would be to implement such an encryption scheme, where the patient (User) is the only one who can read his/her every medical record, and is the one who can further distribute the rights to some pieces of data. The User's MD gets access to all of the medical data, a surgeon gets access to most of the User's data, while a nurse can only see a limited number of records necessary for the upcoming procedures. We must also decide where we are going to store the records, tokens, users credentials, etc.

Having decided upon the general outline of the Hermes-based app we want to build, we can proceed to the App building plan.

Practice

01. (Make sure you) Understand the integration points.

Hermes-core doesn't include communication and storage components, only interfaces. Communication and storage must be implemented separately (through external means) before you'll be able to start building and using Hermes-core.

There is only one requirement towards the communication between the components of Hermes-core — security. For this reason, Hermes-core has a built-in wrapper that creates Themis' Secure Session communication channel under the abstract transport that needs to be implemented by the user. Such transport can be created using any available mechanism and it must be able to implement the following interface:

(include/hermes/rpc/transport.h)

Hermes-core requires three separate storage entities:

There are no special requirements towards these storage entities.

02. Plan out the key life cycle and the trust scheme.

All the necessary ephemeral key pairs and symmetric keys are only stored in memory when they are needed. After usage, they are securely zero-filled (replaced with zeros). The only secret that the user needs to control him/herself is the private user key. The main requirement for the private key is that it mustn't ever leave the client machine (as in "leak outside the machine").

03. Integrate Client & Server.

Now that all the components (backends) are ready, the actual implementation of the client and server parts of the application can be carried out.

Hermes-core includes helper objects for implementing the Server side:

A typical usage of helper objects looks like this:

  hm_credential_store_service_t* service=hm_credential_store_service_create(client_transport, db);
  if(!service){
    // handle error
  }
  hm_credential_store_service_start(service);
  hm_credential_store_service_destroy(&service);

04. Implement key lifecycle scheme in Client.

For implementing the client side, Hermes-core provides an object mid_hermes_t, which provides an interface that includes all the necessary functions. You can see an example implementation of mid_hermes_t in docs/examples/c/mid_hermes/client/hermes_client.c. The key lifecycle scheme on the Client is fully controlled by the mid_hermes_t instance.

What's next

Read a detailed explanation of app components in Create your client server app in the tutorials section.

Consider checking full tutorials to understand how to add and update blocks, grant READ and UPDATE access right for users and revoke access rights.

  • Usage examples describe how examples work and what are possible usages for Hermes-core.
  • C tutorial, where both service and client app are written in C.
  • Python tutorial, where service side is C-based, but client code runs on Python.
  • Go tutorial, where service side is C-based, but client code runs on Go.

Hermes-core Client

The Client for Hermes-core needs to have access to all the 3 Stores (Data store, Credential store, Keystore). The key (token) used by the Client must be present in the Credential store on the Server side of Hermes-core to allow for further operations on documents.

It's also recommended that you take a look at the example Clients written in C, Python, and Go before proceeding with building your own.

To create your own Hermes-based app (the one that includes Server, Client, and Transport between the components), use the detailed instruction in Client-server app (Step by step).

Hermes-core client API interface

The Client-side component of Hermes-core is called mid_hermes_t and is declared in hermes/mid_hermes/mid_hermes.h.

mid_hermes_t represents all the Hermes-core's functions as a simple CRUD interface to an abstract remote storage.

Interface

include/hermes/mid_hermes/mid_hermes.h:

typedef struct mid_hermes_type mid_hermes_t;

mid_hermes_t *mid_hermes_create(
        const uint8_t *user_id, const size_t user_id_length,
        const uint8_t *private_key, const size_t private_key_length,
        hm_rpc_transport_t *key_store_transport,
        hm_rpc_transport_t *data_store_transport,
        hm_rpc_transport_t *credential_store_transport);

hermes_status_t mid_hermes_destroy(mid_hermes_t **mh);

hermes_status_t mid_hermes_create_block(
        mid_hermes_t *mid_hermes,
        uint8_t **id, size_t *id_length,
        const uint8_t *block, const size_t block_length,
        const uint8_t *meta, const size_t meta_length);

hermes_status_t mid_hermes_read_block(
        mid_hermes_t *mid_hermes,
        const uint8_t *block_id, const size_t block_id_length,
        uint8_t **block, size_t *block_length,
        uint8_t **meta, size_t *meta_length);

hermes_status_t mid_hermes_update_block(
        mid_hermes_t *mid_hermes,
        const uint8_t *block_id, const size_t block_id_length,
        const uint8_t *block, const size_t block_length,
        const uint8_t *meta, const size_t meta_length);

hermes_status_t mid_hermes_delete_block(
        mid_hermes_t *mid_hermes, const uint8_t *block_id, const size_t block_id_length);

hermes_status_t mid_hermes_rotate_block(
        mid_hermes_t *mid_hermes, const uint8_t *block_id, const size_t block_id_length);

hermes_status_t mid_hermes_grant_read_access(
        mid_hermes_t *mid_hermes,
        const uint8_t *block_id, const size_t bloc_id_length,
        const uint8_t *user_id, const size_t user_id_length);

hermes_status_t mid_hermes_grant_update_access(
        mid_hermes_t *mid_hermes,
        const uint8_t *block_id, const size_t bloc_id_length,
        const uint8_t *user_id, const size_t user_id_length);

hermes_status_t mid_hermes_deny_read_access(
        mid_hermes_t *mid_hermes,
        const uint8_t *block_id, const size_t bloc_id_length,
        const uint8_t *user_id, const size_t user_id_length);

hermes_status_t mid_hermes_deny_update_access(
        mid_hermes_t *mid_hermes,
        const uint8_t *block_id, const size_t bloc_id_length,
        const uint8_t *user_id, const size_t user_id_length);

To construct the mid_hermes_t instance function mid_hermes_create, in addition to the user id and the user private key, three transports instances to Credential store, Data store, and Keystore are needed respectively.

You can check client the implementation in docs/examples/c/mid_hermes/client/hermes_client.c

First, you need to connect to all the services of Credential store, Data store, and Keystore, and create a Secure Session between them using the Hermes wrapper (unless you want to use non-secure unencrypted open transports like TCP/UDP/Websocket or want to implement your own means of supporting encryption like TLS/SSL).

Create a connection to Credential store:

transports_container_t container = {NULL, NULL, NULL, NULL, NULL, NULL};
container.raw_credential_store_transport = server_connect(CREDENTIAL_STORE_IP, CREDENTIAL_STORE_PORT);

Here calling the function server_connect(CREDENTIAL_STORE_IP, CREDENTIAL_STORE_PORT) must return the implementation of the transport interface defined in include/hermes/rpc/transport.h.

A simple TCP/IP socket transport implementation can be found in the following examples: docs/examples/c/mid_hermes/common/transport.h docs/examples/c/mid_hermes/common/transport.c

The transport needs to be wrapped into Secure Session by calling the create_secure_transport function from include/hermes/secure_transport/transport.h and passing the user's id, user's public key that will be used for establishing the session, ID of the service we're connecting to (in this case it is Credential store), the service's public key, and the transport that's being wrapped.

The type of connection that needs to be established must also be indicated here - either the server type (then the last parameter will be true) or the client type (the last parameter will be false).

The necessity to indicate the connection type is due to the fact that the session is always initialized by the Client who needs to send a request for establishing a session. You can read more here.

container.credential_store_transport = create_secure_transport(
        user_id, user_id_length, 
        user_private_key, user_private_key_length, 
        credential_store_pk, sizeof(credential_store_pk),
        credential_store_id, strlen((char*)credential_store_id), 
        container.raw_credential_store_transport, 
        false);

The connection with Data store and Keystore is created in a similar manner 1, 2.

Now the mid_hermes object can be created and the requests to the API will be sent through it. When creating the mid_hermes object, the following parameters need to be passed - user_id and its private_key, as well as the 3 connections to the services, created earlier:

mh = mid_hermes_create(
    user_id, user_id_length,
    user_private_key, user_private_key_length,
    container.key_store_transport,
    container.data_store_transport,
    container.credential_store_transport)

After a successful creation of an instance of mid_hermes_t, all the instances of the interface method can be called. Each mid_hermes_t interface method represents one of the Hermes operations:

1. CREATE block

hermes_status_t mid_hermes_create_block(
        mid_hermes_t *mid_hermes,
        uint8_t **id, size_t *id_length,
        const uint8_t *block, const size_t block_length,
        const uint8_t *meta, const size_t meta_length);

2. READ block

hermes_status_t mid_hermes_read_block(
        mid_hermes_t *mid_hermes,
        const uint8_t *block_id, const size_t block_id_length,
        uint8_t **block, size_t *block_length,
        uint8_t **meta, size_t *meta_length);

3. UPDATE block

hermes_status_t mid_hermes_update_block(
        mid_hermes_t *mid_hermes,
        const uint8_t *block_id, const size_t block_id_length,
        const uint8_t *block, const size_t block_length,
        const uint8_t *meta, const size_t meta_length);

4. DELETE block

hermes_status_t mid_hermes_delete_block(
        mid_hermes_t *mid_hermes, 
        const uint8_t *block_id, const size_t block_id_length);

5. ROTATE block

hermes_status_t mid_hermes_rotate_block(
        mid_hermes_t *mid_hermes, 
        const uint8_t *block_id, const size_t block_id_length);

6. GRANT read access

hermes_status_t mid_hermes_grant_read_access(
        mid_hermes_t *mid_hermes,
        const uint8_t *block_id, const size_t bloc_id_length,
        const uint8_t *user_id, const size_t user_id_length);

7. GRANT update access

hermes_status_t mid_hermes_grant_update_access(
        mid_hermes_t *mid_hermes,
        const uint8_t *block_id, const size_t bloc_id_length,
        const uint8_t *user_id, const size_t user_id_length);

8. DENY read access

hermes_status_t mid_hermes_deny_read_access(
        mid_hermes_t *mid_hermes,
        const uint8_t *block_id, const size_t bloc_id_length,
        const uint8_t *user_id, const size_t user_id_length);

9. DENY update access

hermes_status_t mid_hermes_deny_update_access(
        mid_hermes_t *mid_hermes,
        const uint8_t *block_id, const size_t bloc_id_length,
        const uint8_t *user_id, const size_t user_id_length);

Hermes-core Server

Hermes-core might be also described as a wrapper around a database where the exact database type used does not matter.

We've designed Hermes-core to be used in a fashion where different stores (Credential store, Data store, and Keystore) can be local, remote, unified, or have intentionally different architectures. They are passed to Hermes-core as parameters, and if the implementation fits the interface, anything can be used for storing Hermes-core's data, including with something as esoteric as a possible usage of blockchain for Keystore (i.e. for revocation and validation).

Each of the storage entities for Hermes-core has its own interface: see include/hermes/data_store/db.h, include/hermes/credential_store/db.h, and include/hermes/key_store/db.h respectively.

To create your own Hermes-based app (the one that includes Server, Client, and Transport between the components), use the detailed instructions in Client-server app (Step by step).

Data store

Hermes-core doesn't have special requirements for the Data store database. Such database needs to be able to implement the following interface.

include/hermes/data_store/db.h:

typedef uint32_t(*hm_ds_db_insert_block)(
        void* db, const uint8_t* block, const size_t block_length,
        const uint8_t* meta, const size_t meta_length,
        const uint8_t* mac, const size_t mac_length,
        uint8_t** id, size_t* id_length);
typedef uint32_t(*hm_ds_db_insert_block_with_id)(
        void* db, const uint8_t* id, const size_t id_length,
        const uint8_t* block, const size_t block_length,
        const uint8_t* meta, const size_t meta_length,
        const uint8_t* mac, const size_t mac_length);
typedef uint32_t(*hm_ds_db_get_block)(
        void* db, const uint8_t* id, const size_t id_length,
        uint8_t** block, size_t*  block_length,
        uint8_t** meta, size_t*  meta_length);
typedef uint32_t(*hm_ds_db_update_block)(
        void* db, const uint8_t* id, const size_t id_length,
        const uint8_t* block, const size_t block_length,
        const uint8_t* meta, const size_t meta_length,
        const uint8_t* mac, const size_t mac_length,
        const uint8_t* old_mac, const size_t old_mac_length);
typedef uint32_t(*hm_ds_db_rem_block)(
        void* db, const uint8_t* id, const size_t id_length,
        const uint8_t* old_mac, const size_t old_mac_length);

typedef struct hm_ds_db_type{
    void* user_data;
    //insert block to data store. Will return id of added block
    hm_ds_db_insert_block insert_block; 
    //insert block to data store with predefined id. if block with provided id is already present in data store error will return
    hm_ds_db_insert_block_with_id insert_block_with_id;
    //read block from data store
    hm_ds_db_get_block get_block;
    //update block in data store 
    hm_ds_db_update_block update_block;
    //delete block from data store
    hm_ds_db_rem_block rem_block;
}hm_ds_db_t;

A simple filesystem-based data storage implementation can be found in docs/examples/c/mid_hermes/data_store_service/db.h and docs/examples/c/mid_hermes/data_store_service/db.c.

Сredential store

Hermes-core doesn't have special requirements towards the Credential store database. The database needs to be able to implement the following interface — include/hermes/credential_store/db.h:

typedef uint32_t(*hm_cs_db_get_pub_by_id_t)(void *db, const uint8_t *id, const size_t id_length, uint8_t **key, size_t *key_length);

typedef struct hm_cs_db_type {
    void *user_data;
    // get public key by provided user id
    hm_cs_db_get_pub_by_id_t get_pub;
} hm_cs_db_t;

A simple filesystem-based Credential store implementation can be found in the following examples: docs/examples/c/mid_hermes/credential_storage_service/db.h docs/examples/c/mid_hermes/credential_storage_service/db.c

Keystore

Hermes-core doesn't have special requirements towards the Keystore database. The database needs to be able to implement the following interface (include/hermes/key_store/db.h):

typedef uint32_t(*hm_ks_db_set_token)(
        void* db, const uint8_t* block_id, const size_t block_id_length,
        const uint8_t* user_id, const size_t user_id_length,
        const uint8_t* owner_id, const size_t owner_id_length,
        const uint8_t* read_token, const size_t read_token_length);
typedef uint32_t(*hm_ks_db_get_token)(
        void* db, const uint8_t* block_id, const size_t block_id_length,
        const uint8_t* user_id, const size_t user_id_length,
        uint8_t** write_token, size_t* write_token_id_length,
        uint8_t** owner_id, size_t* owner_id_length);
typedef uint32_t(*hm_ks_db_del_token)(
        void* db, const uint8_t* block_id, const size_t block_id_length,
        const uint8_t* user_id, const size_t user_id_length,
        const uint8_t* owner_id, const size_t owner_id_length);
typedef uint32_t(*hm_ks_db_get_indexed_rights)(
        void* db, const uint8_t* block_id, const size_t block_id_length,
        const size_t index, uint8_t** user_id,
        size_t* user_id_length, uint32_t* rights_mask);

typedef struct hm_ks_db_type{
    void* user_data;
    // set read token
    hm_ks_db_set_token set_rtoken;
    // set update token
    hm_ks_db_set_token set_wtoken;
    // get read token
    hm_ks_db_get_token get_rtoken;
    // get update token
    hm_ks_db_get_token get_wtoken;
    // return user id and user rights mask ("r" or "w") with shift by index 
    hm_ks_db_get_indexed_rights get_indexed_rights;
    // delete read token
    hm_ks_db_del_token del_rtoken;
    // delete update token
    hm_ks_db_del_token del_wtoken;
}hm_ks_db_t;

A simple filesystem-based Keystorage implementation can be found in the following examples: docs/examples/c/mid_hermes/key_storage_service/db.h and docs/examples/c/mid_hermes/key_storage_service/db.c.

Hermes-core Transport

The Transport in Hermes-core provides communication between the components.

As an abstract framework, Hermes-core doesn't include any communication and storage components, only interfaces. Communication and storage entities (Data store, Credential store, Keystore) must be implemented before using Hermes-core.

There is only one requirement towards the communication between the components of Hermes-core — security.

For this reason, Hermes-core has a built-in wrapper that creates Themis' Secure Session communication channel under the abstract transport that needs to be implemented by the user. Such transport can be created using any available mechanism and it must be able to implement the following interface (include/hermes/rpc/transport.h):

typedef uint32_t(*hm_rpc_transport_send_t)(void *transport, const uint8_t *buffer, const size_t buffer_length);

typedef uint32_t(*hm_rpc_transport_recv_t)(void *transport, uint8_t *buffer, size_t buffer_length);

typedef uint32_t(*hm_rpc_transport_get_remote_id_t)(void *transport, uint8_t **id, size_t *id_length);

typedef struct hm_rpc_transport_type {
    hm_rpc_transport_send_t send;
    hm_rpc_transport_recv_t recv;
    hm_rpc_transport_get_remote_id_t get_remote_id;
    void *user_data;
} hm_rpc_transport_t;

There is an already implemented Secure Session transport interface in include/hermes/secure_transport/transport.h.

typedef struct secure_transport_type {
    // transport that will be wrapped
    hm_rpc_transport_t* user_transport;
    // secure session for this connection
    secure_session_t* session;
    secure_session_user_callbacks_t* session_callback;

} secure_transport_t;

uint32_t destroy_secure_transport(secure_transport_t** transport_);
uint32_t destroy_rpc_secure_transport(hm_rpc_transport_t** transport_);
hm_rpc_transport_t* create_secure_transport(
        const uint8_t *user_id, size_t user_id_length,
        const uint8_t *private_key, size_t private_key_length,
        const uint8_t *public_key, size_t public_key_length,
        const uint8_t *public_key_id, size_t public_key_id_length,
        hm_rpc_transport_t* user_transport,
        bool is_server);

hm_rpc_transport_t* create_secure_transport_with_callback(
        const uint8_t *user_id, size_t user_id_length,
        const uint8_t *private_key, size_t private_key_length,
        secure_session_user_callbacks_t* callback,
        hm_rpc_transport_t* user_transport,
        bool is_server);

A simple TCP/IP socket transport implementation can be found in the following examples:
docs/examples/c/mid_hermes/common/transport.h
docs/examples/c/mid_hermes/common/transport.c

Here are the examples of the way to wrap a simple transport into Secure Session: docs/examples/c/mid_hermes/client/hermes_client.c docs/examples/c/mid_hermes/credential_store_service/main.c docs/examples/c/mid_hermes/key_store_service/main.c docs/examples/c/mid_hermes/data_store_service/main.c

To check out the transport interfaces for Python and Go, see the corresponding examples: - Python examples: docs/examples/python/hermes_client.py, - Golang examples: docs/examples/go/hermes_client.go.

Alternatively, you may choose to implement transport using your own preferred means (i.e. TLS or an unencrypted connection).

To see the step-by-step instruction of implementing transport between all the Hermes-core components that need to be connected using transport (with Secure Session) and create your own Hermes-based app, see Create your client server app document.

Contributing to Hermes-core

Hermes-core is an open-source software licensed with GNU Affero General Public License v3.0. It is maintained by Cossack Labs and we highly encourage you to contribute. Please see the Contacts and assistance for more details on contributing to Hermes-core.

Filemap (What's where)

This table should help you navigate through the Hermes-core sources.

Note: The following table consists of 3 columns: File, Description, Object. Use the arrow keys on your keyboard to see the contents of "Object: column where available.

File Description Object
include/hermes/rpc/server.h
src/rpc/server.c
Server part of RPC hm_rpc_server_t
include/hermes/rpc/client.h
src/rpc/client.c
Client part of RPC hm_rpc_client_sync_t
include/hermes/rpc/param_pack.h
src/rpc/param_pack.c
RPC parameter serialiser hm_param_pack_t
include/hermes/rpc/buffers_list.h
src/rpc/buffers_list.c
Buffer list interface and implementation hm_buffers_list_t
include/hermes/rpc/transport.h RPC transport interface hm_rpc_transport_t
include/hermes/secure_transport/transport.h
src/secure_transport/transport.c
Transport wrapper that uses Secure Session secure_transport_t
include/hermes/secure_transport/session_callback.h
src/secure_transport/session_callback.c
Session callback for Secure Session -
include/hermes/common/utils.h Some useful defines
include/hermes/common/hash_table.h
src/common/hm_hash_table.c
Hash table data type hm_hash_table_t
include/hermes/common/errors.h Definitions and handlers for error codes -
include/hermes/common/buffer.h
src/common/buffer.c
{buffer, size} type buffer_t
include/hermes/key_store/server.h
src/key_store/server.c
Keystore server component hm_key_store_server_t
include/hermes/key_store/service.h
src/key_store/service.c
Keystore service creation helper hm_key_store_service_t
include/hermes/key_store/client.h
src/key_store/client.c
Synchronous Keystore client component hm_key_store_client_sync_t
include/hermes/key_store/functions.h
src/key_store/functions.c
Keystore functions, proxies, and stubs -
include/hermes/key_store/db.h Keystore database interface hm_ks_db_t
include/hermes/mid_hermes/mid_hermes.h
src/mid_hermes/mid_hermes.c
The main hermes client component mid_hermes_t
src/mid_hermes/credential_store_impl.h
src/mid_hermes/credential_store_impl.c
Mid Hermes Credential store bindings -
src/mid_hermes/data_store_impl.h
src/mid_hermes/data_store_impl.c
Mid Hermes Data store bindings -
src/mid_hermes/key_store_impl.h
src/mid_hermes/key_store_impl.c
Mid Hermes Keystore bindings -
include/hermes/data_store/server.h
src/data_store/server.c
Data store server component hm_data_store_server_t
include/hermes/data_store/service.h
src/data_store/service.c
Data store service creation helper hm_data_store_service_t
include/hermes/data_store/client.h
src/data_store/client.c
Synchronous data store client component hm_data_store_client_sync_t
include/hermes/data_store/db.h Data store database interface hm_ds_db_t
include/hermes/data_store/functions.h
src/data_store/functions.c
Data store functions, proxies, and stubs -
include/hermes/credential_store/server.h
src/credential_store/server.c
Credential store server component hm_credential_data_server_t
include/hermes/credential_store/service.h
src/credential_store/service.c
Credential store service creation helper hm_credential_store_service_t
include/hermes/credential_store/client.h
src/credential_store/client.c
Synchronous credential store client component hm_credential_store_client_sync_t
include/hermes/credential_store/db.h Credential store database interface hm_cs_db_t
include/hermes/credential_store/functions.h
src/credential_store/functions.c
Credential store functions, proxies, and stubs -
include/hermes/mid_hermes_ll/mid_hermes_ll_rights_list.h
src/mid_hermes_ll/mid_hermes_ll_rights_list.c
Low-level Hermes-core bindings for list of block access rights mid_hermes_ll_rights_list_t
include/hermes/mid_hermes_ll/mid_hermes_ll_token.h
src/mid_hermes_ll/mid_hermes_ll_token.c
Low-level Hermes-core READ/UPDATE token class mid_hermes_ll_token_t
include/hermes/mid_hermes_ll/mid_hermes_ll_block.h
src/mid_hermes_ll/mid_hermes_ll_block.c
Low-level Hermes-core block class mid_hermes_ll_block_t
include/hermes/mid_hermes_ll/mid_hermes_ll_buffer.h
src/mid_hermes_ll/mid_hermes_ll_buffer.c
Low-level Hermes-core buffer class mid_hermes_ll_buffer_t
include/hermes/mid_hermes_ll/mid_hermes_ll_user.h
src/mid_hermes_ll/mid_hermes_ll_user.c
Low-level Hermes-core user class mid_hermes_ll_user_t
include/hermes/mid_hermes_ll/interfaces/credential_store.h Low-level Hermes-core Credential store interface hermes_credential_store_t
include/hermes/mid_hermes_ll/interfaces/key_store.h Low-level Hermes-core Keystore interface hermes_key_store_t
include/hermes/mid_hermes_ll/interfaces/data_store.h Low-level Hermes-core datastore interface hermes_data_store_t
include/hermes/mid_hermes_ll/utils.h
src/mid_hermes_ll/utils.c
Hermes-core main cryptography functions definition -
- - -
tests/rpc/rpc_test.c Hermes-core PRC tests -
tests/rpc/param_pack_test.c Hermes-core RPC param_pack test -
tests/rpc/server_test.c Hermes-core RPC client server test -
tests/common/test_transport.c Common transport test -
tests/common/test_utils.h Tests utils header -
tests/common/test_utils.c Tests utils -
tests/common/test_key_store_db.h Keystore tests -
tests/common/test_credential_store_db.c Credential store tests -
tests/common/test_transport.h Transport tests -
tests/common/test_credential_store_db.h Credential store header test -
tests/common/test_data_store_db.c Data store test -
tests/common/test_data_store_db.h Data store header test -
tests/common/sput.h Test framework -
tests/common/test_key_store_db.c Keystore test -
tests/key_store/key_store_tests.c Keystore tests -
tests/data_store/data_store_tests.c Data store tests -
tests/credential_store/credential_store_tests.c Credential store tests -
tests/mid_hermes/mid_hermes.c Main Hermes client tests -
tests/mid_hermes_ll/main.c Entry point for low-level tests -
tests/mid_hermes_ll/block_tests.h Low-level Hermes-core block tests header -
tests/mid_hermes_ll/block_tests.c Low-level Hermes-core block tests -
- - -
pyhermes/pyhermes.c Python bindings for Hermes -
pyhermes/setup.py The main pyHermes setup script -
pyhermes/transport.h
pyhermes/transport.c
Python bindings for PRC Hermes transport that used internally in extension -
pyhermes/py_transport.h
pyhermes/py_transport.c
Wraps user's transport implementation into wrapper that can be used by MidHermes -
pyhermes/py_secure_transport.h
pyhermes/py_secure_transport.c
Wraps user's transport implementation using Secure Session wrapper from Hermes-core and can be used by MidHermes -
pyhermes/py_transport_wrapper.h
pyhermes/py_transport_wrapper.c
Internal python transport interface that MidHermes expects on initialization -
pyhermes/py_midhermes.h
pyhermes/py_midhermes.c
Hermes client bindings (main data block operations) -
- - -
gohermes/mid_hermes.go Hermes client bindings (main data block operations) -
gohermes/transport.h
gohermes/transport.c
gohermes/transport.go
Go bindings for both PRC and Secure Session Hermes transport -
gohermes/utils.go Some usefil bits -
- - -
libs/themis Themis library reference -
- - -
docs/examples/c/mid_hermes/ Mid Hermes example in C -
docs/examples/c/utils/ Some common examples for utils function -
docs/examples/c/mid_hermes_low_level/ Mid Hermes low-level example in C -
docs/examples/c/key_gen/ Key pair generator -
docs/examples/python/ pyHermes example -
docs/examples/go/ Golang example -
docs/examples/test_keys/ Predefined keys used in examples -

Hermes and GDPR

As a framework for cryptographically assured access control and data security, Hermes can help you reach better compliance with the General Data Protection Regulation (GDPR) as a "state of the art" cryptographic tool.

To find out about the exact ways Hermes can help secure the users' data in your product(s)/infrastructure, apply for the DataGuardian Assistance Program.

You may also want to read about the way our encryption proxy Acra can help you meet the demands of several GDPR articles at once.

Contacts and assistance

If you would like to receive a one-time customised assistance in implementing Hermes for the needs of your app/service, consider applying for our DataGuardian Assistance Program.

Although Hermes-core is an in-house-developed product by Cossack Labs Limited, it is still open-source software. This means you can hack it any way you want and contribute things back if you'd like to. As a software company, we focus on implementing features that are important to our products but would gladly spend some time on making Hermes useful for everybody.

"I'd like to help somehow, but don't know what will be useful. What should I do?"

If you're looking for something to contribute to and gain our eternal respect (or something more tangible - you might be in for a surprise when contributing to us, who knows ;), just pick the things in the list of issues.

If you'd like to do independent parts (implement new procedures/objects, do a language wrapper or a set of examples for languages or architectures we don't have and don't even plan yet) - just go ahead and let us know when you finish.

We highly encourage you to:

  • Report bugs and request features via GitHub Issues.
  • Report a bug and fix it with a patch via our GitHub Pull request (after creating a corresponding issue and leaving a link to the pull there).
  • Add something new to Hermes-core. There is a certain design scheme according to which we'd like to keep developing Hermes-core. If your contributions fall along with it, we'd be glad to accept some fundamental additions. It's better to discuss the subject using email before taking action.

Every commit that goes into the master branch is audited and reviewed by somebody from Cossack Labs, don't be surprised if the process is long-ish.

If you'd like to participate in the core development more closely and deeply, get in touch.

Getting in touch

  • Requests/bugfixes/queries should go through GitHub Issues.
  • To get in touch with the developers, use this email at your own discretion :). Make sure you've tried reaching out through GitHub Issues first before writing a direct email.
  • To talk to the business wing of Cossack Labs Limited, drop us an email.

Implementation assistance

If you would like to receive a one-time customized assistance in implementing Hermes-core for the needs of your app/service, consider applying for our DataGuardian Assistance Program.