Using Themis in Python

Introduction

The pythemis extension (wrapper) provides access to the features and functions of the Themis cryptographic library in Python (2.7+ and 3.x):

  • Key generation: creation of public/private key pairs, used in Secure Message and Secure Session.
  • Secure Message: secure exchange of messages between two parties. RSA + PSS + PKCS#7 or ECC + ECDSA (based on key choice), AES GCM container.
  • Secure Storage (aka Secure Cell): provides secure storage of record-based data through symmetric encryption and data authentication. AES GCM / AES CTR containers.
  • Secure Session: a session between two peers, within which the data can be securely exchanged with higher security guarantees. EC + ECDH, AES container.
  • Secure Comparator: comparison of a secret between two parties without leaking anything related to the secret: Zero-Knowledge Proof-based authentication system. Hardened Socialist Millionaire Protocol + ed25519.

You can learn more about the Themis library in the general documentation.

There are also example console utils available for the Python wrapper for Themis (as well as for some other wrappers — see the full list here). They help understand the specific mechanics of encryption/decryption processes of this specific wrapper. You can find the example console utils for the Python wrapper here.

Supported versions

pythemis is tested on various python versions – starting 2.7 to 3.5 and higher.

Quickstart

Installing stable version from pip

PyThemis wrapper is available on pip. In order to use the wrapper, you still need to have the core library installed.

1. Install Themis Core as a system library using your system's package manager.

IMPORTANT: PyThemis requires core Themis headers to be installed.
This means that you need to install the development package:
- libthemis-dev for Debian and Ubuntu,
- libthemis-devel for RHEL and CentOS.
The headers are required at run-time so you need to have this package installed on the system you are deploying to.

2. Install PyThemis wrapper via pip:

pip install pythemis

NOTE: You'll need to use sudo if you're installing it system-wide, not just in a dedicated or virtual user environment.

Building latest version from source

If the stable package version does not suit your needs, you can manually build and install the latest version of Themis from source code.

IMPORTANT: Remember that in addition to the common set of build tools, Themis currently requires either the OpenSSL or LibreSSL package with the developer version of the package (as it provides header files). In either case, we strongly recommend you using the most recent version of these packages.

1. Build and install Themis Core library into your system.

2. Build and install PyThemis wrapper from source code.

Importing Themis

Add to your code:

import pythemis

Now you're good to go!

You can also import only a certain module (keypair generation, for example):

from pythemis import skeygen

Examples

Using Themis

Keypair generation

Themis supports both Elliptic Curve and RSA algorithms for asymmetric cryptography. Algorithm type is chosen according to the generated key type. Asymmetric keys are necessary for Secure Message and Secure Session objects.

⚠️ WARNING: When you distribute private keys to your users, make sure the keys are sufficiently protected. You can find the guidelines here.

NOTE: When using public keys of other peers, make sure they come from trusted sources.

Keypair generation interface

class KEY_PAIR_TYPE(object):
    EC = 'EC'
    RSA = 'RSA'
    CHOICES = (EC, RSA)


class GenerateKeyPair(object):
    def __init__(self, alg)
    def export_private_key(self) -> bytes
    def export_public_key(self) -> bytes

Description:

  • __init__(self, alg)
    Generates key pair. Parameter alg sets the algorithm to be used:
    • KEY_PAIR_TYPE.EC — for Elliptic Curve
    • KEY_PAIR_TYPE.RSA — for RSA
    Throws ThemisError on failure.
  • export_private_key(self)
    Return bytes with generated private key.
  • export_public_key(self)
    Return bytes with generated public key.

Example

from pythemis.skeygen import GenerateKeyPair, KEY_PAIR_TYPE

# or KEY_PAIR_TYPE.RSA for RSA
obj = GenerateKeyPair(KEY_PAIR_TYPE.EC)
private_key = obj.export_private_key()
public_key = obj.export_public_key()

Secure Message

The Secure Message functions provide a sequence-independent, stateless, contextless messaging system. This may be preferred in cases that don't require frequent sequential message exchange and/or in low-bandwidth contexts. This is secure enough to exchange messages from time to time, but if you'd like to have Perfect Forward Secrecy and higher security guarantees, please consider using Secure Session instead.

The Secure Message functions offer two modes of operation:

In Sign/Verify mode, the message is signed using the sender's private key and is verified by the receiver using the sender's public key. The message is packed in a suitable container and ECDSA is used by default to sign the message (when RSA key is used, RSA+PSS+PKCS#7 digital signature is used).

In Encrypt/Decrypt mode, the message will be encrypted with a randomly generated key (in RSA) or a key derived by ECDH (in ECDSA), via symmetric algorithm with Secure Cell in seal mode (keys are 256 bits long).

The mode is selected by using appropriate methods. The sender uses wrap and unwrap methods for encrypt/decrypt mode. A valid public key of the receiver and a private key of the sender are required in this mode. For sign/verify mode sign and verify methods should be used. They only require a private key for signing and a public key for verification respectively.

Read more about the Secure Message's cryptographic internals here.

Secure Message interface

class SMessage(object):
   def __init__(self, private_key: bytes, peer_public_key: bytes)
   def wrap(self, message) -> bytes
   def unwrap(self, message) -> bytes

def ssign(private_key: bytes, message: bytes) -> bytes
def sverify(public_key: bytes, message: bytes) -> bytes

Description:

  • class SMessage — encrypted secure message
  • __init__(self, private_key: bytes, peer_public_key: bytes)
    Initialise Secure Message object with private_key and peer_public_key. Asymmetric keys are needed for Secure Message and Secure Session. Throws ThemisError on failure.
  • wrap(self, message: bytes) -> bytes
    Encrypt message. Return encrypted Secure Message container as binary string. Throws ThemisError on failure.
  • unwrap(self, message: bytes) -> bytes
    Decrypt encrypted Secure Message container passed as binary string (message). Return decrypted string. Throws ThemisError on failure.
  • signed Secure Message
  • ssign(private_key: bytes, message: bytes) -> bytes
    Sign message with private_key. Return signed Secure Message container as binary string. Throws ThemisError on failure.
  • sverify(public_key: bytes, message: bytes) -> bytes
    Verify binary vector contained signed secure message container (message) with public_key. Return binary string. Throws ThemisError on failure. Use public key from the same keypair as private key for verifying the message.

Example

Initialise encrypter:

from pythemis.skeygen import KEY_PAIR_TYPE, GenerateKeyPair
from pythemis.smessage import SMessage, ssign, sverify
from pythemis.exception import ThemisError

keypair1 = GenerateKeyPair(KEY_PAIR_TYPE.EC)
keypair2 = GenerateKeyPair(KEY_PAIR_TYPE.EC)

smessage = SMessage(keypair1.export_private_key(), keypair2.export_public_key())

Encrypt message:

try:
    encrypted_message = smessage.wrap(b'some message')
except ThemisError as e:
    print(e)

Decrypt message:

try:
    message = smessage.unwrap(encrypted_message)
except ThemisError as e:
    print(e)

Sign message:

For signing/verifying make sure that you use keys from the same keypair – private key for signing message and public key for verifying message.

try:
    signed_message = ssign(keypair1.export_private_key(), b'some message')
except ThemisError as e:
    print(e)

Verify message:

try:
    message = sverify(keypair1.export_public_key(), signed_message)
except ThemisError as e:
    print(e)

Secure Cell

The Secure Сell functions provide the means of protection for arbitrary data contained in stores, i.e. database records or filesystem files. These functions provide both strong symmetric encryption and data authentication mechanisms.

The general approach is that given: - input: some source data to protect, - key: a password, - context: plus an optional "context information",

Secure Cell functions will produce: - cell: the encrypted data, - authentication tag: some authentication data.

The purpose of the optional "context information" (i.e. a database row number or file name) is to establish a secure association between this context and the protected data. In short, even when the password is known, if the context is incorrect, the decryption will fail.

The purpose of the authentication data is to verify that given a correct password (and context), the decrypted data is indeed the same as the original source data.

The authentication data must be stored somewhere. The most convenient way is to simply append it to the encrypted data, but this is not always possible due to the storage architecture of an application. The Secure Cell functions offer different variants that address this issue.

By default, the Secure Cell uses the AES-256 encryption algorithm. The generated authentication data is 16 bytes long.

Secure Cell is available in 3 modes:

  • Seal mode: the most secure and user-friendly mode. Your best choice most of the time.
  • Token protect mode: the most secure and user-friendly mode. Your best choice most of the time.
  • Context imprint mode: length-preserving version of Secure Cell with no additional data stored. Should be used with care and caution.

You can learn more about the underlying considerations, limitations, and features here.

Secure Cell Seal mode interface

class SCellSeal(object):
   def __init__(self, key: bytes)
   def encrypt(self, message: bytes, context=None: bytes) -> bytes
   def decrypt(self, message: bytes, context=None: bytes) -> bytes

Description:
- __init__(self, key: bytes)
Initialise Secure Cell in seal mode with key. - encrypt(self, message: bytes, context=None: bytes) -> bytes
Encrypt message with an optional context. Return encrypted message. Throws ThemisError on failure. - decrypt(self, message: bytes, context=None: bytes) -> bytes
Decrypt message with an optional context. Return plain message. Throws ThemisError on failure.

Example

Initialise encrypter/decrypter:

from pythemis.scell import SCellSeal
scell = SCellSeal(b'password')

Encrypt:

encrypted_message = scell.encrypt(b'message', b'context') 

Decrypt:

message = scell.decrypt(encrypted_message, b'context')  

Secure Cell Token-protect Mode

Token-protect mode interface
class SCellTokenProtect(object)
   def __init__(self, key: bytes)
   def encrypt(self, message: bytes, context=None: bytes) -> (bytes, bytes)
   def decrypt(self, message: bytes, additional_auth_data: bytes, context=None: bytes) -> bytes

Description: - __init__(self, key)
Initialise Secure Cell in token protect mode with key. - encrypt(self, message: bytes, context=None: bytes) -> (bytes, bytes)
Encrypt message with an optional context. Return two binary strings containing encrypted message and token. Throws ThemisError on failure. - decrypt(self, message: bytes, additional_auth_data: bytes, context=None: bytes) -> bytes
Decrypt message with additional_auth_data (aka token) and an optional context. Return plain message. Throws ThemisError on failure.

Example

Initialise encrypter/decrypter:

from pythemis.scell import SCellTokenProtect
scell = SCellTokenProtect(b'password')

Encrypt:

encrypted_message, additional_auth_data = scell.encrypt(b'message', b'some context') 

Decrypt:

message = scell.decrypt(encrypted_message, additional_auth_data, b'some context') 

Secure Cell Context-Imprint Mode

Context-imprint mode interface
class SCellContextImprint(object):
   def __init__(self, key: bytes)
   def encrypt(self, message: bytes, context: bytes) -> bytes
   def decrypt(self, message: bytes, context: bytes) -> bytes

Description: - __init__(self, key: bytes)
Initialise Secure Cell in context imprint mode with key. - encrypt(self, message: bytes, context: bytes) -> bytes
Encrypt message with context. Return encrypted message. Throws ThemisError on failure. - decrypt(self, message: bytes, context: bytes) -> bytes
Decrypt message with context. Return plain message. Throws ThemisError on failure.

Example

Initialise encrypter/decrypter:

from pythemis.scell import SCellContextImprint
scell = SCellContextImprint(b'some password')

Encrypt:

encrypted_message = scell.encrypt(b'test message', b'test context') 

Decrypt:

message = scell.decrypt(encrypted_message, b'test context') 

Secure Session

Secure Session is a sequence- and session- dependent, stateful messaging system. It is suitable for protecting long-lived peer-to-peer message exchanges where the secure data exchange is tied to a specific session context.

Secure Session operates in two stages: session negotiation where the keys are established and cryptographic material is exchanged to generate ephemeral keys and data exchange where exchanging of messages can be carried out between peers.

You can read a more detailed description of the process here.

Put simply, Secure Session takes the following form:

  • Both clients and server construct a Secure Session object, providing:
    • an arbitrary identifier,
    • a private key, and
    • a callback function that enables it to acquire the public key of the peers with which they may establish communication.
  • A client will generate a "connect request" and by whatever means it will dispatch that to the server.
  • A server will enter a negotiation phase in response to a client's "connect request".
  • Clients and servers will exchange messages until a "connection" is established.
  • Once a connection is established, clients and servers may exchange secure messages according to whatever application level protocol was chosen.

Secure Session interface:

class SSession(object):
   def __init__(self, id: bytes, private_key: bytes, transport: TransportStruct)
   def is_established(self) -> bool
   def connect_request(self) -> bytes
   def wrap(self, message: bytes) -> bytes
   def unwrap(self, message: bytes) -> bytes
   def connect(self)
   def send(self, message: bytes) -> int
   def receive(self) -> bytes

Description:

  • __init__(self, id: bytes, private_key: bytes, transport: TransportStruct)
    Initialie Secure Session object with id, private_key and transport. Throws ThemisError on failure.
  • is_established(self) -> bool
    Check whether Secure Session connection has been established. Throws ThemisError on failure.
  • connect(self)
    Create and send a Secure Session initialisation message to peer. Returns nothing. Throws ThemisError on failure.
  • send(self, message: bytes) -> int
    Encrypt message and send it to peer. Returns internal status code. Throws ThemisError on failure.
  • receive(self) -> bytes
    Receive a message from peer, decrypt and return it. Throws ThemisError on failure.
  • connect_request(self) -> bytes
    Return a Secure Session initialisation message. Throws ThemisError on failure.
  • wrap(self, message: bytes) -> bytes
    Encrypt message for peer. Throws ThemisError on failure.
  • unwrap(self, message: bytes) -> bytes
    Decrypt message from peer. Throws ThemisError on failure.
    Unwrapped message may contain either plaintext data or a control message. If is_control property is true, the message must be sent to peer as is.

Secure Session Workflow

Secure Session can be used in two ways: - send/receive - when communication flow is fully controlled by the Secure Session object. - wrap/unwrap - when communication is controlled by the user.

Secure Session has two parties called "client" and "server" for the sake of simplicity, but they could be more precisely called "initiator" and "acceptor" — the only difference between them is in who starts the communication.

Secure Session relies on the user's passing a number of callback functions to send/receive messages — and the keys are retrieved from local storage (see more in Secure Session cryptosystem description).

Communication flow is fully controlled by the Secure Session object

Transport class for send/receive

Implement all methods in the callbacks class:

class CustomTransport(object):
    def __init__(self, *args, **kwargs)
        # init communication channel with peer

    def send(self, message):
        # send message to peer

    def receive(self, buffer_length):
        # wait and receive at most buffer_length bytes from peer 
        return accepted_message

    def get_pub_key_by_id(self, user_id):
        # retrieve public key for peer user_id from trusted storage (file, db etc.)    
        return public_key
Secure Session client

First, initialise the session:

from pythemis.ssession import SSession
session = SSession(b'some client id', client_private_key, CustomTransport())
session.connect()
while not session.is_established():
    session.receive()

After the loop finishes, Secure Session is established and is ready to be used.

To send a message over the established session, use:

session.send(message)

To receive a message from the session:

message = ssession.receive()
Secure Session server

First, initialise the session:

session = SSession(b'some server id', server_private_key, CustomTransport())
# there is no need to call connect() method on the server side
while not session.is_established():
    session.receive()

Sending/receiving messages works in a manner similar to the client side after the connection is established.

To encrypt and send an outgoing message to the client use:

session.send(message)

To receive and decrypt a message from the client use:

message = session.receive()

Communication controlled by user

Transport class for wrap/unwrap

Implement only required methods in the callback class:

from pythemis.ssession import MemoryTransport


class CustomSimpleTransport(MemoryTransport):
   def __init__(self, *args, **kwargs)
       # initialize trusted public keys storage
       super(CustomSimpleTransport, self).__init__()       

   def get_pub_key_by_id(self, user_id):
       # retreive public key for peer user_id from trusted storage (file, db etc.)    
       return public_key
Secure Session client

First, the initialisation:

session = SSession(b'user_id2', client_private_key, CustomSimpleTransport())
# this call is only made by the client
encrypted_message = session.connect_request()

# send connect request to the peer
response_bytes = user_communication_send_method(encrypted_message)    
message = session.unwrap(response_bytes)

# establish the session
while not session.is_established():
    response_bytes = user_communication_send_method(message)    
    message = session.unwrap(response_bytes)

After the loop finishes, Secure Session is established and is ready to be used.

To encrypt the outgoing message, use:

encrypted_message = session.wrap(message)
# send encrypted_message to peer by any prefered method

To decrypt the received message, use:

# receive encrypted_message from peer 
message = session.unwrap(encrypted_message)
Secure Session server

First, initialise everything:

session = SSession(b'server_id_1', server_private_key, CustomSimpleTransport())
# there is no need to call connect() or connect_request() on the server side

encrypted_message = user_communication_recieve_method() 
message = session.unwrap(encrypted_message)

# NOTE: The condition is different for the server because we need to send
# the last piece of data to the client after establishing the session.
while message.is_control:
    # just return the unwrapped message to the user
    user_communication_send_method(message)

    encrypted_message = user_communication_recieve_method() 
    message = session.unwrap(encrypted_message)

Secure Session is ready.

Send/receive works in the same way as the client's example above.

Secure Comparator

Secure Comparator is an interactive protocol for two parties that compares whether they share the same secret or not. It is built around a Zero Knowledge Proof-based protocol (Socialist Millionaire's Protocol), with a number of security enhancements.

Secure Comparator is transport-agnostic and only requires the user(s) to pass messages in a certain sequence. The protocol itself is ingrained into the functions and requires minimal integration efforts from the developer.

Secure Comparator interface

class SComparator:
   def __init__(self, shared_secret: bytes)
   def begin_compare(self) -> bytes
   def proceed_compare(self, message: bytes) -> bytes
   def is_compared(self) -> bool
   def is_equal(self) -> bool
   def result(self)

Description:
- __init__(self, shared_secret: bytes)
Initialise Secure Comparator object with a shared_secret. Throws ThemisError on failure. - begin_compare(self)
Return a Secure Comparator initialisation message. Throws ThemisError on failure. - proceed_compare(self, message)
Process message and create the next protocol message (when necessary). Throws ThemisError on failure. - is_compared(self) -> bool
Return True if comparison is finished. Throws ThemisError on failure. - is_equal(self) -> bool
Return True if comparison is finished and secrets are equal, otherwise return False. Throws ThemisError on failure. - result(self) -> int
Return the original comparison result code. Throws ThemisError on failure.

Secure Comparator workflow

Secure Comparator has two parties — called "client" and "server" — the only difference between them is in who starts the comparison.

Secure Comparator client

from pythemis.scomparator import SComparator


comparator = SComparator(b'shared_secret')

# this call is specific to the client
comparison_message = comparator.begin_compare()

while not comparator.is_compared():
    user_send_function(comparison_message)
    response = user_recieve_function()
    comparison_message = comparator.proceed_compare(response)

After the loop finishes, the comparison is over and its result can be checked by calling comparator.is_equal().

Secure Comparator server

from pythemis.scomparator import SComparator


comparator = SComparator(b'shared_secret')

while not comparator.is_compared():
     comparison_message = user_receive_function()
     comparison_message = comparator.proceed_compare(comparison_message)
     user_send_function(comparison_message)

After the loop finishes, the comparison is over and its result can be checked by calling comparator.is_equal().