What is AcraTranslator

AcraTranslator is a lightweight server that receives AcraStructs and returns the decrypted data. This element of Acra is necessary in the use-cases when applications store the encrypted data as separate blobs (files that are not in a database - i.e. in the S3 bucket, local file storage, etc.).

By its nature, AcraTranslator is a separate daemon that runs in an isolated environment (separate virtual machine or physical server). AcraTranslator is responsible for holding all the secrets required for data decryption and for actually decrypting the data.

AcraTranslator doesn't care about the source of the data, it accepts AcraStructs via HTTP or gRPC API. An application can store AcraStructs anywhere it is convenient: as cells in the database, as files in the file storage (local or cloud storage, like S3). An application sends AcraStructs as binary data and receives plaintext (or decryption error) from AcraTranslator.

However, sending plaintext data via a non-secure channel is a bad idea, so AcraTranslator requires the use of Themis Secure Session encryption channel (which is basically encrypted TCP/UNIX sockets). To establish a Secure Session connection, an application doesn't need to include the crypto-code itself, only to direct the traffic through AcraConnector instead.

AcraConnector — a client-side daemon, that is responsible for providing an encrypted and authenticated connection between application and AcraTranslator. AcraConnector runs under a separate user / in a separate container and acts as a proxy/middleware between application and AcraTranslator. It accepts connection from the application, adds extra transport encryption layer using Themis Secure Session, sends data to AcraTranslator, receives the result and sends it back to the application. AcraConnector is an optional component and can be replaced with TLS v1.2, however, Themis Secure Session provides better security guarantees 'out of the box' than average TLS configuration found in a wild.

You can use both AcraServer and AcraTranslator in your application, they can share the decryption key.

Architecture and DataFlow of AcraTranslator-based infrastructure

Architecture and DataFlow of AcraTranslator-based infrastructure

Architecture and DataFlow of AcraTranslator-based infrastructure.

Acra design values simplicity as much as security. Decrypting the data should not require much tweaking — just run AcraTranslator, point its address to AcraConnector, and you're good to go:

  1. The application sends AcraStructs to AcraConnector via HTTP or gRPC API.
  2. AcraConnector sends that request to AcraTranslator using Secure Session (socket protection protocol).
  3. AcraTranslator accepts the request and attempts to decrypt an AcraStruct. If the decryption is successful, it sends the resulting plaintext in the response. If decryption fails, AcraTranslator sends out a decryption error.
  4. AcraTranslator returns the data to AcraConnector (with Secure Session), which in turn returns it to the application.

AcraTranslator needs a separate transport keypair to initialise the Secure Session and a storage private key to decrypt the data.

AcraTranslator reads decryption keys from key folder and stores them in memory in encrypted form. It uses LRU cache to increase performance by keeping only actively used keys in memory. The size of LRU cache can be configured depending on your server's load.

Configuration parameters

AcraTranslator can work either using HTTP API or gRPC API, but not simultaneously.

--config_file
    path to config
-d  Log everything to stderr
--dump_config
    dump config
--generate_markdown_args_table
    Generate with yaml config markdown text file with descriptions of all args
--incoming_connection_close_timeout
    Time that AcraTranslator will wait (in seconds) on stop signal before closing all connections (default 10)
--incoming_connection_grpc_string
    Default option: connection string for gRPC transport like grpc://0.0.0.0:9696
--incoming_connection_http_string
    Connection string for HTTP transport like http://0.0.0.0:9595
--incoming_connection_prometheus_metrics_string
    URL which will be used to expose Prometheus metrics (use <URL>/metrics address to pull metrics)
--jaeger_agent_endpoint
    Jaeger agent endpoint that will be used to export trace data (default "127.0.0.1:6831")
--jaeger_basic_auth_password
    Password used for basic auth (optional) to jaeger
--jaeger_basic_auth_username
    Username used for basic auth (optional) to jaeger
--jaeger_collector_endpoint
    Jaeger endpoint that will be used to export trace data (default "127.0.0.1:14268")
--keys_dir
    Folder from which will be loaded keys (default ".acrakeys")
--keystore_cache_size
    Count of keys that will be stored in in-memory LRU cache in encrypted form. 0 - no limits, -1 - turn off cache
--logging_format
    Logging format: plaintext, json or CEF (default "plaintext")
--poison_detect_enable
    Turn on poison record detection, if server shutdown is disabled, AcraTranslator logs the poison record detection and returns error (default true)
--poison_run_script_file
    On detecting poison record: log about poison record detection, execute script, return decrypted data
--poison_shutdown_enable
    On detecting poison record: log about poison record detection, stop and shutdown
--securesession_id
    Id that will be sent in secure session (default "acra_translator")
--tracing_jaeger_enable
    Export trace data to jaeger
--tracing_log_enable
    Export trace data to log
-v  Log to stderr all INFO, WARNING and ERROR logs

gRPC API

gRPC API is the recommended way of using AcraTranslator for high-load services. Each gRPC request contains a client_id, a zone_id, and an AcraStruct itself.

The Client Id (client_id) is defined as one of the AcraConnector's CLI parameters. The Client Id determines what client public key was used for creating the AcraStruct. Due to some peculiarities of the gRPC protocol, it's required to send a client_id in every request, so we'll have to send the Client Id in every request. Client Id is a required parameter.

Zone Id (zone_id) determines which zone identifier was used for creating the AcraStruct. Zone Id is an optional parameter and can be omitted.

See a gRPC’s request and response structure below:

// cmd/acra-translator/grpc_api/api.proto

syntax = "proto3";

message DecryptRequest {
   bytes client_id = 1;
   bytes zone_id = 2;
   bytes acrastruct = 3;
}

message DecryptResponse {
   bytes data = 1;
}

service Reader {
   rpc Decrypt(DecryptRequest) returns (DecryptResponse) {}
}


service Reader {
   rpc Decrypt(DecryptRequest) returns (DecryptResponse) {}
}

HTTP API

HTTP API is useful for the non-highload services. Each HTTP API request contains AcraStruct and a Zone Id required for decrypting an AcraStruct. The server responds with decrypted data or with a decryption error. HTTP API is a recommended way of debugging AcraTranslator's configuration (to do it, you need to check your connection and make sure that keys are placed in the correct folders).

The Client Id (client_id) is defined as one of in the AcraConnector's CLI parameters. Client Id determines what client public key was used for creating the AcraStruct.

Zone Id (zone_id) determines which Zone Id was used for creating the AcraStruct. The Zone Id is an optional parameter and can be omitted.

Creating a request for decrypting an AcraStruct

1. Create a POST request to the /v1/decrypt URI. 2. Add a query parameter zone_id=.

Example: For a zone with id "gsj312vsd783":

POST <AcraConnectorHost:Port>/v1/decrypt?zone_id=gsj312vsd783

3. If AcraStruct was encrypted without a Zone, you can omit the zone_id part completely:

POST <AcraConnectorHost:Port>/v1/decrypt 

4. Add the AcraStruct binary data to the body of the request.

5. Add the following HTTP headers:

Content-Type. Set to the MIME media type to application/octet-stream if you're sending a binary AcraStruct.

Content-Length. Set to the number of bytes you are uploading.

6. Send the request:

POST <AcraConnectorHost:Port>/v1/decrypt?zone_id=<zone_id_string>
Content-Type: application/octet-stream
Content-Length: [NUMBER_OF_BYTES_IN_FILE]
[ACRASTRUCT]

Sending request to decrypt AcraStruct (Example)

The following example shows a simple request for decryption of AcraStruct encrypted using a Zone Id heh3547h:

POST <AcraConnectorHost:9595>/v1/decrypt?zone_id=heh3547h
Content-Type: application/octet-stream
Content-Length: [NUMBER_OF_BYTES_IN_FILE]
[ACRASTRUCT]

If the request is successful, the server will return the HTTP 200 OK status code along with the decrypted data in the body:

HTTP/1.1 200
Content-Type: application/octet-stream
[DECRYPTED DATA]

If the request fails, the server will return an appropriate HTTP code along with a user-friendly error message in the body, for example:

HTTP/1.1 400
Content-Type: text/plain
HTTP request version is not supported: expected v1, got v4

Handling errors and troubleshooting

When using AcraTranslator service, make sure you understand error codes and take appropriate actions.

  1. API versioning. Remember to indicate the correct API version into the request URI. The latest available version is /v1.If the request has no version or features an unsupported version number, the server will return HTTP Code 400 BAD REQUEST, and error string in the body of the message.
  2. HTTP Code 400 BAD REQUEST is the result of an invalid (misconfigured) request URL or of a missing body. The possible reasons could be: unsupported parameters' name, unsupported endpoint, empty body, mismatch of the content's length and the body length. See the corresponding source file to learn about the checks and the error messages. To fix the errors of this kind, please make double sure that your application sends correct parameters.
  3. HTTP Code 422 Unprocessable Entity is the result of an error during decryption. The possible causes for getting this error could be: missing or invalid (wrong) decryption keys, invalid AcraStruct structure (if an AcraStruct is corrupt or the binary data doesn't have an AcraStruct header at all), a detection of a poison record (being addressed). For the security reasons, AcraTranslator doesn't return the real cause of the error message, masking it with a generic message "Can't decrypt AcraStruct". The underlying error is being logged to the AcraTranslator console/log file, so the only person who has the access to logs can see the error message.

We suggest that you see the reasons causing errors in the AcraTranslator logs while you are setting up the environment and fine-tune the configuration file appropriately (inputting the correct keys, setting up alarms upon poison record detection, etc.).

Setup AcraConnector and AcraTranslator using Docker

AcraTranslator and AcraConnector are available as a Docker image which you can download from the Cossack Labs official Docker repository or you can find it in the Docker folder in the GitHub repository.

Note: Using Docker is recommended for testing purposes only. Please don't rely on Docker in real-life production settings. We provide docker-compose files with pre-generated keys, that are visible for everyone.

Docker setup for HTTP API

1. Clone the Acra repository, build images and start Docker compose with AcraConnector and AcraTranslator in HTTP mode, and Secure Session between them:

git clone https://github.com/cossacklabs/acra.git
make docker
docker-compose -f docker/docker-compose.translator-ssession-connector-http.yml up

This command generates keys for AcraTranslator, AcraConnector, AcraWriter and put them into docker/.acrakeys folder.

2. Generate AcraStruct using keys from docker/.acrakeys and small python script from tests folder:

python tests/generate_acrastruct.py --data="my acrastruct"

Output file is testclientid.acrastruct.

3. Check that AcraStruct is generated and has data inside:

cat ./testclientid.acrastruct | base64
IiIiIiIiIiJVRUMyAAAALfazgygDi8YvUMWlO0et6E3ETg3RQKfIbN2y0W7OD7ZyQwzrhqogJwQmVAAAAAABAUAMAAAAEAAAACAAAABixXN9vEHTHVpZV1V21JjfL3YuLyxVNAPFTIuekhBcC5ktS9oNToTt8eNULtilcXCF1cce2oaGLenU07A3AAAAAAAAAAABAUAMAAAAEAAAAAsAAADzSwk1Ndtyyg8LLmBS0N9sZ21afg4NPZhaNLc7b55WJF4ROxCf4pc=

4. Send AcraStruct to AcraConnectors' address:

curl --request POST --data-binary @testclientid.acrastruct http://127.0.0.1:9494/v1/decrypt

5. Receive the result: “my acrastruct”.

Docker setup for gRPC API

1. Clone the Acra repository, build images and start Docker compose with AcraConnector and AcraTranslator in gRPC mode, and Secure Session between them:

git clone https://github.com/cossacklabs/acra.git
make docker
docker-compose -f docker/docker-compose.translator-ssession-connector-grpc.yml up

This command generates keys for AcraTranslator, AcraConnector, AcraWriter and put them into docker/.acrakeys folder.

2. Generate AcraStruct using keys from docker/.acrakeys and small python script from tests folder:

python tests/generate_acrastruct.py --data="my acrastruct"

Output file is testclientid.acrastruct.

3. Check that AcraStruct is generated and has data inside:

cat ./testclientid.acrastruct | base64
IiIiIiIiIiJVRUMyAAAALfazgygDi8YvUMWlO0et6E3ETg3RQKfIbN2y0W7OD7ZyQwzrhqogJwQmVAAAAAABAUAMAAAAEAAAACAAAABixXN9vEHTHVpZV1V21JjfL3YuLyxVNAPFTIuekhBcC5ktS9oNToTt8eNULtilcXCF1cce2oaGLenU07A3AAAAAAAAAAABAUAMAAAAEAAAAAsAAADzSwk1Ndtyyg8LLmBS0N9sZ21afg4NPZhaNLc7b55WJF4ROxCf4pc=

4. Send testclientid.acrastruct to AcraConnector's address using gRPC client.

Setup AcraConnector and AcraTranslator manually

Step-by-step instruction to download and run every service, generate every key. Secure, but a long way.

Manual setup for HTTP API

1. Generate the (Master Key)[/pages/documentation-acra/#generating-all-the-acra-keys-in-one-go].
2. Put AcraWriter's private key (the decryption key used for decryption of AcraStructs) into the KeyStore for AcraTranslator. The Key's default name is <client_id>_storage (by default, the key should be placed into the KeyStore - by default that is a folder with .acrakeys name. You can change this key folder to any other folder).
3. Generate the transport keys. AcraConnector and AcraTranslator should have appropriate keypairs for initializing the Secure Session connection. Use the same client_id that AcraWriter was using while encrypting the data.

$GOPATH/bin/acra-keymaker --client_id=client --generate_acratranslator_keys --generate_acraconnector_keys

Put _translator.pub into the AcraConnector keys' folder and also put .pub into the AcraTranslator keys' folder.

4. Start AcraConnector:

$GOPATH/bin/acra-connector --mode=acratranslator --client_id=client --acratranslator_securesession_id=acra_translator --incoming_connection_string=tcp://127.0.0.1:8000 --acratranslator_connection_string=tcp://127.0.0.1:9595

5. Start AcraTranslator using HTTP API:

$GOPATH/bin/acra-translator --securesession_id:acra_translator --incoming_connection_http_string=tcp://127.0.0.1:9595

6. Generate AcraStruct using AcraWriter's public key and handy python utility from tests folder:

python tests/generate_acrastruct.py --client_id=client --keys_dir=.acrakeys --data="my acrastruct struct struct struct"

Output file is client.acrastruct.

7. Check that AcraStruct is generated and has data inside:

cat ./client.acrastruct | base64
IiIiIiIiIiJVRUMyAAAALfazgygDi8YvUMWlO0et6E3ETg3RQKfIbN2y0W7OD7ZyQwzrhqogJwQm
VAAAAAABAUAMAAAcvvldAAABixXN9vEHTHVpZV1V21JjfL3YuLyxVNAPFTIuekhBcC5ktS9oN
ToTt8eNULtifdhgkyGLenU07A3AAAAAA48fb03MAAAAEAAAAAsAAADzSwk1Ndtyyg8L
LmBS0N9sZ21afg4NPZhaNLc7b55WJF4ROxCf4pc=

8. Send a HTTP request with AcraStruct:

curl -X POST --data-binary @client.acrastruct --header "Content-Type: application/octet-stream" http://127.0.0.1:8000

9. Receive the result 'my acrastruct struct struct struct';

Manual setup for gRPC API

1.-4. Repeat the steps 1-4 for the manual setup using HTTP (above).

5. Start AcraTranslator using gRPC API:

$GOPATH/bin/acra-translator --securesession_id:acra_translator --incoming_connection_grpc_string=tcp://127.0.0.1:9595

6. Generate AcraStruct using AcraWriter's public key and handy python utility from tests folder:

python tests/generate_acrastruct.py --client_id=client --keys_dir=.acrakeys --data="my acrastruct struct struct struct"

Output file is client.acrastruct.

7. Check that AcraStruct is generated and has data inside:

cat ./client.acrastruct | base64
IiIiIiIiIiJVRUMyAAAALfazgygDi8YvUMWlO0et6E3ETg3RQKfIbN2y0W7OD7ZyQwzrhqogJwQm
VAAAAAABAUAMAAAcvvldAAABixXN9vEHTHVpZV1V21JjfL3YuLyxVNAPFTIuekhBcC5ktS9oN
ToTt8eNULtifdhgkyGLenU07A3AAAAAA48fb03MAAAAEAAAAAsAAADzSwk1Ndtyyg8L
LmBS0N9sZ21afg4NPZhaNLc7b55WJF4ROxCf4pc=

8. For your convenience, you can generate the client code from the service description. The most simple and convenient way is to use the file we are providing ("cmd/acra-translator/grpc_api/api.proto") and generate the client code using our service definition.

For Python it will look like this:

python -m grpc_tools.protoc -I cmd/acra-translator/grpc_api --python_out=destination_folder/ --grpc_python_out=destination_folder/ cmd/acra-translator/grpc_api/api.proto

After that, you will have api_pb2.py and api_pb2_grpc.py files in the which you passed in the command above

9. Use the generated client to send the request with client.acrastruct from your Python code:

import grpc
import api_pb2_grpc
import api_pb2

with open(“/path/to/acrastruct_in_file” “rb”) as f:
 acrastruct = f.read()

channel = grpc.insecure_channel('127.0.0.1:8000')
stub = api_pb2_grpc.ReaderStub(channel)

# client_id == defined client_id in AcraConnector cli args
response = stub.Decrypt(api_pb2.DecryptRequest(client_id=”client”, acrastruct=client.acrastruct)

print(response.data)

Poison records and AcraTranslator

If the client application is hacked and the attacker is trying to decrypt all the AcraStructs, you can detect it using poison records.

The client application adds poison records to the database. In a normal application flow, poison records should never be read.

If a malicious application tries to grab the data, AcraServer will detect poison records and stop executing the query, preventing the data from leaking to an untrusted destination.

Please see a separate documentation piece on generating poison records.

AcraTranslator has three parameters related to poison records:

  • --poison_detect_enable- if true, AcraTranslator will try to decrypt AcraStruct using poison record keys if the decryption with storage keys fails. Upon detecting a poison record, AcraTranslator will log "Recognized poison record".

If false, AcraTranslator won't try to decrypt AcraStructs with poison record keys.

The default value is true.

  • --poison_run_script_file - if set, AcraTranslator will execute script upon detecting a poison record.
  • --poison_shutdown_enable - if set to true, AcraTranslator will shutdown upon detecting a poison record.

Remember: AcraTranslator should have a poison record's private key in its KeyStorage to be able to detect the poison records.