tdsa-service

Getting Started

The tdsa-service docker image can be downloaded from the Sepior docker repository. Just type:

docker pull nexus.sepior.net:19000/tdsa-service:VERSION

You will need credentials for the Sepior Nexus server.

Once the TDSA service is running, API documentation is available at the following URL:

http://<hostname>:<api port>/docs

Communication

The TDSA service listens on three separate TCP ports.

Protocol Port

This port is used by the MPC protocols and must be exposed to the other participants (both client and other servers). The protocol used is HTTP or HTTPS. HTTPS is recommended, but not required. You will need to provide your own certificate.

API Port

The service on this port accepts key management-related requests such as key registration, key share generation, presignature generation, etc. This mirrors the operations in the TDSA library. Access to this port must be restricted and communication should preferably be done using HTTPS.

Administration Port

This port is used for health checks and for extracting metrics from both the TDSA service and the underlying TDSA library. This port should only be accessible by the operations team, load balancer etc.

Service Startup

The Sepior TDSA service is delivered as a Docker image. Configuration is done via a configuration file which can be passed to the container in two ways:

Configuration

1. Base64 encodes the contents of the configuration file and passes it to the container in the environment variable TDSA_CONFIG.
2. If the environment variable above is not set or empty the TDSA service will look for the files /config/config.yml and /config/config.yaml. In other words, you will have to mount a folder containing config.yaml or config.yml on /config.

Database Tables

When the service starts up it will try to create the required database tables. If the database user is not allowed to create tables, one will need to create them first and then disable table creation at startup (see the service configuration section).

To create tables manually one needs to set four environment variables DB_DRIVER_CLASS, DB_URL, DB_USERNAME and DB_PASSWORD, and then start the Docker container with the command create-tables or create-tables-no-startup. The former command will attempt to create database tables using information from the environment variables, and then continue starting the TDSA service. The latter will attempt to create the tables and exit afterwards.

Example

docker run -e DB_DRIVER_CLASS="org.postgresql.Driver" -e DB_URL="jdbc:postgresql://postgres.example.com:5432/mydb" -e DB_USERNAME="myuser" -e DB_PASSWORD="mypassword" tdsa-service:latest create-tables

Service Configuration

The TDSA service is configured through a YAML configuration file. Since it was built with Dropwizard the configuration file format matches that of Dropwizard. The following configuration sections are specific to the TDSA service:

• tdsaLibrary: Configuration of the underlying TDSA library.
• tdsaServer: Configuration for the HTTP server that is specific to the TDSA service.
• vault: Configuration of an external HashiCorp Vault that can be used to store configuration values in a secure way.

Furthermore, the following standard Dropwizard configuration sections should also be configured for proper operation in a production environment:

• server: Configuration of the HTTP server.
• httpClient: Configuration of the HTTP client instance that is used to communicate with other TDSA service participants.
• logging: Configuration of logging.
• database: Configuration of the database used for key storage.
• health: Configuration of health checks.

Note that binary data in the configuration file is always url-safe base64 encoded without padding.

TDSA Library Configuration (tdsaLibrary)

This section configures the TDSA library used by the TDSA service. For more information about each configuration option, please consult the java-tdsa-lib documentation.

Each remote participant contains the following keys:

The storage configuration has the following keys:

Example

tdsaLibrary:
  remoteParticipants:
    -
    - - id: participant1
        uri: http://service1:9080
        primaryPublicKey: MCowBQYDK2VuAyEAgx875KMt0MA6wvzfSl598V7pCF4Sv_40Xc3aBt5IiH4
        secondaryPublicKey: MEIwBQYDK2VvAzkAf18tl_XyLzqoKmmi-9Q1BqNFWmi-yQKRgdFQ0pvD5hnM5P1xnhtqEa8zxAjU0cx4lO5GznJSAHM
    - - id: participant2
        uri: http://service2:9080
        primaryPublicKey: MCowBQYDK2VuAyEANEZj3cKTbrPIp-AFHiNgaidGzNa9bOMozwUIxVahk0w
        secondaryPublicKey: MEIwBQYDK2VvAzkAovrwuTlBqPMEUB99_ykABPewVIoe51RLZGHCZ-hh4DD_6gjDMA3KfjjIfg2IjNLXjL0NiqVqVuY
  participantIds:
    - 
    - participant1
    - participant2
  clientIndex: 0
  participantIndex: 1
  participantPrimaryPrivateKey: MC4CAQAwBQYDK2VuBCIEIPDcEiDMWAq9GWo3kIbOHoH_WCO5XPfoM_UusfYx5Hhv
  participantSecondaryPrivateKey: MEYCAQAwBQYDK2VvBDoEOJwWgGHPGnVUTHQTu2awOe2flV6PCu7lPN3gcn5X9M889TbKycBQt8vUYZcSG3-9JBn0CdPtwTzH
  sessionTimeout: 60
  protocolExecutionThreads: 8
  protocolCommunicationThreads: 4
  storageConfiguration:
    skipCreateTables: false
    cacheSize: 100
    partialSignatureTimeout: 3600
    encryptionKey: topSecretEncryptionKey

TDSA Server Configuration (tdsaServer)

This configuration section contains specific options that apply only to the TDSA service.

Example

tdsaServer:
  serverGroupId: server_group_id_string

Vault Configuration (vault)

The TDSA service supports retrieving selected configuration values from a HashiCorp Vault. The vault instance must be configured through this configuration section to use this feature. The following options are available.

Example

vault:
  address: http://127.0.0.1:8200
  token: s.daMl067mv8VF8KtsIsk5CvHt
  unwrap: false
  engineVersion: 2

Retrieving Configuration Values From Vault

The following configuration keys can be fetched from the vault:

• tdsaLibrary.participantPrimaryPrivateKey
• tdsaLibrary.participantSecondaryPrivateKey
• tdsaLibrary.encryptionKey
• tdsaServer.serverGroupId
• database.user
• database.password

Two kinds of vault secrets engines are supported: The K/V engine (version 1 or 2) and the database engine. To retrieve a value from the vault, specify a selector in the configuration option. The selector has the following format:

K/V selector

• A prefix: "vault:" which is used to tell that the value is stored in the vault.
• A secrets engine selector. Here "kv:".
• Path to the Secrets document.
• A literal at-sign, "@".
• The key to select from the secrets document.

Example: To fetch the participantPrivateKey from the vault use the following value as the selector:

vault:kv:secret/tdsalibrary@participantPrivateKey

Database selector

• A prefix: "vault:" which is used to tell that the value is stored in the vault.
• A secrets engine selector. Here "database:".
• Path of the database engine.
• A literal at-sign, "@".
• The name of the database role

Example. To configure PostgreSQL credentials use the following values in the database configuration:

user: vault:database:database@my-role@username
password: vault:database:database@my-role@password

If only one of username and password is set, both the username and the password are still fetched from the vault.

If the database secrets engine has a TTL set, the TDSA service will renew database credentials before they expire.

HTTP Server Configuration (server)

This configuration section configures the Dropwizard HTTP server. Refer to the Dropwizard documentation for the different options.

TDSA Service Specific HTTP Server Configuration

The TDSA service requires two defined applicationConnectors. Implicitly, the first application connector is used for protocol communication, the second connector is used for the management API.

In order to use the health checks and metrics available, there must be an adminConnectors defined. This connector is only used for the health check and metrics functionality and can be omitted if this is not needed.

Example

server:
  maxThreads: 100
  minThreads: 10
  maxQueuedRequests: 50
  allowedMethods: [ HEAD, GET, PUT, POST, OPTIONS, DELETE ]
  requestLog:
    appenders: []
  applicationConnectors:
    - type: http
      port: 9080
    - type: http
      port: 8080
  adminConnectors:
    - type: http
      port: 8081

HTTP Client Configuration (httpClient)

This configuration section configures the Dropwizard HTTP client. Refer to the Dropwizard documentation for the different options.

Example

httpClient:
  timeout: 30s
  connectionTimeout: 10s
  timeToLive: 60s
  cookiesEnabled: false
  maxConnections: 1024
  maxConnectionsPerRoute: 1024
  validateAfterInactivityPeriod: 30s
  keepAlive: 30s
  retries: 3

Logging (logging)

This configuration section configures the logging within the TDSA service. Refer to the Dropwizard documentation for the different options.

Example

logging:
  level: INFO
  appenders:
    - type: console
      threshold: TRACE
      target: stdout
  loggers:
    "org.reflections": ERROR
    "org.apache": WARN
    "org.apache.axis2": WARN
    "org.apache.axiom": WARN
    "httpclient.wire": WARN
    "com.sepior.tdsa": INFO
    "com.sepior.tdsaservice": INFO

Database Configuration (database)

This configuration section configures the database used by the TDSA service. Refer to the Dropwizard documentation for the different options.

The TDSA service supports the following databases:

The user and password database options supports fetching credentials from vault if this is configured.

Example

database:
  driverClass:
    org.postgresql.Driver
  url:
    jdbc:postgresql://127.0.0.1:5432/database_name
  user: database_username
  password: database_password
  properties:
    charSet: UTF-8

Health Check Configuration (health)

Refer to the Health Check and Metrics section.

Health Check and Metrics

The TDSA service serves health information on the administration port. The health check runs a number of checks to check if the system is operating normally. The health checks are customizable through the service configuration.

The URI of the health check endpoint is:

http://<tdsa-service-ip>:<administration-port>/healthcheck

If all of the health checks succeed the endpoint will return 200 OK. If any of the checks fail it returns 500 Server Error. The failing health check will be reported in the response along with a message.

The TDSA service also serves as a JSON document with performance counters, timers and other numbers. This metrics document can be accessed on the administration port and can be used to monitor how many operations the service is performing per second.

The URI of the metrics report is:

http://<tdsa-service-ip>:<administration-port>/metrics

Configuring Health Checks

The health check endpoint uses a configurable list of health checks along with two internal ones. The internal ones check for database connectivity and that no deadlocks have occurred in the application. The health checks can be configured through the health object in the configuration file.

The health object contains a list of monitored metrics in the monitor array. Each monitor looks at a metric and uses a statistic such as the mean rate or exponentially decaying moving average over the last 15 minutes, etc. The monitor checks the currently measured value against a minimum and/or maximum value. If either is exceeded the monitor reports a failure. The accurate list of metric names is obtained through the metrics endpoint.

An example configuration

health:
  monitors:
    - name: max_aborted_presiggen
      metric: com.sepior.tdsa.ABORTED.PRESIG
      statistic: m5_rate
      max: 5.0

This example monitors the rate of aborted presignatures in the TDSA library by looking at a moving average over the past 5 minutes. The health check is configured to fail if there are more than 5 aborts per second. The key "min" is supported at the same level as "max" and will report a failure if the value is less than the minimum value.

Metric types

There are 4 different types of metrics that are exposed through the metrics endpoint and available to the health check configuration. The types are meters, timers, counters and histograms. Each metric type has a different set of statistics to choose from. The following matrix lists the different statistics and whether a metric supports it.

Authentication Keys

A primary and secondary key pair in the correct format can be generated with the following script. This requires OpenSSL 1.1.1.

#!/bin/bash

x25519_private_key_file="$(mktemp)"
x25519_public_key_file="$(mktemp)"
x448_private_key_file="$(mktemp)"
x448_public_key_file="$(mktemp)"

function finish {
  rm -rf "${x25519_private_key_file}" "${x25519_public_key_file}" "${x448_private_key_file}" "${x448_public_key_file}"
}
trap finish EXIT

openssl genpkey -algorithm X25519 -outform DER -out "${x25519_private_key_file}"
openssl pkey -inform DER -in "${x25519_private_key_file}" -pubout -outform DER -out "${x25519_public_key_file}"
openssl genpkey -algorithm X448 -outform DER -out "${x448_private_key_file}"
openssl pkey -inform DER -in "${x448_private_key_file}" -pubout -outform DER -out "${x448_public_key_file}"

echo "Primary Public key:    $(openssl base64 -A < "${x25519_public_key_file}" | sed 's@+@-@g; s@/@_@g; s@=@@g')"
echo "Primary Private key:   $(openssl base64 -A < "${x25519_private_key_file}" | sed 's@+@-@g; s@/@_@g; s@=@@g')"
echo "Secondary Public key:  $(openssl base64 -A < "${x448_public_key_file}" | sed 's@+@-@g; s@/@_@g; s@=@@g')"
echo "Secondary Private key: $(openssl base64 -A < "${x448_private_key_file}" | sed 's@+@-@g; s@/@_@g; s@=@@g')"

Load Balancing

If a single tdsa-service is not sufficient to handle the load, then multiple tdsa-service instances can be run in parallel. This requires a load balancer and that all tdsa-service instances share a common database.

The connectors on the tdsa-service require different load-balancing strategies:

The load balancer can terminate TLS connections and forward requests to the tdsa-service instances over plain HTTP. Note that data sent over the protocol port are still encrypted and authenticated on the application level.

Database Encryption

The TDSA library supports using a custom implementation of the storage encryptor. To use this with the TDSA service the implementation must be packaged as one or more jar files. These jar files must be placed in a folder which is mounted in the container under the path /tdsa/lib.

The configuration is similar to that of the TDSA library.

API Port Authentication

As mentioned before, access to the API port must be restricted. One way of doing this is to use TLS with client certificates. The script at the end of this section can be used to generate keys and certificates for such a setup. It should be invoked with the following command:

generate-certificates.sh output_dir hostname1 hostname2, ..., hostnameN

Where output_dir is the directory where the generated files should be placed. This is followed by a list of host names to generate server certificates. The keystores are generated with a password of changeit. You can either change the password in the script or change the password afterwards using Java's keytool command.

The script will generate the following files:

Once the files are generated the API port should be configured for HTTPS with client authentication. Here is an example of a production-ready configuration that works with the tdsa-service:

server:
  applicationConnectors:
    - type: http
      port: 9080
    - type: https
      port: 8443
      keyStorePath: output_dir/keystore.www.example.com.pkcs12
      keyStorePassword: changeit
      trustStorePath: output_dir/truststore.pkcs12
      trustStorePassword: changeit
      needClientAuth: true
      maxCertPathLength: 1
      jceProvider: Conscrypt
      supportedProtocols: [ TLSv1.2 ]
      supportedCipherSuites: [ TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256, TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256, TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384 ]

Here is the complete script:

#!/bin/bash

set -e

if [ "$#" -lt 2 ]; then
  echo "Usage: $0 <output_directory> <hostname> [hostnames...]"
  exit 1
fi

output_directory="$1"
if [ ! -d "${output_directory}" ]; then
  echo "Output directory does not exist"
  exit 1
fi
shift 1

for cmd in openssl keytool; do
  if ! command -v "${cmd}" 1>/dev/null; then
    echo "Command not found in path: ${cmd}"
    exit 1
  fi
done

ca_cert="${output_directory}/ca.crt"

private_key_file="$(mktemp)"
private_key_temp_file="$(mktemp)"
cert_req_file="$(mktemp)"
cert_file="$(mktemp)"
config_file="$(mktemp)"
serial_file="$(mktemp -u)"
ca_key_file="$(mktemp)"

function finish {
  rm -f "${private_key_file}" "${private_key_temp_file}" "${cert_req_file}" "${cert_file}" "${config_file}" "${serial_file}" "${ca_key_file}"
}
trap finish EXIT

function generate_private_key() {
  local -r private_key="$1"
  openssl ecparam -name prime256v1 -genkey -param_enc named_curve -outform PEM -out "${private_key_temp_file}"
  openssl pkcs8 -topk8 -nocrypt -in "${private_key_temp_file}" -outform PEM -out "${private_key}"
}

function generate_ca() {
  local -r truststore="${output_directory}/truststore.pkcs12"
  generate_private_key "${ca_key_file}"
  openssl req -new -key "${ca_key_file}" -nodes -sha256 -x509 -days 36525 -out "${ca_cert}" -subj "/CN=TDSA CA" -config "${config_file}" -extensions ca_cert
  rm -f "${truststore}"
  keytool -import -noprompt -file "${ca_cert}" -alias ca -trustcacerts -storepass changeit -storetype pkcs12 -keystore "${truststore}"
}

function generate_client() {
  local -r client_key="${output_directory}/client.key"
  local -r client_cert="${output_directory}/client.crt"
  generate_private_key "${client_key}"
  openssl req -new -key "${client_key}" -out "${cert_req_file}" -subj "/CN=TDSA Client" -config "${config_file}" -extensions client_cert
  openssl x509 -req -days 36525 -in "${cert_req_file}" -CA "${ca_cert}" -CAkey "${ca_key_file}" -CAcreateserial -CAserial "${serial_file}" -out "${client_cert}" -extfile "${config_file}" -extensions client_cert 
}

function generate_server() {
  local -r hostname="$1"
  local -r keystore="${output_directory}/keystore.${hostname}.pkcs12"
  generate_private_key "${private_key_file}"
  openssl req -new -key "${private_key_file}" -out "${cert_req_file}" -subj "/CN=${hostname}" -config "${config_file}" -extensions server_cert
  openssl x509 -req -days 36525 -in "${cert_req_file}" -CA "${ca_cert}" -CAkey "${ca_key_file}" -CAcreateserial -CAserial "${serial_file}" -out "${cert_file}" -extfile "${config_file}" -extensions server_cert 
  openssl pkcs12 -inkey "${private_key_file}" -in "${cert_file}" -CAfile "${ca_cert}" -caname ca -chain -export -out "${keystore}" -password pass:changeit -name server
}

cat <<EOL >> "${config_file}"
[ req ]
distinguished_name = req_distinguished_name

[ req_distinguished_name ]

[ ca_cert ]
basicConstraints = critical, CA:TRUE
keyUsage = critical, digitalSignature, cRLSign, keyCertSign

[ server_cert ]
basicConstraints = critical, CA:FALSE
nsCertType = server
keyUsage = critical, digitalSignature, keyEncipherment, keyAgreement
extendedKeyUsage = serverAuth

[ client_cert ]
basicConstraints = critical, CA:FALSE
nsCertType = client
keyUsage = critical, nonRepudiation, digitalSignature, keyEncipherment, keyAgreement
extendedKeyUsage = clientAuth
EOL

generate_ca
generate_client
for hostname in "$@"; do
  generate_server "${hostname}"
done