sett - Secure Encryption and Transfer Tool

• 1: What is sett?
• 2: Quick start guide
• 3: Download sett
• 4: OpenPGP key management
• 5: Encrypting, transferring, and decrypting data
• 6: Generating SSH keys
• 7: Authenticated Mode
• 8: Settings
• 9: Data package specification
• 10: Docker image
• 11: Frequently asked questions and bug reports
• 12: Source code
• 13: Benchmarks

1 - What is sett?

sett stands for “Secure Encryption and Transfer Tool” and is an application that facilitates and automates data packaging, encryption, and transfer. It is written in Rust, a fast and memory-safe programming language. sett is available both as a desktop and a command line application.

sett is developed as part of the BioMedIT project. It is licensed under the GPLv3 (GNU General Public License) and the source code is available from its public GitLab repository.

2 - Quick start guide

For the complete guide on how to use sett, please refer to Encrypting, transferring, and decrypting data and OpenPGP key management.

GUI (Graphical User Interface)

Initial setup

1. Download sett-gui from the download page. If you downloaded an installer, install sett-gui by double-clicking on the installer file.
2. Run sett-gui by double-clicking on the executable file or by launching the installed app.

Key management

1. If you do not already have a private/public OpenPGP key pair, go to the Keys tab and create one clicking on Add > Generate new key pair. See also the instructions given in the Generate a new public/private OpenPGP key pair section.

   You should then see your new key listed in the Keys tab, along with “Private” label that indicates that the private material for this key is present in the local keystore.

   BioMedIT

   Your OpenPGP key must be registered with the BioMedIT portal before it can be used to sign or decrypt data packages. Please see the Register your public OpenPGP key in the BioMedIT portal section for details.

2. If not already done, download the public OpenPGP key of the recipient(s) to whom you intend to send data (or from whom you will receive data). Go to the Keys tab and click on Add > Import from keyserver. See also the instructions given in the download public OpenPGP keys from the keyserver section.

3. Just after downloading the recipient’s OpenPGP key, verify it to make sure that it is genuine. This can be done by either:

   • If you are a BioMedIT user: verify that the recipient’s key is labelled with a green Approved label. You can also expand the details of the key by clicking on the key in the list or on the small down arrow button to the right and verify that the Approval status is set to “Key is approved on Portal”, and the Revocation status is set to “Valid”.
   • Alternatively, contact the key owner and verify the key fingerprint with them.

Encrypting and sending data

1. Go to the Encrypt and Transfer Data tab.

2. Add one or more files and directories to encrypt by clicking the Add files or Add directories buttons.

3. Select sender: select your own OpenPGP key. This is the key that will be used to sign the encrypted data.

4. Select recipients: add one or more recipients by selecting them in the drop-down. These are the keys that will be used to encrypt the data, i.e. only these recipients will be able to decrypt the data.

   BioMedIT

   The selected recipients must be approved Data Managers of the project for which data is being encrypted.

5. Data Transfer ID: specifying a valid Data Transfer Request ID is mandatory when a data package is transferred into the BioMedIT network. For other destinations, the Data Transfer ID field can be left empty (or set to any arbitrary value), and the Verify package checkbox must be disabled (in the Settings tab).

   BioMedIT

   The Verify package checkbox should always be enabled, since a valid Data Transfer ID is required by the BioMedIT network.

6. Select destination: select local and choose a destination directory to encrypt to your local file system. Select s3 or sftp to encrypt and transfer directly to an S3 object store or an SFTP server, respectively.

7. Click Encrypt data (local) or Encrypt and transfer data (s3 or sftp) to run the encryption workflow on your data.

Sending existing data packages

1. Go to the Encrypt and Transfer Data tab.

2. Select a file to transfer using the add sett Package button.

3. Select the Destination to be used (sftp, s3).

4. Enter the required destination parameters.

   BioMedIT

   For transfers into the BioMedIT network, the destination parameters are provided by:

   • Your local BioMedIT node, for transfers over sftp;
   • By BioMedIT Portal, for transfers over s3. Please see the dedicated section of the Portal user guide for more information.

5. Click Transfer data to start transferring your data package.

Decrypting data

1. Go to the Decrypt tab.
2. Select a data package to decrypt using the Select Package button.
3. Specify your desired destination directory.
4. Click on Decrypt package.

CLI (Command Line Interface)

The main commands to manage keys, encrypt, transfer and decrypt data with sett command line interface are given here.

sett --help
sett encrypt local -h
sett encrypt local --help

OpenPGP key management

# Generate a new key pair
sett keys generate

# Import sender/recipient(s) public keys:
sett keys import from-keyserver alice@example.com

Encrypt and transfer data

# Data encryption only, saving to local disk
sett encrypt local --signer alice@email.com --recipient bob@example.com --output . FILES_OR_DIRECTORIES_TO_ENCRYPT

# Data encryption and transfer to object store
sett encrypt s3 --signer SIGNER_KEY --recipient RECIPIENT_KEY --recipient-path RECIPIENT_PATH
--endpoint ENDPOINT --bucket BUCKET --access-key ACCESS_KEY --secret-key SECRET_KEY \
FILES_OR_DIRECTORIES_TO_ENCRYPT

# Data encryption and transfer to object store, using portal authentication
sett encrypt s3-portal --signer SIGNER_KEY --recipient RECIPIENT_KEY --dtr DATA_TRANSFER_ID  \
FILES_OR_DIRECTORIES_TO_ENCRYPT

Transfer a sett package

# Data transfer of an existing sett package

# to S3 object store
sett transfer s3 --endpoint ENDPOINT \
--bucket BUCKET --access-key ACCESS_KEY --secret-key SECRET_KEY \
SETT_PACKAGE_TO_TRANSFER

# to S3 object store, using portal authentication.
# metadata.json inside to package needs to contain a valid data transfer ID
sett transfer s3-portal SETT_PACKAGE_TO_TRANSFER

# to SFTP server
sett transfer sftp --host HOST --username USERNAME --base-path DESTINATION_DIRECTORY --key-path SSH_KEY_LOCATION --key-pwd SSH_KEY_PASSWORD SETT_PACKAGE_TO_TRANSFER

Decrypt a sett package

# decrypt a local sett package
sett decrypt local SETT_PACKAGE_TO_DECRYPT

# fetch and decrypt package from S3 object store,
# using portal authentication
sett decrypt s3-portal --dtr DTR SETT_PACKAGE_TO_DECRYPT

# fetch and decrypt package from S3 object store
sett decrypt s3 \
--bucket BUCKET --access-key ACCESS_KEY --secret-key SECRET_KEY \
SETT_PACKAGE_TO_DECRYPT

3 - Download sett

CLI

x86_64

Desktop

Standalone Installer

CLI

aarch64 x86_64

Desktop

.AppImage x86_64 .deb

CLI

aarch64 x86_64

Desktop

.dmg

You can download the latest version of sett for your operating system by clicking on the corresponding link above.

Downloading a specific version of sett

For a complete list of available versions, please visit the releases page.

Updating sett

As only the latest version of sett is officially supported and guaranteed to work, it is strongly recommended to always keep your local installation of sett up-to-date.

If you installed sett using the installer available for your operating system, sett will automatically check for updates and prompt you to install the newest version as soon as it's available.

If you are running a portable executable, make sure you regularly check this page and download the latest version of the executable.

4 - OpenPGP key management

To encrypt and decrypt data, sett uses public key cryptography. If you are not familiar with public key cryptography concepts such as public and private keys, revocation signatures, or keyservers you are advised to read this introductory section.

All key management operations can be performed using both the graphical user interface (GUI) and the command line interface (CLI) of sett.

sett --help
sett encrypt local -h
sett encrypt local --help

Generate a new public/private key pair

A prerequisite to encrypt, decrypt, and transfer files with sett is to have a public/private PGP key pair.

Generate a new key pair (GUI)

To generate a new public/private key pair:

1. Go to the Keys tab and click on + Add. From the drop-down menu, choose Generate new key pair. A dialog box will appear.

2. Fill-in the required fields:

   • Name: key owner first and last name.
   • Comment: an optional comment to help identify the owner of the key (e.g., owner’s institution).
   • Email: enter the email to be associated with the key pair.
   • Password: the password must be at least 10 characters long, and it is highly recommended that it contain a mix of letters, numbers and special characters.
   • Expiration date: a date in the future to ensure that the key is not used indefinitely (default: 3 years). It is possible to extend the expiration date later.

   Important

   It is not possible to retrieve or reset a password from a generated key. Make sure to remember it and store it in a safe place (e.g. a password manager).

3. Click Next to see the summary.

4. Click “Generate key” to create the new key pair (this can take a few seconds).

5. A new pop-up window will appear to confirm that the key was generated successfully and to display the revocation signature associated with the new key.

   Copy the revocation signature to a safe location, ideally a password manager, and click Next.

6. The next screen of the pop-up window allows to upload the public part of the key to the keyserver. Make sure to check the checkbox to associate the key with your email address. To establish this association, the keyserver will send an email to verify that you are the legitimate owner of the email associated with the key. It also means that other people will now be able to search for your key on the keyserver using your email address, otherwise your key will only be searchable using its full fingerprint.

7. The new key should now be listed in the keys tab.

8. Now that your new OpenPGP key is created, make sure to register it in the BioMedIT portal.

Generate a new key pair (CLI)

To generate a new key pair using sett command line interface:

sett keys generate --name "Alice Smith" --email alice.smith@example.com

After creating the new key, sett will display a revocation signature for it. Make sure you keep it in a safe place, such as a password manager. Revocation signatures can optionally be automatically exported to a file by using the --rev-sig option:

sett keys generate --name "Alice Smith" --email alice.smith@example.com --rev-sig alice.rev

List keys

List keys (GUI)

Go to the Keys tab.

List keys (CLI)

Use the keys list subcommand.

sett keys list

Export/import private keys

In some situations (e.g., new computer setup, remote sett environment) you might need to copy or move your private key to a different machine. This can be done by exporting the private key to a file, transferring the file to the new machine, and importing it.

Export key (GUI)

Click on the ... icon next to the key you want to export and select “Export public and private key”.

Export key (Cli)

Use the keys export subcommand:

# The key identifier can be its fingerprint, key ID or the user's email address.
sett keys export -o private_key.pgp -p alice.smith@example.com

Note that keys can also be exported in ASCII format (instead of binary) by adding the -a, --armor option.

Import key (GUI)

Click on the + Add button and select “Import from file or clipboard”.

Import key (CLI)

Use the keys import subcommand.

sett keys import from-file -p private_key.pgp

Note the usage of the -p option to import both the private and public parts of the key. If -p is omitted, only the public part of the key is imported.

Verify that the key has been correctly imported with sett keys list.

Ensure that you store any backed-up secret keys in a secure location and in an encrypted form (typically in a password manager).

Upload public key to the keyserver

sett allows users to upload their public OpenPGP key to a keyserver.

Upload public key to the keyserver (GUI)

1. Click on the ... icon next to the key you want to upload and select “Upload public key to keyserver”.
2. A dialog box will appear to ask for confirmation.
3. Make sure to check the checkbox to associate the key with your email address. To establish this association, the keyserver will send an email to verify that you are the legitimate owner of the email associated with the key. It also means that other people will now be able to search for your key on the keyserver using your email address, otherwise your key will only be searchable using its full fingerprint.
4. Click Upload to keyserver.
5. If you have selected the “associate key with email” checkbox, you will shortly receive an email from keyserver@keys.openpgp.org that contains a confirmation link to prove that you have access to your email.
6. Open the email and click on the link to associate your key with your email address. Other users now can find your public key with your email address.

Upload public key to the keyserver (CLI)

Use the keys upload subcommand.

sett keys upload alice.smith@exaple.com

In order to also trigger email verification for your key, add the -v, --verify option:

sett keys upload --verify alice.smith@exaple.com

Register public OpenPGP key in the BioMedIT portal

If you are not a BioMedIT user, this section is not relevant for you and can be skipped.

BioMedIT OpenPGP key status

OpenPGP keys used to encrypt, sign, and decrypt data within the BioMedIT network require the approval of the BioMedIT key validation authority. The information of whether a key is trusted or not is stored as key status in the BioMedIT portal. This is the reason why all OpenPGP keys used within BioMedIT must be registered with the BioMedIT portal.

When an OpenPGP key is first registered in the BioMedIT portal, its status is initially set to PENDING (i.e. it is awaiting approval). A key must have the APPROVED status before it can be used to encrypt or sign data packages within the BioMedIT network.

The list of key statuses is as follows:

• PENDING: a key approval request was submitted, but the key has not been approved yet. This is a manual process and can take from a few hours or up couple of days.
• APPROVED: key is approved for usage within the BioMedIT network. Only approved keys can be used to encrypt, sign, and decrypt data packages within the BioMedIT network.
• APPROVAL-REVOKED: approval of the key has been revoked by the BioMedIT key validation authority.
• KEY-REVOKED: key has been revoked by its owner.
• REJECTED: key is not trusted by the BioMedIT key validation authority.
• DELETED: key has been removed from the keyserver by its owner.
• UNKNOWN KEY: key has not been registered on the BioMedIT portal. If it is your own key, please register it. If it is the key of someone else, please ask them to register their key.

To verify that a key is trusted, sett connects to the BioMedIT portal and retrieves the status of the key. For this reason, it is important that BioMedIT users register their OpenPGP key with the BioMedIT portal.

In cases where sett is used outside of the BioMedIT project, or the portal is not reachable, sett can still be used to encrypt, decrypt, and transfer data. In this case, you need to uncheck Verify package in the Settings tab.

The status of a public key in the BioMedIT portal can be easily checked in GUI by going to the Keys tab and clicking on one of the keys. Approved keys will also have a green “Approved” badge.

How to register your public OpenPGP key in the BioMedIT portal?

To register a new OpenPGP key with the BioMedIT portal, proceed as follows:

1. Make sure you have successfully uploaded your key to the keyserver and that you have completed the email verification procedure with the keyserver (i.e. your key must be verified with the keyserver).

2. To make sure that your key is present on the keyserver and is verified, go to the keyserver home page in your browser and search for the email associated with your key. You should get a message saying that a key was found for your email address and the corresponding key fingerprint should be displayed.

3. Copy the fingerprint (40-character string) of your key.

4. Log in to the BioMedIT portal.

5. Go to the Profile / OpenPGP Keys.

6. Click on the "+ OPENPGP KEY" button, a dialog box will open.

   Note: if the button is missing, it is probably because you already have an active key in the portal. Each user can only have 1 active key at a time - see the information box above.

7. Enter your full key fingerprint (must be exactly 40 characters long) in the dialog box, then press the green search icon to the right.

   This will retrieve the User ID and Email address associated with the fingerprint from the keyserver and display them in the dialog box.

8. Verify the user ID and mail address. If they are correct for your OpenPGP key, then click on Confirm.

9. A request to approve your key has now been sent to the BioMedIT key validation authority. Generally requests are processed quickly (in a matter of hours), but occasionally it might take slightly longer as this is a manual process.

   Please contact the BioMedIT support if your key has not been approved after a couple of days.

Download public keys from the keyserver

In order to encrypt data for a specific recipient (who will be able to decrypt it), you will need to have the public OpenPGP key of that recipient(s) available in your local keyring.

Download public keys (GUI)

To download a public PGP key from the keyserver:

1. In the Keys tab, click on + Add. From the drop-down menu, choose Import from keyserver. A dialog box will open, allowing to search for public keys stored on the keyserver.

2. In the search field, enter either the full email address or fingerprint of the public key you are looking for and click Import key.

   For instance, if searching for the key “Bob Test Key bob@example.com” with fingerprint “AEED7A96F339881F6FE8291464A1E0150613807D”, one can search for either “bob@example.com” or “AEED7A96F339881F6FE8291464A1E0150613807D”.

3. If the key you are looking for was found, you will see a success message with key details.

Download public keys (CLI)

The keys import from-keyserver subcommand can be used to download public OpenPGP keys from the keyserver.

sett keys import from-keyserver alice.smith@example.com

Remove your public keys from the keyserver

While it is not possible to remove an actual key from the keyserver, it is possible to remove all personally identifiable information from it (user ID and email). Such keys are called unverified.

To remove personal information associated with a key, go to the keyserver’s manage key page, enter the email associated to the key and click on Send link.

You will receive an email with further instructions on how to proceed to remove your key’s user ID and email from the keyserver.

Delete keys from your local machine

Users are strongly discouraged from deleting keys from their local keystore. Instead of deleting a key, users should revoke it. Revoking a key informs others that the key should not be used anymore, and is a safer way to handle unused or compromised keys.

Deleting private keys is possible from sett.

• GUI: Click on the ... icon next to the key you want to delete and select “Delete”.

• CLI: Use the keys delete-private-key subcommand.

  sett keys delete-private-key alice.smith@exaple.com
  

Deleting public keys is not possible with sett. However, it is possible to delete a public key by manually deleting the file containing the keys using your file explorer or a shell command.

In the local sett public keystore, keys are stored as files. They are named after the fingerprint of the key’s primary key. For instance, a key with fingerprint 3b17f529665fe012ef54f4a1714fdf98b6e828df would be stored under:

• Public key: <keystore-path>/3b/17f529665fe012ef54f4a1714fdf98b6e828df

The location of the keystore (<keystore-path> above) is operating-system dependent:

• Linux: ~/.local/share/pgp.cert.d
• Windows: %UserProfile%\AppData\Roaming\pgp.cert.d
• MacOS: ~/Library/Application Support/pgp.cert.d

You can also get the exact location of the key in sett GUI:

1. Click on the ... icon next to the key you want to delete and select “Delete”.
2. A dialog box with important information about key deletion, as well as the exact location of the key will be displayed.

Generate a revocation signature

A prerequisite for revoking an OpenPGP key is to have generated a revocation signature for it. If the OpenPGP key to revoke was generated with sett, you should already have a revocation signature ready to use. If you do not have a revocation signature yet, you can generate one with GUI:

1. Click on the ... icon next to the key you want to generate revocation signature and select “Create revocation signature”.

2. Fill-in the required fields:

   • Reason: the reason for revoking the key.
   • Message: a short explanation to the reason for revoking the key.
   • Password: the password for your private key.

3. Click Generate signature.

4. A new pop-up window will appear to confirm that the revocation signature was generated successfully. Copy the revocation signature to a safe location, ideally a password manager (anyone with access to a revocation signature can revoke the key for which it was generated).

Revoke your key

If a private OpenPGP key has been compromised, is no longer usable (e.g. password is lost), or should no longer be used for any other reason, it must be revoked.

A prerequisite for revoking a PGP key is to have generated a revocation signature for it. If the OpenPGP key to revoke was generated with sett, you should already have a revocation signature ready to use. If you do not have a revocation signature yet, please generate one by referring to the Generate a revocation signature section.

Revoke your key (GUI)

1. Click on the ... icon next to the key you want to revoke and select “Revoke”.

2. Either paste you revocation signature from the clipboard or load it from a file.

   Warning: proceed with caution, a revoked key cannot be “un-revoked”.

3. Click on Revoke key.

4. When expanding the revoked key from the List of available keys, you should now see the Revocation status as “Revoked”. From this point on, the key can no longer be used with sett.

5. If you have previously shared your key via a keyserver (e.g. keys.openpgp.org), you must also re-upload your revoked key to that keyserver.

   This will allow other users to update their local copy of your public key, informing them that it is no longer valid. To upload your revoked public key, please refer to the Upload your public PGP key to the keyserver .

   If your key was never present on any keyserver, this step should be skipped.

Revoke your key (CLI)

Use the keys revoke subcommand.

sett keys revoke alice.smith@example.com compromised "My dog ate it"

After a key has been revoked, it must be uploaded again to any keyserver(s) where it is present, so that the revocation can be shared with others. This can be done with sett as illustrated in the Upload your public PGP key to the keyserver.

Update expiration date for your key

It is possible to set an expiration date for an OpenPGP key. This is useful to ensure that the key is not used indefinitely. It is possible to extend the expiration date at any time.

Update expiration date (GUI)

sett GUI warns you when your key is about to expire. On the Keys tab, you will notice an Expiring badge starting from three months before the expiration date. On top of that, a warning will be displayed every time you use that key for signing or decryption.

To set a new expiration date for your key from the GUI:

1. Click on the ... icon next to the key you want to update the expiration date for.

2. Click on Update expiration date.

3. Choose a new expiry date.

4. Click on Set new expiry date.

5. Unlock your key by entering the password, if needed.

Update expiration date (CLI)

sett CLI warns you every time you use a key which is about to expire for signing or decryption. This happens starting from three months before the expiration date.

To set a new expiration date for your key from the CLI:

sett keys expire alice.smith@example.com --expiration 3y

For details about the supported time specifications for the --expiration option, run:

sett keys expire --help

Migrating keys from the GnuPG keyring to the sett keystore

Public OpenPGP keys located in your GnuPG keyring are not automatically detected by sett, and they must be migrated to sett’s public keystore. In contrast, private keys present in GnuPG can be used in sett without migration (however, it is still possible to migrate them).

Note that to migrate public keys (e.g., keys from other people), you can simply re-download them from the keyserver as shown in the Download public PGP keys from the keyserver section.

• GUI:

  1. In the Keys tab, click on + Add. From the drop-down menu, choose Import from GnuPG. A dialog box will open, containing a list of keys in your GnuPG keyring.
  2. Look for the correct key and click on Import. You should now see the key listed in the List of available keys. Note that if you are importing a private key, you will be asked to enter its password.

• CLI: use the keys import from-gpg subcommand.

  # The search term can be an email or fingerprint.
  sett keys import from-gpg alice.smith@example.com
  

Introduction to public-key cryptography and OpenPGP

Public-key cryptography is a method for secure communication between two or more users. In this system, each user has a pair of unique keys consisting of a private key and a public key. Public and private keys are linked in the sense that, data encrypted with a given public key can only be decrypted with the matching private key, and data signed with a given private key will only be recognized by the matching public key.

Because these keys are based on the OpenPGP protocol, they will here be referred to as OpenPGP keys.

Public and private OpenPGP keys:

• Public keys are used to encrypt data, as well as for verifying signatures made on files or emails. By design, public keys are intended to be shared with other people and therefore no particular effort is required to keep them secret. In fact, public keys are often uploaded to public servers, known as keyservers, where they are accessible to anyone. No password is required to use a public key.

  Typically, public keys are used by data senders to encrypt data for one or more recipient(s), and by data recipients to verify signatures of files or emails (to ensure the sender is genuine).

• Private keys, sometimes also referred to as a secret keys, are used to decrypt data, sign files and sign other people’s public keys. To increase security, private keys should always be password protected.

Typically, private keys are used by data recipients to decrypt data, and by data senders to sign the files they encrypt.

sett uses the open source implementation of public-key cryptography provided by Sequoia-PGP: a modular OpenPGP implementation in Rust.

It is possible - and often desirable - to both encrypt and sign a file. This ensures that the data can only be read by the intended recipient, and that the recipient can be confident the sender is legitimate. This is precisely what sett does:

• Encrypting files, so that only the intended recipient(s) can read them.
• Signing files, so that the recipient(s) can trust the sender is genuine.

Key fingerprints

Each pair of public/private OpenPGP keys is identified by a unique fingerprint. Fingerprints are 40 characters long hexadecimal strings (digits and upper case A-F letters) that look like this:

238565936FCFF3F200219990941A3EC20555F781

Since nothing is preventing two OpenPGP keys to have the same user name and email address, it is critical that users always verify the genuineness of new keys before (or just after) importing them into their local keyring (i.e. their local OpenPGP key database).

Ensuring a key is genuine can be done in two different ways:

• Ask the key owner to provide their key’s fingerprint via a trusted communication channel (e.g. over the phone), and then verify that the fingerprint of the newly imported key does indeed match the fingerprint communicated to you by its owner.
• Using sett-gui, verify that the key status of the key is APPROVED (can only be checked after you imported the key).

File encryption

In public key cryptography, the sender encrypts a file using one or more recipient(s) public key(s). Once a file is encrypted, no one can read the file without having access to a private key that matches the public key(s) used for encryption. This ensures that only the intended recipient(s) can decrypt the file, because they are the only one to have access to the matching private key.

File signing

The objective of file signing is to guarantee to the recipient of a file (or email) that the sender is genuine, and not someone else trying to impersonate the sender.

To achieve this, the sender signs the file with their private key (password required), and shares their public key with the recipient (typically via a keyserver). The recipient can then validate the authenticity of the signature using the public key of the sender. Since public keys are non-sensitive, they can be distributed publicly. In fact they are intended for this purpose, hence their name.

Revocation signatures

In the unfortunate event that a user either i) forgets their private key’s password or ii) have their private key and password stolen/compromised, they will need a way to let other people know that their public key should no longer be trusted and used.

This is because:

• If the password was forgotten: the key owner won’t be able to decrypt data anymore.
• If the private key was compromised: someone else might be able to decrypt data encrypted with the public key, and to illegitimately sign files!

This situation is what revocation signatures are for: by applying a revocation signature to a public key, and then sharing the revoked key with others (e.g. via a keyserver), the key owner signals that their key is now “revoked” and should no longer be trusted nor used. After a key has been revoked, it can no longer be used to encrypt/decrypt data with sett.

Revocation signatures can be generated at anytime from the private key, but the best practice is to generate them directly after a new key pair is created. This ensures that the revocation signature will be available even if the private key or its password is lost.

Since anyone with access to a revocation signature will be able to revoke the associated key, revocation signatures must be stored securely - e.g. in a password manager - and should never be shared with anyone.

Exchanging public keys via a keyserver

Encrypting files for a specific recipient requires to have the recipient’s public key in one’s local keyring (a keyring is a local database containing OpenPGP keys). Similarly, verifying a signature on a file or a public key requires to have the signee’s public key available in one’s local keyring.

Public keys are not sensitive data, and therefore can be sent unencrypted via email. However, when having frequent key exchanges between multiple actors, sending public OpenPGP keys around by email quickly becomes cumbersome. A solution to this problem is using a so called keyserver to share public keys. Keyservers are public or private servers whose sole purpose is to store public OpenPGP keys and allow users to search for them.

5 - Encrypting, transferring, and decrypting data

The instructions below describe the process of encrypting, transferring, and decrypting data. sett provides both a graphical user interface (GUI) and a command line interface (CLI) for these operations.

sett --help
sett encrypt local -h
sett encrypt local --help

Encrypting and sending data

sett allows the encryption of any combination of individual files and directories.

The files are first compressed into a single data.tar.gz archive, which is then encrypted with the public key of one or more recipient(s), and signed with the sender’s key. The encrypted data (data.tar.gz.gpg) is then bundled with a metadata file - a plain text file that contains information about who is sending the file and to whom it should be delivered - into a single .zip file. The specifications of the output .zip files produced by sett are described in data package specification.

sett supports multi-recipient data encryption. This allows the encrypted file to be decrypted by multiple recipients.

sett also ensures the integrity of the transferred files by computing checksums on each file that is packaged, and adding this information to the encrypted data. The integrity of each file is verified automatically upon decryption of the file by sett, providing the guarantee that all files were transferred flawlessly.

Output file naming scheme

By default, encrypted output files produced by sett are named after the pattern:

<project code>_<YYYYMMDD>T<HHMMSS>.zip

where:

• <project code> is the abbreviation/code associated with the project. If Verify package is disabled, no project code is added as a prefix to the output file name.
• <YYYYMMDD> is the current date (Year, Month, Day).
• <HHMMSS> is the current time (Hours, Minutes, Seconds).

Example: demo_20220211T143311.zip, here demo is the project code.

Using the sett command line interface when encrypting to the local file system, it is possible to completely override the above output file naming scheme by passing the -o, --output option. Overriding the naming scheme is not possible when using sett-gui or when encrypting to a remote S3 or SFTP destination.

Encrypting and sending data (GUI)

Data can be encrypted and either saved to the local file system, or sent directly to a remote server that supports one of the following protocols:

• S3 object storage
• SFTP

1. Go to the Encrypt and Transfer Data tab.

2. Select files and/or directories to encrypt: using the Files and Folders buttons, select at least one file or directory to encrypt. Files and directories can also be dragged and dropped into sett.

   Note

   If you already have an encrypted data package in your local file system, you can select it with sett Package (or drag and drop it into sett), then choose a remote destination (continue reading at Select destination).

3. Select data sender: in the drop-down list found under Select sender, select your own OpenPGP key (you are the data sender). For most users, there should be only one key in the Sender drop-down menu: their own key.

   Note

   The Sender key is used to sign the encrypted data, so that the recipient(s) can be confident that the data have been sent by a trusted source.

4. Select data recipients: add one or more recipients by selecting them from the drop-down list found under Select recipients. Recipients are the people for whom data should be encrypted: their public OpenPGP key will be used to encrypt the data, and only they will be able to decrypt it.

   BioMedIT

   Only recipients assigned to the role of Data Manager of the project for which data is being encrypted are permitted as data recipients.

5. Data Transfer ID: Data Transfer Request ID associated with the data package that is being encrypted. Specifying a valid DTR ID is mandatory to transfer data into the BioMedIT network.

   For data not intended to be transferred into the BioMedIT network, the DTR ID field can be left empty (or set to any arbitrary value). In this case, Verify package must be disabled (in the Settings tab).

   BioMedIT

   DTR ID field is mandatory. Only files encrypted with a valid and authorized DTR ID value can be transferred into the secure BioMedIT network. For this reason, BioMedIT users should always leave the Verify package checkbox enabled.

6. This section is only visible if “Enable extra metadata” is selected in the Settings tab.

   Here you can add extra metadata to the data package in the form of key-value. This metadata will be stored in the metadata file of the encrypted data package. After inserting the key and value, click the + button (or hit “Enter”) to add the new entry. You can add multiple key-value pairs.

7. Select destination: destination where to encrypt (and send) the data.

   • Local: The local file system. When using this destination, it’s possible to specify:

     • Output location: directory where the encrypted file should be saved. By default, output files are saved to the user’s home directory. This default behavior can be changed by changing Default output directory in the Settings tab.

   • S3: A remote S3 compatible object store.

   • SFTP: A remote SFTP server.

   When encrypting to a remote destination, a number of options must be specified. Please refer to remote destination options for details.

8. You are now ready to create an encrypted data package: click Encrypt data or Encrypt and transfer data if you are encrypting to a remote S3 or SFTP destination. A pop-up will appear, asking for the password associated with the sender’s key. After the password is entered, data compression and encryption will start. Progress and error messages are displayed in the Tasks tab.

   When the encryption completed successfully, a notification will pop-up with a message that reads: “Encryption job finished”.

At this point, all input files are compressed, encrypted, and bundled into a single .zip file. If the destination was S3 or SFTP, data has also been transferred to the remote destination.

Encrypting and sending data (CLI)

To create an encrypted data package and save it to the local file system, use the encrypt subcommand. If you already have an encrypted data package in your local file system and want to transfer it to a remote destination, use the transfer subcommand. Both subcommands share the same options for specifying the remote destination.

# General syntax:
sett encrypt local --signer SENDER --recipient RECIPIENT --dtr DATA_TRANSFER_ID --output OUTPUT_FILENAME_OR_DIRECTORY FILES_OR_DIRECTORIES_TO_ENCRYPT

# Example (long command line options):
sett encrypt local --signer alice@example.com --recipient bob@example.com --dtr 42 --output . ./file_to_encrypt.txt ./directory_to_encrypt

# Example (short command line options):
sett encrypt local -s alice@example.com -r bob@example.com -dtr 42 -o . ./file_to_encrypt.txt ./directory_to_encrypt

Data can be encrypted for more than one recipient by repeating the flag -r/--recipient, e.g. -r RECIPIENT1 -r RECIPIENT2 option:

# In this example, Alice encrypts data for both Bob and Chuck.
sett encrypt local -s alice@example.com -r bob@example.com -r chuck@example.com -o . FILES_OR_DIRECTORIES_TO_ENCRYPT

--output is an optional argument for specifying the location and/or name for the encrypted output file. The --output argument can be one of the following:

• Not provided: the encrypted data is written to stdout. This can e.g. be useful to pipe the data into another application.
• A directory: the encrypted data package file is written to the specified directory, and is given a name that follows the default naming convention in sett.
• A file name: the encrypted data is written to a new file with the specified name, and in the specified directory, if the file name contains one. This overrides the default output file naming schema.

local subcommand can be replaced with s3 or sftp to encrypt and transfer data directly to a remote destination in a single command. When using s3 or sftp, the data is encrypted and streamed directly to the destination, without creating any files in the local file system. The streaming approach is faster than creating a data package locally and transferring it separately. It also saves space on the local machine when transferring large datasets.

For more information about remote destination options, please refer to the remote destination options section.

sett encrypt s3 -s alice@example.com -r bob@example.com --endpoint https://minio.my-node.ch --bucket my-project \
--access-key 23VO8RB2SIB2SF8EUL9V --secret-key wvrt7YoTTERGftf0zWnppWYSdcGplNtxuLHMn7op --session-token eyJhbGciOiJ\
IUzUxMiIsInR5cCI6IkpXVCJ9.eyJhY2Nlc3NLZXkiOiI5Vk84UkIyvimUMlKIFUVVTDc3WSIsImF0X7hhc2giOiIyRnVlZ3JmSjhTUWFXYkw2V0puek\
F3IiwiYXVkLjpbIm1pbmlvIl0sImF1dGhfdGltTRI6MTcyMTezODIxMywiZXhwIjoxNzIx0DMxODEzLCJpYXQiOjE3MjE4MzfiLKLsImlzwqI6Imh\
0dHBzOi8vcD3ydGFsLXN0YWasfmcuZGNjLnNpYi5zd2lzcy9hdXRoL38hdXRoIiwibmFtZSI6ImJpd2ciLCJqt5xpY5kiOiJjb25zb2xlQWRfgW4iLC\
JzdWIiOiIxOSJ9.PcvXcAli5Bz8ete1T265TPB1cbfgX7k8NDXU5gXy1nflxq203cG5qwAF9Oxyn1mKmwa87jsHj8HU2VUY9p5S1Q \
FILES_OR_DIRECTORIES_TO_ENCRYPT

Adding the --check option will run the encrypt command in the test mode, i.e., checks are made but no data is encrypted or sent.

Data compression algorithm can be changed using the --compression-algorithm flag. The available options are:

• zstandard (default), optimal compression and speed.
• gzip, available for compatibility with older versions of sett.
• stored, no compression.

The data compression level used by sett can be manually adjusted using the --compression-level option. Possible values depend on the selected compression algorithm:

• zstandard (default: 3)
  • 1 (lowest compression, fastest)
  • 21 (highest compression, slowest).
• gzip (default: 6)
  • 1 (lowest compression, fastest)
  • 9 (highest compression, slowest)

Before encrypting data using the local subcommand and --output flag, sett verifies that there is enough free disk space available on the local machine to save the encrypted output file. If this is not the case an error message is displayed and the operation is aborted. Since the compression ratio of the input data cannot be known in advance, sett uses the conservative estimate that the minimum disk space required is equal to the total size of all input files to be encrypted.

To automate the encryption process, you can use environment variables to store the OpenPGP password.

sett performs DTR verification if the --verify option is passed. For non-BioMedIT-related transfers, the --verify option should not be passed.

Decrypting files

Decrypt and decompress encrypted data packages.

Please note that the decryption process includes the verification of the sender’s signature, which ensures the authenticity of the data. Verification of the sender’s signature is only possible if the sender’s public key is available in your local keyring. For security reasons, the sender’s key will not be automatically downloaded from a key server; it must be downloaded/imported manually (for more information see download public keys).

To decrypt data, you must therefore have in your local keyring:

• The private key for which the data was encrypted. In principle, this is your own private key. This key will be used to decrypt the data.
• The data sender’s public key. This key is used for signature verification purposes.

Only files that follow the sett packaging specification can be decrypted with sett.

Decrypting data (GUI)

To decrypt and decompress a file:

1. Go to the Decrypt tab.

2. Select the source from which you wish to decrypt data - either local or s3.

3. When decrypting from local, select a data package to decrypt with Select Package, or drag and drop the file into sett. When decrypting from s3, a number of options must be specified - please refer to remote destination options for details. After doing so, click on Load package.

4. Select destination directory: select a location where to decrypt/decompress the file.

   By default, output files are saved to the user’s home directory. This default behavior can be changed by changing Default output directory in the Settings tab.

5. Click Decrypt package to start the decryption and decompression process. A pop-up dialog box will appear to ask for the password associated with the OpenPGP key used to encrypt the files.

Decrypting data (CLI)

Use the decrypt subcommand to decrypt and decompress data:

# General syntax:
sett decrypt local --output OUTPUT_DIRECTORY ENCRYPTED_FILES.zip

# Example:
sett decrypt local --output /home/alice/data/unpack_dir /home/alice/data/test_data.zip

local subcommand can be replaced with s3 to decrypt data package from an S3 object store. When using s3, the data is streamed and decrypted directly from the S3 object store, without creating any temporary files in the local file system.

To decrypt data without decompressing it, add the -d, --decrypt-only option.

If the -o, --output option is omitted, the data is decrypted in the current working directory.

To automate the decryption process, you can use environment variables to store the OpenPGP password.

Remote destination options

Both GUI and CLI require specific parameters when transferring data to a remote destination. The parameters are different depending on the destination type.

• S3: A remote S3 compatible object store. When using this destination a number of options must be specified:

  • URL: The URL of the S3 object store.
  • Bucket: The name of the bucket where the encrypted data should be stored.
  • Access key: The access key to use to authenticate with the S3 object store. It is also possible to use a username instead.
  • Secret key: The secret key to use to authenticate with the S3 object store. It is also possible to use a password instead.
  • Session token: The session token to use to authenticate with the S3 object store. It is optional and only required when authenticating using temporary credentials (STS), which is the case of most users interacting with the BioMedIT infrastructure.

  BioMedIT

  This information must be retrieved from BioMedIT Portal by a data provider data engineer. Please refer to the dedicated section of the Portal user guide for instructions on how to do so. Portal allows copying the credentials to the clipboard, so that they can be pasted into sett by using the Paste from clipboard button.

• SFTP: A remote SFTP server. When using this destination, a number of options must be specified:

  • Host: URL address of the server where the files should be sent.

  • User name: the user name with which to connect to the SFTP server.

  • Destination directory: absolute path of directory where files should be saved on the server.

  • SSH key location: path of the private SSH key used for authentication to the SFTP server. This is only required if the SSH key is in a non-standard location. If missing, sett will use the SSH agent to provide the key.

    Do not confuse SSH keys - which are used to authenticate yourself when connecting to an SFTP server during file transfer - with OpenPGP keys - which are used to encrypt and sign data.

  • SSH key password: password associated with the private SSH key given under SSH key location. If your SSH key password contains characters that are not ASCII characters, and that this results in an error, please see the SSH private key with non-ASCII characters section of this guide.

  BioMedIT

  For BioMedIT users, the SFTP connection parameters User name, Host URL, and Destination directory will be provided by your local BioMedIT node.

6 - Generating SSH keys

SSH (Secure SHell) keys are pairs of small text files that are used to securely identify users and give them access to remote servers, e.g. when transferring data via SFTP.

SSH keys use public-key encryption and always come in pairs: a public and a private (or secret) key.

• Public key: the public key of an SSH key pair is meant to be placed on the remote machine to which a user wants to connect. Public keys are non-sensitive, and are typically shared by users with system administrators, who will place them on the machine to which the user is granted access.
• Private key: the private key of an SSH key pair is what uniquely identifies a user as the legitimate owner of a public key. In other words, having the private key that matches a given public key will give access to any machine on which a copy of public key is stored. Private SSH keys are sensitive information: they must be kept private at all times and should never be shared with anyone - not even your system administrator. Private keys can (and should) be protected by a password, so that even if someone else has access to them, they remain unusable.

Generating a new pair of SSH keys must be done only once, and, in the context of sett, is only needed if you intend to transfer data. If you are a user who only decrypts data, you do not need an SSH key.

Also, do not confuse SSH keys - used to identify yourself on a remote server - with OpenPGP keys - used to encrypt and sign data.

To generate a new SSH key pair, type the command below in your terminal (Linux and Mac) or PowerShell (Windows users - to start it, search for “powershell” in the Start menu). Note that you must replace "alice@example.org" with your own email. This will generate an SSH key pair using the ed25519 algorithm, currently the most secure public-key algorithm:

ssh-keygen -a 100 -t ed25519 -C "alice@example.org"

Windows users who do not have the ssh-keygen command installed, please see section below.

When executing the ssh-keygen command above, you will be prompted for the following information:

1. The name and location where to save the newly created keys. Here you should simply accept the default values by not entering anything and pressing “Enter” on your keyboard. Default locations are ~/.ssh/id_ed25519 on Linux and MacOS, and C:\Users\%username%\.ssh\id_ed25519 on Windows.
2. A password to protect your private SSH key against illegitimate use. Please use a password that is long enough (>=12 characters) and composed only of ASCII characters.

When the command completes, two new files are produced: id_ed25519.pub (the public key) and id_ed25519 (the private key).

On Linux and MacOS systems, after the public key is generated, its permissions must be changed with the following command (this step is not needed for Windows users):

chmod 600 ~/.ssh/id_ed25519.pub

Windows users: enabling the ssh-keygen command

Not all versions of windows come with the ssh-keygen command pre-installed. If this command is unavailable on your machine, please install it as follows:

1. Open the windows settings (shortcut: windows key + i).
2. Search for, and select, “Add an optional feature”.
3. Click on “Add a feature”.
4. Search for, and select, “Open SSH Client”.
5. Click “Install”.
6. Restart your computer.

SSH private key with non-ASCII characters password

Even though it is possible to create an SSH key pair using a password containing non-ASCII characters, it seems like those characters are encoded differently between different operating systems.

As an SSH key might be moved to a machine with another operating system, or encoding might change with a new version, it is impossible to guess the correct encoding in any case. For this reason, we recommend not to use non-ASCII characters to protect SSH private keys.

7 - Authenticated Mode

Why authenticated mode?

In order to simplify the data transfer process, sett offers an authenticated mode. Once the user is authenticated, the following functionalities are offered:

• list of available DTR (data transfer request) for the logged-in user
• package verification is always enforced
• data transfer from and to a BioMedIT S3-type object storage without entering credentials

Device authorization flow explained

The authenticated mode in sett implements a so-called device authorization flow, an authorization mechanism usually used by electronic devices with internet connection that lack any input capacities. In both the CLI and GUI of sett, the user is asked to visit a specific website (an OIDC provider) to identify himself. While the user is logging in, the device (in our case: sett) is periodically polling the website, until the user has successfully logged in. sett then receives a JWT token (JSON Web Token) from the OIDC provider, which allows it to fetch additional data from Portal, e.g. the list of available data transfers for the logged-in user, as well as the credentials used for transferring data to and from the S3 object storage.

The diagram below illustrates how this device authorization flow works in sett when requesting the list of available DTRs or when up/downloading data.

Device authorization flow as implemented by sett

Glossary

• CLI: Command Line Interface. This is how sett is used in a terminal.
• GUI: Graphical User Interface. E.g. the standalone sett application that runs in its own window.
• S3 object storage: the database that (temporarily) stores all sett packages. Every node runs its own storage. S3 stands for Simple Storage Service and was originally developed by Amazon.
• SFTP: Secure File Transfer Protocol. The legacy way to transfer data.
• OIDC provider: an system that redirects a user to Switch edu-ID to authenticate himself. It can issue tokens which allow sett to fetch information on behalf of the user. It plays a crucial role in implementing the device authorization flow. OIDC stands for Open ID Connect.
• JWT token: a piece of structured data which contains information about an authenticated user. It is signed, so a third party (in our case: Portal) can verify the integrity of its content. JWT stands for JSON Web Token.

8 - Settings

GUI settings

The sett desktop app allows a number of options to be customized via its Settings ⚙️ page. For instance, you may change the default output directory, or enable/disable package verification before a transfer.

Each setting has a predefined default value, which is used when first running the tool or if loading the current settings fails for any reasons.

Changes made to Settings become effective immediately. Changes can be reset back to their factory default by clicking on the Reset settings button.

Settings are divided into three sections: “basic”, “advanced”, and “non-editable”.

Basic

Verify package

When enabled (the default value), the following checks are made before encrypting or transferring data:

• DTR ID is valid and the transfer is authorized.
• Sender and Recipients public OpenPGP keys are approved by the BioMedIT key validation authority.
• Recipients are approved Data Managers of the BioMedIT project for which data is being encrypted.
• The name of the data package matches the pattern <project_code>_<date_format>.zip. This ensures no sensitive information is mistakenly included in the file name.

Note that that some of the above checks require communication with the BioMedIT portal. When using sett outside of a BioMedIT project, this setting should therefore be disabled.

Default output directory

Default destination directory for operations such as encryption to the local filesystem or decryption. User’s home directory is used by default.

Advanced

Enable extra metadata

When enabled, additional metadata can be added to the package in the data encryption form.

OIDC issuer URL

URL of the OpenID Connect issuer used for authentication (BioMedIT specific).

Portal URL

URL of a BioMedIT portal instance. Portal is used for key approval verification, DTR (Data Transfer Request) validation, and retrieval of data associated with a given DTR (when sett is being used in authenticated mode). The default value of this setting is: https://portal.dcc.sib.swiss.

Public key store

Directory where public OpenPGP keys are stored.

Private key store

Directory where private OpenPGP keys are stored.

Non-editable

This section displays values of setting that cannot be modified by the user. These settings are displayed here for convenience. They can be copied to the clipboard via a dedicated “copy to clipboard” button.

Log directory

Directory where log files are stored. Location of the log directory depends on the operating system:

• Linux: ${XDG_DATA_HOME}/ch.biomedit.sett/log or $HOME/.local/share/ch.biomedit.sett/log
• macOS: $HOME/Library/Application Support/ch.biomedit.sett/log
• Windows: {FOLDERID_RoamingAppData}\ch.biomedit.sett\log

Keyserver URL

URL of the OpenPGP key server used to retrieve and publish public keys. The default value is https://keys.openpgp.org.

CLI settings

The sett-cli is stateless by design, meaning that there is no persistent configuration file where settings can be modified and stored.
Instead, settings can be set via the following shell environment variables. All settings are optional and have a default value.

SETT_OPENPGP_KEY_PWD

Password to unlock the secret OpenPGP key used to decrypt or sign data. When this environmental variable is set, sett uses its content instead of interactively asking the user to enter a password.

SETT_OPENPGP_KEY_PWD_FILE

Full path and name of a file containing the password to unlock the secret OpenPGP key used to decrypt or sign data. When this environmental variable is set, sett uses its content instead of interactively asking the user to enter a password. The file containing the password should not be encrypted.

SETT_PORTAL_URL

URL of the BioMedIT Portal instance to be used. For details see the description of the GUI Portal URL setting. This setting defaults to https://portal.dcc.sib.swiss.

SETT_OIDC_CLIENT_ID

Client ID with which sett should identify with the OpenID Connect issuer (see SETT_OIDC_ISSUER_URL). Only relevant when using sett in authenticated mode. This setting defaults to sett.

SETT_OIDC_ISSUER_URL

URL of the OpenID Connect issuer used when authenticating with the BioMedIT Portal. Only relevant when using sett in authenticated mode. This setting defaults to https://login.biomedit.ch/realms/biomedit.

SETT_KEYSTORE

Directory where private OpenPGP keys are stored. Location is platform dependent:

• Linux: ${XDG_DATA_HOME}/sequoia/keystore or $HOME/.local/share/sequoia/keystore
• MacOS: $HOME/Library/Application Support/org.Sequoia-PGP.sequoia/keystore
• Windows: {FOLDERID_RoamingAppData}\org.Sequoia-PGP.sequoia\keystore

PGP_CERT_D

Directory where public OpenPGP keys are stored. Location is platform dependent:

• Linux: ${XDG_DATA_HOME}/pgp.cert.d or $HOME/.local/share/pgp.cert.d
• MacOS: $HOME/Library/Application Support/pgp.cert.d
• Windows: {FOLDERID_RoamingAppData}\pgp.cert.d

SETT_METADATA_EXTRA

Additional metadata to be included in the data package. Extra fields must be provided in the form of key=value pairs separated by a comma and without spaces. For example: SETT_METADATA_EXTRA="foo='value 1',bar=value_2"

9 - Data package specification

sett compresses, encrypts, and packages files in a single .zip file whose specification is described below. Only files adhering to these specifications can be transferred or decrypted by sett, and failure to comply with the specification will generate an error.

File structure

sett .zip files have the following structure:

YYYYMMDDThhmmss.zip
├── metadata.json
├── metadata.json.sig
└── data.tar.gz.gpg
    └── data.tar.gz
        ├── content/
        |   ├── [file1]
        |   ├── [file2]
        |   └── ...
        └── checksum.sha256

metadata.json

Metadata file containing the following information:

• transfer_id: numeric ID associated to each named data transfer. In authenticated mode, or if verify package is activated, sett checks via Portal that the sender is associated with the given data transfer and verifies the data transfer is valid and authorized.
• sender: fingerprint of the sender's public OpenPGP key, a 40-character hexadecimal string.
• recipients list of fingerprints of the recipients' public OpenPGP keys, i.e. a list of 40-character hexadecimal strings.
• timestamp: point in time when a metadata file was generated. Uses the RFC 3339 format, e.g. 2024-09-03T14:06:32.675879Z
• checksum: calculated value used to verify the integrity of the data. sett uses a cryptographically safe hash algorithm which allows the receiver of the package to verify the integrity of its content before starting to decrypt and unpack it. The checksum is calculated on the compressed and encrypted data.
• checksum_algorithm: algorithm used to compute checksum. Currently SHA256 is used, where SHA stands for Secure Hash Algorithm 256 bits (64 characters). The checksum of any data package – regardless of its size – is therefore always 64 characters.
• compression_algorithm: algorithm used to compress the data before encryption in order to decrease a data package’s size and ultimately speed up the transfer time. sett uses zstandard by default, which is currently the fastest compression algorithm available. Other options are gzip and stored (no compression).
• purpose (optional): intended use for the data package. Values can be PRODUCTION, TEST, or null (optional).
• version: version of the sett package specifications (not to be confused with the version of sett itself). The sett application is designed to be backwards compatible with older sett packages.
• extra (optional): additional, custom information regarding this data package. Must be given in a key-value format (see example below). Both keys and values are strings. To be able to add additional information to a sett package, you need to [x] Enable extra metadata in the sett settings before encrypting data.

metadata.json.sig

Detached PGP signature for the metadata.json file.

data.tar.gz.gpg

A tarball encrypted using the receiver's public PGP key and signed with the sender's private key. It is compressed with compression_algorithm.

[file1], [file2]

One or several data files can be transferred. Data to be sent can be in any format, e.g. .txt, .csv, .dat.

checksum.sha256

sha256 checksum file of all files present in data.tar.gz. This is used to make sure nothing was corrupted during the encryption/transfer/decryption process.

File example

Examples of the content and structure of the metadata and checksum files.

metadata.json

{
  "transfer_id": 42,
  "sender": "AAABFBC698539AB6CE60BDBE8220117C2F906548",
  "recipients": ["D99AD936FC83C9BABDE7C33E1CF8C1A2076818C3"],
  "timestamp": "2020-01-29T15:31:42+0100",
  "checksum": "a8eb0ee5a6a53326b1c6f9bf94136da5d98a1dc6dceee21d62f694d71c4cf184",
  "checksum_algorithm": "SHA256",
  "compression_algorithm": "gzip",
  "purpose": "PRODUCTION",
  "version": "0.7",
  "extra": {
    "key1": "value1",
    "key2": "value2"
  }
}

checksum.sha256

41421f0c4a5353a5a0cdd37de3fd80a840a190ca997ad8044a67c4c1683f7b63 file1.csv
35ba157ed1c3269d731a438c466790a4f481bb49805e2d1f380df0c636792ff6 folder1/file.txt
fd9ebdbcc1a5fc35ded6e78a6b16ef658502c9d0b05dd4a2185d0f94ccf165cf folder1/folder2/file.txt

10 - Docker image

In addition to the desktop app and a standalone CLI tool, sett is also distributed as a Docker image, which can be used to run the tool in a containerized environment. This page provides instructions on how to use the sett Docker image.

All images are available in the GitLab container registry.

Pull an image with the following command:

docker pull registry.gitlab.com/biomedit/sett-rs/sett:5.2.0
# To simplify the subsequent commands, tag the image as `sett`
docker tag registry.gitlab.com/biomedit/sett-rs/sett:5.2.0 sett:5.2.0

Most sett commands require access to the public and private OpenPGP keys. Keys can be provided in two ways:

• As individual key files (see the --signer-path and --recipient-path flags in the encrypt and decrypt subcommands).
• As key stores.

When using individual key files, mount the directory containing these files as a volume in the container (or mount individual files):

docker run -it --rm -v ./openpgp:/openpgp:ro -v ./data:/data \
sett:5.2.0 encrypt local -S /openpgp/alice.pgp -R /openpgp/bob.pgp -o /data /data/README.md

Key stores can be mounted in a similar way. However, they must be mounted at the specific location within the container.

docker run --rm \
-v ./pub_store:/root/.local/share/pgp.cert.d \
-v ./priv_store:/root/.local/share/sequoia/keystore \
sett:5.2.0 keys list

If you want to use custom locations for the key stores, you can specify them using environment variables. For more information on key store location see CLI settings.

docker run --rm \
-e PGP_CERT_D=/certificate_store \
-e SETT_KEYSTORE=/keystore \
-v ./pub_store:/pub_store \
-v ./priv_store:/priv_store \
sett:5.2.0 keys list

Encryption and decryption subcommand might need a password for unlocking the secret OpenPGP key. This can be provided as an environment variable (see CLI settings):

# Password as an environment variable
docker run -it --rm -v ./openpgp:/openpgp:ro -v .:/data \
-e SETT_OPENPGP_KEY_PWD="secret" \
sett:5.2.0 encrypt local -S /openpgp/alice.pgp -R /openpgp/bob.pgp -o /data /data/README.md

# Password from a file
docker run -it --rm -v ./openpgp:/openpgp:ro -v .:/data \
-e SETT_OPENPGP_KEY_PWD_FILE=/pgp_key_pwd \
-v ./password_file:/pgp_key_pwd:ro \
sett:5.2.0 encrypt local -S /openpgp/alice.pgp -R /openpgp/bob.pgp -o /data /data/README.md

11 - Frequently asked questions and bug reports

1. Proxy

ALL_PROXY=https://host.domain:port sett-gui

2. Logging

Bug reports

To report a problem with sett or if you have a question not covered in the present documentation please open an issue on our public gitlab repo https://gitlab.com/biomedit/sett-rs/-/issues

12 - Source code

sett is licensed under the GPLv3 (GNU General Public License) and the source code is available at https://gitlab.com/biomedit/sett-rs

sett is developed as part of the BioMedIT project.

13 - Benchmarks

sett performance benchmarks

Test setup

The benchmarks were run on an Apple Silicon M3 machine with 16-Core CPUs, 128GB memory, and 2TB disk. Version 5.2.0 of sett CLI was used.

One single binary file was used, with default compression (Zstandard, level 3). Decreasing the compression level and/or using text data will influence the results.

Transfer speed is highly dependent on the infrastructure connecting the data sender and recipients. In these benchmarks, data was transferred to the same machine sending the data, therefore representing a best-case scenario in terms of transfer speed.

Results

The table and figure below present throughput speeds for different sett workflows, all values are averages of 5 runs performed with hyperfine:

• Encrypt: compress, encrypt package and transfer (S3, SFTP) data.
• Transfer: transfer already packaged data. There is no compression and encryption involved in this workflow.
• Decrypt: decrypt and decompress a data package, either locally or streamed from S3.

Throughput as function of file size: as can be seen, the curves in the figure below are linear, indicating that the throughput is constant regardless of the file size.