User Guides

• 1: sett - Secure Encryption and Transfer Tool
• 1.1: What is sett?
• 1.2: Quick start guide
• 1.3: Download sett
• 1.4: OpenPGP key management
• 1.5: Encrypting, transferring, and decrypting data
• 1.6: Generating SSH keys
• 1.7: Authenticated Mode
• 1.8: Settings
• 1.9: Data package specification
• 1.10: Docker image
• 1.11: Frequently asked questions and bug reports
• 1.12: Source code
• 1.13: Benchmarks
• 2: BioMedIT Portal User Guide
• 2.1: Login
• 2.2: Home
• 2.3: Profile
• 2.4: Groups
• 2.5: Projects
• 2.6: Data Transfers
• 2.7: Terms Of Use
• 2.8: Contact Us
• 2.9: Central Services
• 2.9.1: DCC GIT
• 2.9.2: Harbor
• 3: Quick links for SWITCH edu-ID
• 4: SPHN-DCC Confluence

1 - sett - Secure Encryption and Transfer Tool

Welcome to the official sett documentation page.

Please use the table of contents menu to the left or below to navigate the topics.

Table of Contents

1.1 - What is sett?

sett stands for “Secure Encryption and Transfer Tool” and is an application that facilitates and automates data packaging, encryption, and transfer. It is written in Rust, a fast and memory-safe programming language. sett is available both as a desktop and a command line application.

sett is developed as part of the BioMedIT project. It is licensed under the GPLv3 (GNU General Public License) and the source code is available from its public GitLab repository.

1.2 - Quick start guide

For the complete guide on how to use sett, please refer to Encrypting, transferring, and decrypting data and OpenPGP key management.

GUI (Graphical User Interface)

Initial setup

1. Download sett-gui from the download page. If you downloaded an installer, install sett-gui by double-clicking on the installer file.
2. Run sett-gui by double-clicking on the executable file or by launching the installed app.

Key management

1. If you do not already have a private/public OpenPGP key pair, go to the Keys tab and create one clicking on Add > Generate new key pair. See also the instructions given in the Generate a new public/private OpenPGP key pair section.

   You should then see your new key listed in the Keys tab, along with “Private” label that indicates that the private material for this key is present in the local keystore.

   BioMedIT

   Your OpenPGP key must be registered with the BioMedIT portal before it can be used to sign or decrypt data packages. Please see the Register your public OpenPGP key in the BioMedIT portal section for details.

2. If not already done, download the public OpenPGP key of the recipient(s) to whom you intend to send data (or from whom you will receive data). Go to the Keys tab and click on Add > Import from keyserver. See also the instructions given in the download public OpenPGP keys from the keyserver section.

3. Just after downloading the recipient’s OpenPGP key, verify it to make sure that it is genuine. This can be done by either:

   • If you are a BioMedIT user: verify that the recipient’s key is labelled with a green Approved label. You can also expand the details of the key by clicking on the key in the list or on the small down arrow button to the right and verify that the Approval status is set to “Key is approved on Portal”, and the Revocation status is set to “Valid”.
   • Alternatively, contact the key owner and verify the key fingerprint with them.

Encrypting and sending data

1. Go to the Encrypt and Transfer Data tab.

2. Add one or more files and directories to encrypt by clicking the Add files or Add directories buttons.

3. Select sender: select your own OpenPGP key. This is the key that will be used to sign the encrypted data.

4. Select recipients: add one or more recipients by selecting them in the drop-down. These are the keys that will be used to encrypt the data, i.e. only these recipients will be able to decrypt the data.

   BioMedIT

   The selected recipients must be approved Data Managers of the project for which data is being encrypted.

5. Data Transfer ID: specifying a valid Data Transfer Request ID is mandatory when a data package is transferred into the BioMedIT network. For other destinations, the Data Transfer ID field can be left empty (or set to any arbitrary value), and the Verify package checkbox must be disabled (in the Settings tab).

   BioMedIT

   The Verify package checkbox should always be enabled, since a valid Data Transfer ID is required by the BioMedIT network.

6. Select destination: select local and choose a destination directory to encrypt to your local file system. Select s3 or sftp to encrypt and transfer directly to an S3 object store or an SFTP server, respectively.

7. Click Encrypt data (local) or Encrypt and transfer data (s3 or sftp) to run the encryption workflow on your data.

Sending existing data packages

1. Go to the Encrypt and Transfer Data tab.

2. Select a file to transfer using the add sett Package button.

3. Select the Destination to be used (sftp, s3).

4. Enter the required destination parameters.

   BioMedIT

   For transfers into the BioMedIT network, the destination parameters are provided by:

   • Your local BioMedIT node, for transfers over sftp;
   • By BioMedIT Portal, for transfers over s3. Please see the dedicated section of the Portal user guide for more information.

5. Click Transfer data to start transferring your data package.

Decrypting data

1. Go to the Decrypt tab.
2. Select a data package to decrypt using the Select Package button.
3. Specify your desired destination directory.
4. Click on Decrypt package.

CLI (Command Line Interface)

The main commands to manage keys, encrypt, transfer and decrypt data with sett command line interface are given here.

sett --help
sett encrypt local -h
sett encrypt local --help

OpenPGP key management

# Generate a new key pair
sett keys generate

# Import sender/recipient(s) public keys:
sett keys import from-keyserver alice@example.com

Encrypt and transfer data

# Data encryption only, saving to local disk
sett encrypt local --signer alice@email.com --recipient bob@example.com --output . FILES_OR_DIRECTORIES_TO_ENCRYPT

# Data encryption and transfer to object store
sett encrypt s3 --signer SIGNER_KEY --recipient RECIPIENT_KEY --recipient-path RECIPIENT_PATH
--endpoint ENDPOINT --bucket BUCKET --access-key ACCESS_KEY --secret-key SECRET_KEY \
FILES_OR_DIRECTORIES_TO_ENCRYPT

# Data encryption and transfer to object store, using portal authentication
sett encrypt s3-portal --signer SIGNER_KEY --recipient RECIPIENT_KEY --dtr DATA_TRANSFER_ID  \
FILES_OR_DIRECTORIES_TO_ENCRYPT

Transfer a sett package

# Data transfer of an existing sett package

# to S3 object store
sett transfer s3 --endpoint ENDPOINT \
--bucket BUCKET --access-key ACCESS_KEY --secret-key SECRET_KEY \
SETT_PACKAGE_TO_TRANSFER

# to S3 object store, using portal authentication.
# metadata.json inside to package needs to contain a valid data transfer ID
sett transfer s3-portal SETT_PACKAGE_TO_TRANSFER

# to SFTP server
sett transfer sftp --host HOST --username USERNAME --base-path DESTINATION_DIRECTORY --key-path SSH_KEY_LOCATION --key-pwd SSH_KEY_PASSWORD SETT_PACKAGE_TO_TRANSFER

Decrypt a sett package

# decrypt a local sett package
sett decrypt local SETT_PACKAGE_TO_DECRYPT

# fetch and decrypt package from S3 object store,
# using portal authentication
sett decrypt s3-portal --dtr DTR SETT_PACKAGE_TO_DECRYPT

# fetch and decrypt package from S3 object store
sett decrypt s3 \
--bucket BUCKET --access-key ACCESS_KEY --secret-key SECRET_KEY \
SETT_PACKAGE_TO_DECRYPT

1.3 - Download sett

CLI

x86_64

Desktop

Standalone Installer

CLI

aarch64 x86_64

Desktop

.AppImage x86_64 .deb

CLI

aarch64 x86_64

Desktop

.dmg

You can download the latest version of sett for your operating system by clicking on the corresponding link above.

Downloading a specific version of sett

For a complete list of available versions, please visit the releases page.

Updating sett

As only the latest version of sett is officially supported and guaranteed to work, it is strongly recommended to always keep your local installation of sett up-to-date.

If you installed sett using the installer available for your operating system, sett will automatically check for updates and prompt you to install the newest version as soon as it's available.

If you are running a portable executable, make sure you regularly check this page and download the latest version of the executable.

1.4 - OpenPGP key management

To encrypt and decrypt data, sett uses public key cryptography. If you are not familiar with public key cryptography concepts such as public and private keys, revocation signatures, or keyservers you are advised to read this introductory section.

All key management operations can be performed using both the graphical user interface (GUI) and the command line interface (CLI) of sett.

sett --help
sett encrypt local -h
sett encrypt local --help

Generate a new public/private key pair

A prerequisite to encrypt, decrypt, and transfer files with sett is to have a public/private PGP key pair.

Generate a new key pair (GUI)

To generate a new public/private key pair:

1. Go to the Keys tab and click on + Add. From the drop-down menu, choose Generate new key pair. A dialog box will appear.

2. Fill-in the required fields:

   • Name: key owner first and last name.
   • Comment: an optional comment to help identify the owner of the key (e.g., owner’s institution).
   • Email: enter the email to be associated with the key pair.
   • Password: the password must be at least 10 characters long, and it is highly recommended that it contain a mix of letters, numbers and special characters.
   • Expiration date: a date in the future to ensure that the key is not used indefinitely (default: 3 years). It is possible to extend the expiration date later.

   Important

   It is not possible to retrieve or reset a password from a generated key. Make sure to remember it and store it in a safe place (e.g. a password manager).

3. Click Next to see the summary.

4. Click “Generate key” to create the new key pair (this can take a few seconds).

5. A new pop-up window will appear to confirm that the key was generated successfully and to display the revocation signature associated with the new key.

   Copy the revocation signature to a safe location, ideally a password manager, and click Next.

6. The next screen of the pop-up window allows to upload the public part of the key to the keyserver. Make sure to check the checkbox to associate the key with your email address. To establish this association, the keyserver will send an email to verify that you are the legitimate owner of the email associated with the key. It also means that other people will now be able to search for your key on the keyserver using your email address, otherwise your key will only be searchable using its full fingerprint.

7. The new key should now be listed in the keys tab.

8. Now that your new OpenPGP key is created, make sure to register it in the BioMedIT portal.

Generate a new key pair (CLI)

To generate a new key pair using sett command line interface:

sett keys generate --name "Alice Smith" --email alice.smith@example.com

After creating the new key, sett will display a revocation signature for it. Make sure you keep it in a safe place, such as a password manager. Revocation signatures can optionally be automatically exported to a file by using the --rev-sig option:

sett keys generate --name "Alice Smith" --email alice.smith@example.com --rev-sig alice.rev

List keys

List keys (GUI)

Go to the Keys tab.

List keys (CLI)

Use the keys list subcommand.

sett keys list

Export/import private keys

In some situations (e.g., new computer setup, remote sett environment) you might need to copy or move your private key to a different machine. This can be done by exporting the private key to a file, transferring the file to the new machine, and importing it.

Export key (GUI)

Click on the ... icon next to the key you want to export and select “Export public and private key”.

Export key (Cli)

Use the keys export subcommand:

# The key identifier can be its fingerprint, key ID or the user's email address.
sett keys export -o private_key.pgp -p alice.smith@example.com

Note that keys can also be exported in ASCII format (instead of binary) by adding the -a, --armor option.

Import key (GUI)

Click on the + Add button and select “Import from file or clipboard”.

Import key (CLI)

Use the keys import subcommand.

sett keys import from-file -p private_key.pgp

Note the usage of the -p option to import both the private and public parts of the key. If -p is omitted, only the public part of the key is imported.

Verify that the key has been correctly imported with sett keys list.

Ensure that you store any backed-up secret keys in a secure location and in an encrypted form (typically in a password manager).

Upload public key to the keyserver

sett allows users to upload their public OpenPGP key to a keyserver.

Upload public key to the keyserver (GUI)

1. Click on the ... icon next to the key you want to upload and select “Upload public key to keyserver”.
2. A dialog box will appear to ask for confirmation.
3. Make sure to check the checkbox to associate the key with your email address. To establish this association, the keyserver will send an email to verify that you are the legitimate owner of the email associated with the key. It also means that other people will now be able to search for your key on the keyserver using your email address, otherwise your key will only be searchable using its full fingerprint.
4. Click Upload to keyserver.
5. If you have selected the “associate key with email” checkbox, you will shortly receive an email from keyserver@keys.openpgp.org that contains a confirmation link to prove that you have access to your email.
6. Open the email and click on the link to associate your key with your email address. Other users now can find your public key with your email address.

Upload public key to the keyserver (CLI)

Use the keys upload subcommand.

sett keys upload alice.smith@exaple.com

In order to also trigger email verification for your key, add the -v, --verify option:

sett keys upload --verify alice.smith@exaple.com

Register public OpenPGP key in the BioMedIT portal

If you are not a BioMedIT user, this section is not relevant for you and can be skipped.

BioMedIT OpenPGP key status

OpenPGP keys used to encrypt, sign, and decrypt data within the BioMedIT network require the approval of the BioMedIT key validation authority. The information of whether a key is trusted or not is stored as key status in the BioMedIT portal. This is the reason why all OpenPGP keys used within BioMedIT must be registered with the BioMedIT portal.

When an OpenPGP key is first registered in the BioMedIT portal, its status is initially set to PENDING (i.e. it is awaiting approval). A key must have the APPROVED status before it can be used to encrypt or sign data packages within the BioMedIT network.

The list of key statuses is as follows:

• PENDING: a key approval request was submitted, but the key has not been approved yet. This is a manual process and can take from a few hours or up couple of days.
• APPROVED: key is approved for usage within the BioMedIT network. Only approved keys can be used to encrypt, sign, and decrypt data packages within the BioMedIT network.
• APPROVAL-REVOKED: approval of the key has been revoked by the BioMedIT key validation authority.
• KEY-REVOKED: key has been revoked by its owner.
• REJECTED: key is not trusted by the BioMedIT key validation authority.
• DELETED: key has been removed from the keyserver by its owner.
• UNKNOWN KEY: key has not been registered on the BioMedIT portal. If it is your own key, please register it. If it is the key of someone else, please ask them to register their key.

To verify that a key is trusted, sett connects to the BioMedIT portal and retrieves the status of the key. For this reason, it is important that BioMedIT users register their OpenPGP key with the BioMedIT portal.

In cases where sett is used outside of the BioMedIT project, or the portal is not reachable, sett can still be used to encrypt, decrypt, and transfer data. In this case, you need to uncheck Verify package in the Settings tab.

The status of a public key in the BioMedIT portal can be easily checked in GUI by going to the Keys tab and clicking on one of the keys. Approved keys will also have a green “Approved” badge.

How to register your public OpenPGP key in the BioMedIT portal?

To register a new OpenPGP key with the BioMedIT portal, proceed as follows:

1. Make sure you have successfully uploaded your key to the keyserver and that you have completed the email verification procedure with the keyserver (i.e. your key must be verified with the keyserver).

2. To make sure that your key is present on the keyserver and is verified, go to the keyserver home page in your browser and search for the email associated with your key. You should get a message saying that a key was found for your email address and the corresponding key fingerprint should be displayed.

3. Copy the fingerprint (40-character string) of your key.

4. Log in to the BioMedIT portal.

5. Go to the Profile / OpenPGP Keys.

6. Click on the "+ OPENPGP KEY" button, a dialog box will open.

   Note: if the button is missing, it is probably because you already have an active key in the portal. Each user can only have 1 active key at a time - see the information box above.

7. Enter your full key fingerprint (must be exactly 40 characters long) in the dialog box, then press the green search icon to the right.

   This will retrieve the User ID and Email address associated with the fingerprint from the keyserver and display them in the dialog box.

8. Verify the user ID and mail address. If they are correct for your OpenPGP key, then click on Confirm.

9. A request to approve your key has now been sent to the BioMedIT key validation authority. Generally requests are processed quickly (in a matter of hours), but occasionally it might take slightly longer as this is a manual process.

   Please contact the BioMedIT support if your key has not been approved after a couple of days.

Download public keys from the keyserver

In order to encrypt data for a specific recipient (who will be able to decrypt it), you will need to have the public OpenPGP key of that recipient(s) available in your local keyring.

Download public keys (GUI)

To download a public PGP key from the keyserver:

1. In the Keys tab, click on + Add. From the drop-down menu, choose Import from keyserver. A dialog box will open, allowing to search for public keys stored on the keyserver.

2. In the search field, enter either the full email address or fingerprint of the public key you are looking for and click Import key.

   For instance, if searching for the key “Bob Test Key bob@example.com” with fingerprint “AEED7A96F339881F6FE8291464A1E0150613807D”, one can search for either “bob@example.com” or “AEED7A96F339881F6FE8291464A1E0150613807D”.

3. If the key you are looking for was found, you will see a success message with key details.

Download public keys (CLI)

The keys import from-keyserver subcommand can be used to download public OpenPGP keys from the keyserver.

sett keys import from-keyserver alice.smith@example.com

Remove your public keys from the keyserver

While it is not possible to remove an actual key from the keyserver, it is possible to remove all personally identifiable information from it (user ID and email). Such keys are called unverified.

To remove personal information associated with a key, go to the keyserver’s manage key page, enter the email associated to the key and click on Send link.

You will receive an email with further instructions on how to proceed to remove your key’s user ID and email from the keyserver.

Delete keys from your local machine

Users are strongly discouraged from deleting keys from their local keystore. Instead of deleting a key, users should revoke it. Revoking a key informs others that the key should not be used anymore, and is a safer way to handle unused or compromised keys.

Deleting private keys is possible from sett.

• GUI: Click on the ... icon next to the key you want to delete and select “Delete”.

• CLI: Use the keys delete-private-key subcommand.

  sett keys delete-private-key alice.smith@exaple.com
  

Deleting public keys is not possible with sett. However, it is possible to delete a public key by manually deleting the file containing the keys using your file explorer or a shell command.

In the local sett public keystore, keys are stored as files. They are named after the fingerprint of the key’s primary key. For instance, a key with fingerprint 3b17f529665fe012ef54f4a1714fdf98b6e828df would be stored under:

• Public key: <keystore-path>/3b/17f529665fe012ef54f4a1714fdf98b6e828df

The location of the keystore (<keystore-path> above) is operating-system dependent:

• Linux: ~/.local/share/pgp.cert.d
• Windows: %UserProfile%\AppData\Roaming\pgp.cert.d
• MacOS: ~/Library/Application Support/pgp.cert.d

You can also get the exact location of the key in sett GUI:

1. Click on the ... icon next to the key you want to delete and select “Delete”.
2. A dialog box with important information about key deletion, as well as the exact location of the key will be displayed.

Generate a revocation signature

A prerequisite for revoking an OpenPGP key is to have generated a revocation signature for it. If the OpenPGP key to revoke was generated with sett, you should already have a revocation signature ready to use. If you do not have a revocation signature yet, you can generate one with GUI:

1. Click on the ... icon next to the key you want to generate revocation signature and select “Create revocation signature”.

2. Fill-in the required fields:

   • Reason: the reason for revoking the key.
   • Message: a short explanation to the reason for revoking the key.
   • Password: the password for your private key.

3. Click Generate signature.

4. A new pop-up window will appear to confirm that the revocation signature was generated successfully. Copy the revocation signature to a safe location, ideally a password manager (anyone with access to a revocation signature can revoke the key for which it was generated).

Revoke your key

If a private OpenPGP key has been compromised, is no longer usable (e.g. password is lost), or should no longer be used for any other reason, it must be revoked.

A prerequisite for revoking a PGP key is to have generated a revocation signature for it. If the OpenPGP key to revoke was generated with sett, you should already have a revocation signature ready to use. If you do not have a revocation signature yet, please generate one by referring to the Generate a revocation signature section.

Revoke your key (GUI)

1. Click on the ... icon next to the key you want to revoke and select “Revoke”.

2. Either paste you revocation signature from the clipboard or load it from a file.

   Warning: proceed with caution, a revoked key cannot be “un-revoked”.

3. Click on Revoke key.

4. When expanding the revoked key from the List of available keys, you should now see the Revocation status as “Revoked”. From this point on, the key can no longer be used with sett.

5. If you have previously shared your key via a keyserver (e.g. keys.openpgp.org), you must also re-upload your revoked key to that keyserver.

   This will allow other users to update their local copy of your public key, informing them that it is no longer valid. To upload your revoked public key, please refer to the Upload your public PGP key to the keyserver .

   If your key was never present on any keyserver, this step should be skipped.

Revoke your key (CLI)

Use the keys revoke subcommand.

sett keys revoke alice.smith@example.com compromised "My dog ate it"

After a key has been revoked, it must be uploaded again to any keyserver(s) where it is present, so that the revocation can be shared with others. This can be done with sett as illustrated in the Upload your public PGP key to the keyserver.

Update expiration date for your key

It is possible to set an expiration date for an OpenPGP key. This is useful to ensure that the key is not used indefinitely. It is possible to extend the expiration date at any time.

Update expiration date (GUI)

sett GUI warns you when your key is about to expire. On the Keys tab, you will notice an Expiring badge starting from three months before the expiration date. On top of that, a warning will be displayed every time you use that key for signing or decryption.

To set a new expiration date for your key from the GUI:

1. Click on the ... icon next to the key you want to update the expiration date for.

2. Click on Update expiration date.

3. Choose a new expiry date.

4. Click on Set new expiry date.

5. Unlock your key by entering the password, if needed.

Update expiration date (CLI)

sett CLI warns you every time you use a key which is about to expire for signing or decryption. This happens starting from three months before the expiration date.

To set a new expiration date for your key from the CLI:

sett keys expire alice.smith@example.com --expiration 3y

For details about the supported time specifications for the --expiration option, run:

sett keys expire --help

Migrating keys from the GnuPG keyring to the sett keystore

Public OpenPGP keys located in your GnuPG keyring are not automatically detected by sett, and they must be migrated to sett’s public keystore. In contrast, private keys present in GnuPG can be used in sett without migration (however, it is still possible to migrate them).

Note that to migrate public keys (e.g., keys from other people), you can simply re-download them from the keyserver as shown in the Download public PGP keys from the keyserver section.

• GUI:

  1. In the Keys tab, click on + Add. From the drop-down menu, choose Import from GnuPG. A dialog box will open, containing a list of keys in your GnuPG keyring.
  2. Look for the correct key and click on Import. You should now see the key listed in the List of available keys. Note that if you are importing a private key, you will be asked to enter its password.

• CLI: use the keys import from-gpg subcommand.

  # The search term can be an email or fingerprint.
  sett keys import from-gpg alice.smith@example.com
  

Introduction to public-key cryptography and OpenPGP

Public-key cryptography is a method for secure communication between two or more users. In this system, each user has a pair of unique keys consisting of a private key and a public key. Public and private keys are linked in the sense that, data encrypted with a given public key can only be decrypted with the matching private key, and data signed with a given private key will only be recognized by the matching public key.

Because these keys are based on the OpenPGP protocol, they will here be referred to as OpenPGP keys.

Public and private OpenPGP keys:

• Public keys are used to encrypt data, as well as for verifying signatures made on files or emails. By design, public keys are intended to be shared with other people and therefore no particular effort is required to keep them secret. In fact, public keys are often uploaded to public servers, known as keyservers, where they are accessible to anyone. No password is required to use a public key.

  Typically, public keys are used by data senders to encrypt data for one or more recipient(s), and by data recipients to verify signatures of files or emails (to ensure the sender is genuine).

• Private keys, sometimes also referred to as a secret keys, are used to decrypt data, sign files and sign other people’s public keys. To increase security, private keys should always be password protected.

Typically, private keys are used by data recipients to decrypt data, and by data senders to sign the files they encrypt.

sett uses the open source implementation of public-key cryptography provided by Sequoia-PGP: a modular OpenPGP implementation in Rust.

It is possible - and often desirable - to both encrypt and sign a file. This ensures that the data can only be read by the intended recipient, and that the recipient can be confident the sender is legitimate. This is precisely what sett does:

• Encrypting files, so that only the intended recipient(s) can read them.
• Signing files, so that the recipient(s) can trust the sender is genuine.

Key fingerprints

Each pair of public/private OpenPGP keys is identified by a unique fingerprint. Fingerprints are 40 characters long hexadecimal strings (digits and upper case A-F letters) that look like this:

238565936FCFF3F200219990941A3EC20555F781

Since nothing is preventing two OpenPGP keys to have the same user name and email address, it is critical that users always verify the genuineness of new keys before (or just after) importing them into their local keyring (i.e. their local OpenPGP key database).

Ensuring a key is genuine can be done in two different ways:

• Ask the key owner to provide their key’s fingerprint via a trusted communication channel (e.g. over the phone), and then verify that the fingerprint of the newly imported key does indeed match the fingerprint communicated to you by its owner.
• Using sett-gui, verify that the key status of the key is APPROVED (can only be checked after you imported the key).

File encryption

In public key cryptography, the sender encrypts a file using one or more recipient(s) public key(s). Once a file is encrypted, no one can read the file without having access to a private key that matches the public key(s) used for encryption. This ensures that only the intended recipient(s) can decrypt the file, because they are the only one to have access to the matching private key.

File signing

The objective of file signing is to guarantee to the recipient of a file (or email) that the sender is genuine, and not someone else trying to impersonate the sender.

To achieve this, the sender signs the file with their private key (password required), and shares their public key with the recipient (typically via a keyserver). The recipient can then validate the authenticity of the signature using the public key of the sender. Since public keys are non-sensitive, they can be distributed publicly. In fact they are intended for this purpose, hence their name.

Revocation signatures

In the unfortunate event that a user either i) forgets their private key’s password or ii) have their private key and password stolen/compromised, they will need a way to let other people know that their public key should no longer be trusted and used.

This is because:

• If the password was forgotten: the key owner won’t be able to decrypt data anymore.
• If the private key was compromised: someone else might be able to decrypt data encrypted with the public key, and to illegitimately sign files!

This situation is what revocation signatures are for: by applying a revocation signature to a public key, and then sharing the revoked key with others (e.g. via a keyserver), the key owner signals that their key is now “revoked” and should no longer be trusted nor used. After a key has been revoked, it can no longer be used to encrypt/decrypt data with sett.

Revocation signatures can be generated at anytime from the private key, but the best practice is to generate them directly after a new key pair is created. This ensures that the revocation signature will be available even if the private key or its password is lost.

Since anyone with access to a revocation signature will be able to revoke the associated key, revocation signatures must be stored securely - e.g. in a password manager - and should never be shared with anyone.

Exchanging public keys via a keyserver

Encrypting files for a specific recipient requires to have the recipient’s public key in one’s local keyring (a keyring is a local database containing OpenPGP keys). Similarly, verifying a signature on a file or a public key requires to have the signee’s public key available in one’s local keyring.

Public keys are not sensitive data, and therefore can be sent unencrypted via email. However, when having frequent key exchanges between multiple actors, sending public OpenPGP keys around by email quickly becomes cumbersome. A solution to this problem is using a so called keyserver to share public keys. Keyservers are public or private servers whose sole purpose is to store public OpenPGP keys and allow users to search for them.

1.5 - Encrypting, transferring, and decrypting data

The instructions below describe the process of encrypting, transferring, and decrypting data. sett provides both a graphical user interface (GUI) and a command line interface (CLI) for these operations.

sett --help
sett encrypt local -h
sett encrypt local --help

Encrypting and sending data

sett allows the encryption of any combination of individual files and directories.

The files are first compressed into a single data.tar.gz archive, which is then encrypted with the public key of one or more recipient(s), and signed with the sender’s key. The encrypted data (data.tar.gz.gpg) is then bundled with a metadata file - a plain text file that contains information about who is sending the file and to whom it should be delivered - into a single .zip file. The specifications of the output .zip files produced by sett are described in data package specification.

sett supports multi-recipient data encryption. This allows the encrypted file to be decrypted by multiple recipients.

sett also ensures the integrity of the transferred files by computing checksums on each file that is packaged, and adding this information to the encrypted data. The integrity of each file is verified automatically upon decryption of the file by sett, providing the guarantee that all files were transferred flawlessly.

Output file naming scheme

By default, encrypted output files produced by sett are named after the pattern:

<project code>_<YYYYMMDD>T<HHMMSS>.zip

where:

• <project code> is the abbreviation/code associated with the project. If Verify package is disabled, no project code is added as a prefix to the output file name.
• <YYYYMMDD> is the current date (Year, Month, Day).
• <HHMMSS> is the current time (Hours, Minutes, Seconds).

Example: demo_20220211T143311.zip, here demo is the project code.

Using the sett command line interface when encrypting to the local file system, it is possible to completely override the above output file naming scheme by passing the -o, --output option. Overriding the naming scheme is not possible when using sett-gui or when encrypting to a remote S3 or SFTP destination.

Encrypting and sending data (GUI)

Data can be encrypted and either saved to the local file system, or sent directly to a remote server that supports one of the following protocols:

• S3 object storage
• SFTP

1. Go to the Encrypt and Transfer Data tab.

2. Select files and/or directories to encrypt: using the Files and Folders buttons, select at least one file or directory to encrypt. Files and directories can also be dragged and dropped into sett.

   Note

   If you already have an encrypted data package in your local file system, you can select it with sett Package (or drag and drop it into sett), then choose a remote destination (continue reading at Select destination).

3. Select data sender: in the drop-down list found under Select sender, select your own OpenPGP key (you are the data sender). For most users, there should be only one key in the Sender drop-down menu: their own key.

   Note

   The Sender key is used to sign the encrypted data, so that the recipient(s) can be confident that the data have been sent by a trusted source.

4. Select data recipients: add one or more recipients by selecting them from the drop-down list found under Select recipients. Recipients are the people for whom data should be encrypted: their public OpenPGP key will be used to encrypt the data, and only they will be able to decrypt it.

   BioMedIT

   Only recipients assigned to the role of Data Manager of the project for which data is being encrypted are permitted as data recipients.

5. Data Transfer ID: Data Transfer Request ID associated with the data package that is being encrypted. Specifying a valid DTR ID is mandatory to transfer data into the BioMedIT network.

   For data not intended to be transferred into the BioMedIT network, the DTR ID field can be left empty (or set to any arbitrary value). In this case, Verify package must be disabled (in the Settings tab).

   BioMedIT

   DTR ID field is mandatory. Only files encrypted with a valid and authorized DTR ID value can be transferred into the secure BioMedIT network. For this reason, BioMedIT users should always leave the Verify package checkbox enabled.

6. This section is only visible if “Enable extra metadata” is selected in the Settings tab.

   Here you can add extra metadata to the data package in the form of key-value. This metadata will be stored in the metadata file of the encrypted data package. After inserting the key and value, click the + button (or hit “Enter”) to add the new entry. You can add multiple key-value pairs.

7. Select destination: destination where to encrypt (and send) the data.

   • Local: The local file system. When using this destination, it’s possible to specify:

     • Output location: directory where the encrypted file should be saved. By default, output files are saved to the user’s home directory. This default behavior can be changed by changing Default output directory in the Settings tab.

   • S3: A remote S3 compatible object store.

   • SFTP: A remote SFTP server.

   When encrypting to a remote destination, a number of options must be specified. Please refer to remote destination options for details.

8. You are now ready to create an encrypted data package: click Encrypt data or Encrypt and transfer data if you are encrypting to a remote S3 or SFTP destination. A pop-up will appear, asking for the password associated with the sender’s key. After the password is entered, data compression and encryption will start. Progress and error messages are displayed in the Tasks tab.

   When the encryption completed successfully, a notification will pop-up with a message that reads: “Encryption job finished”.

At this point, all input files are compressed, encrypted, and bundled into a single .zip file. If the destination was S3 or SFTP, data has also been transferred to the remote destination.

Encrypting and sending data (CLI)

To create an encrypted data package and save it to the local file system, use the encrypt subcommand. If you already have an encrypted data package in your local file system and want to transfer it to a remote destination, use the transfer subcommand. Both subcommands share the same options for specifying the remote destination.

# General syntax:
sett encrypt local --signer SENDER --recipient RECIPIENT --dtr DATA_TRANSFER_ID --output OUTPUT_FILENAME_OR_DIRECTORY FILES_OR_DIRECTORIES_TO_ENCRYPT

# Example (long command line options):
sett encrypt local --signer alice@example.com --recipient bob@example.com --dtr 42 --output . ./file_to_encrypt.txt ./directory_to_encrypt

# Example (short command line options):
sett encrypt local -s alice@example.com -r bob@example.com -dtr 42 -o . ./file_to_encrypt.txt ./directory_to_encrypt

Data can be encrypted for more than one recipient by repeating the flag -r/--recipient, e.g. -r RECIPIENT1 -r RECIPIENT2 option:

# In this example, Alice encrypts data for both Bob and Chuck.
sett encrypt local -s alice@example.com -r bob@example.com -r chuck@example.com -o . FILES_OR_DIRECTORIES_TO_ENCRYPT

--output is an optional argument for specifying the location and/or name for the encrypted output file. The --output argument can be one of the following:

• Not provided: the encrypted data is written to stdout. This can e.g. be useful to pipe the data into another application.
• A directory: the encrypted data package file is written to the specified directory, and is given a name that follows the default naming convention in sett.
• A file name: the encrypted data is written to a new file with the specified name, and in the specified directory, if the file name contains one. This overrides the default output file naming schema.

local subcommand can be replaced with s3 or sftp to encrypt and transfer data directly to a remote destination in a single command. When using s3 or sftp, the data is encrypted and streamed directly to the destination, without creating any files in the local file system. The streaming approach is faster than creating a data package locally and transferring it separately. It also saves space on the local machine when transferring large datasets.

For more information about remote destination options, please refer to the remote destination options section.

sett encrypt s3 -s alice@example.com -r bob@example.com --endpoint https://minio.my-node.ch --bucket my-project \
--access-key 23VO8RB2SIB2SF8EUL9V --secret-key wvrt7YoTTERGftf0zWnppWYSdcGplNtxuLHMn7op --session-token eyJhbGciOiJ\
IUzUxMiIsInR5cCI6IkpXVCJ9.eyJhY2Nlc3NLZXkiOiI5Vk84UkIyvimUMlKIFUVVTDc3WSIsImF0X7hhc2giOiIyRnVlZ3JmSjhTUWFXYkw2V0puek\
F3IiwiYXVkLjpbIm1pbmlvIl0sImF1dGhfdGltTRI6MTcyMTezODIxMywiZXhwIjoxNzIx0DMxODEzLCJpYXQiOjE3MjE4MzfiLKLsImlzwqI6Imh\
0dHBzOi8vcD3ydGFsLXN0YWasfmcuZGNjLnNpYi5zd2lzcy9hdXRoL38hdXRoIiwibmFtZSI6ImJpd2ciLCJqt5xpY5kiOiJjb25zb2xlQWRfgW4iLC\
JzdWIiOiIxOSJ9.PcvXcAli5Bz8ete1T265TPB1cbfgX7k8NDXU5gXy1nflxq203cG5qwAF9Oxyn1mKmwa87jsHj8HU2VUY9p5S1Q \
FILES_OR_DIRECTORIES_TO_ENCRYPT

Adding the --check option will run the encrypt command in the test mode, i.e., checks are made but no data is encrypted or sent.

Data compression algorithm can be changed using the --compression-algorithm flag. The available options are:

• zstandard (default), optimal compression and speed.
• gzip, available for compatibility with older versions of sett.
• stored, no compression.

The data compression level used by sett can be manually adjusted using the --compression-level option. Possible values depend on the selected compression algorithm:

• zstandard (default: 3)
  • 1 (lowest compression, fastest)
  • 21 (highest compression, slowest).
• gzip (default: 6)
  • 1 (lowest compression, fastest)
  • 9 (highest compression, slowest)

Before encrypting data using the local subcommand and --output flag, sett verifies that there is enough free disk space available on the local machine to save the encrypted output file. If this is not the case an error message is displayed and the operation is aborted. Since the compression ratio of the input data cannot be known in advance, sett uses the conservative estimate that the minimum disk space required is equal to the total size of all input files to be encrypted.

To automate the encryption process, you can use environment variables to store the OpenPGP password.

sett performs DTR verification if the --verify option is passed. For non-BioMedIT-related transfers, the --verify option should not be passed.

Decrypting files

Decrypt and decompress encrypted data packages.

Please note that the decryption process includes the verification of the sender’s signature, which ensures the authenticity of the data. Verification of the sender’s signature is only possible if the sender’s public key is available in your local keyring. For security reasons, the sender’s key will not be automatically downloaded from a key server; it must be downloaded/imported manually (for more information see download public keys).

To decrypt data, you must therefore have in your local keyring:

• The private key for which the data was encrypted. In principle, this is your own private key. This key will be used to decrypt the data.
• The data sender’s public key. This key is used for signature verification purposes.

Only files that follow the sett packaging specification can be decrypted with sett.

Decrypting data (GUI)

To decrypt and decompress a file:

1. Go to the Decrypt tab.

2. Select the source from which you wish to decrypt data - either local or s3.

3. When decrypting from local, select a data package to decrypt with Select Package, or drag and drop the file into sett. When decrypting from s3, a number of options must be specified - please refer to remote destination options for details. After doing so, click on Load package.

4. Select destination directory: select a location where to decrypt/decompress the file.

   By default, output files are saved to the user’s home directory. This default behavior can be changed by changing Default output directory in the Settings tab.

5. Click Decrypt package to start the decryption and decompression process. A pop-up dialog box will appear to ask for the password associated with the OpenPGP key used to encrypt the files.

Decrypting data (CLI)

Use the decrypt subcommand to decrypt and decompress data:

# General syntax:
sett decrypt local --output OUTPUT_DIRECTORY ENCRYPTED_FILES.zip

# Example:
sett decrypt local --output /home/alice/data/unpack_dir /home/alice/data/test_data.zip

local subcommand can be replaced with s3 to decrypt data package from an S3 object store. When using s3, the data is streamed and decrypted directly from the S3 object store, without creating any temporary files in the local file system.

To decrypt data without decompressing it, add the -d, --decrypt-only option.

If the -o, --output option is omitted, the data is decrypted in the current working directory.

To automate the decryption process, you can use environment variables to store the OpenPGP password.

Remote destination options

Both GUI and CLI require specific parameters when transferring data to a remote destination. The parameters are different depending on the destination type.

• S3: A remote S3 compatible object store. When using this destination a number of options must be specified:

  • URL: The URL of the S3 object store.
  • Bucket: The name of the bucket where the encrypted data should be stored.
  • Access key: The access key to use to authenticate with the S3 object store. It is also possible to use a username instead.
  • Secret key: The secret key to use to authenticate with the S3 object store. It is also possible to use a password instead.
  • Session token: The session token to use to authenticate with the S3 object store. It is optional and only required when authenticating using temporary credentials (STS), which is the case of most users interacting with the BioMedIT infrastructure.

  BioMedIT

  This information must be retrieved from BioMedIT Portal by a data provider data engineer. Please refer to the dedicated section of the Portal user guide for instructions on how to do so. Portal allows copying the credentials to the clipboard, so that they can be pasted into sett by using the Paste from clipboard button.

• SFTP: A remote SFTP server. When using this destination, a number of options must be specified:

  • Host: URL address of the server where the files should be sent.

  • User name: the user name with which to connect to the SFTP server.

  • Destination directory: absolute path of directory where files should be saved on the server.

  • SSH key location: path of the private SSH key used for authentication to the SFTP server. This is only required if the SSH key is in a non-standard location. If missing, sett will use the SSH agent to provide the key.

    Do not confuse SSH keys - which are used to authenticate yourself when connecting to an SFTP server during file transfer - with OpenPGP keys - which are used to encrypt and sign data.

  • SSH key password: password associated with the private SSH key given under SSH key location. If your SSH key password contains characters that are not ASCII characters, and that this results in an error, please see the SSH private key with non-ASCII characters section of this guide.

  BioMedIT

  For BioMedIT users, the SFTP connection parameters User name, Host URL, and Destination directory will be provided by your local BioMedIT node.

1.6 - Generating SSH keys

SSH (Secure SHell) keys are pairs of small text files that are used to securely identify users and give them access to remote servers, e.g. when transferring data via SFTP.

SSH keys use public-key encryption and always come in pairs: a public and a private (or secret) key.

• Public key: the public key of an SSH key pair is meant to be placed on the remote machine to which a user wants to connect. Public keys are non-sensitive, and are typically shared by users with system administrators, who will place them on the machine to which the user is granted access.
• Private key: the private key of an SSH key pair is what uniquely identifies a user as the legitimate owner of a public key. In other words, having the private key that matches a given public key will give access to any machine on which a copy of public key is stored. Private SSH keys are sensitive information: they must be kept private at all times and should never be shared with anyone - not even your system administrator. Private keys can (and should) be protected by a password, so that even if someone else has access to them, they remain unusable.

Generating a new pair of SSH keys must be done only once, and, in the context of sett, is only needed if you intend to transfer data. If you are a user who only decrypts data, you do not need an SSH key.

Also, do not confuse SSH keys - used to identify yourself on a remote server - with OpenPGP keys - used to encrypt and sign data.

To generate a new SSH key pair, type the command below in your terminal (Linux and Mac) or PowerShell (Windows users - to start it, search for “powershell” in the Start menu). Note that you must replace "alice@example.org" with your own email. This will generate an SSH key pair using the ed25519 algorithm, currently the most secure public-key algorithm:

ssh-keygen -a 100 -t ed25519 -C "alice@example.org"

Windows users who do not have the ssh-keygen command installed, please see section below.

When executing the ssh-keygen command above, you will be prompted for the following information:

1. The name and location where to save the newly created keys. Here you should simply accept the default values by not entering anything and pressing “Enter” on your keyboard. Default locations are ~/.ssh/id_ed25519 on Linux and MacOS, and C:\Users\%username%\.ssh\id_ed25519 on Windows.
2. A password to protect your private SSH key against illegitimate use. Please use a password that is long enough (>=12 characters) and composed only of ASCII characters.

When the command completes, two new files are produced: id_ed25519.pub (the public key) and id_ed25519 (the private key).

On Linux and MacOS systems, after the public key is generated, its permissions must be changed with the following command (this step is not needed for Windows users):

chmod 600 ~/.ssh/id_ed25519.pub

Windows users: enabling the ssh-keygen command

Not all versions of windows come with the ssh-keygen command pre-installed. If this command is unavailable on your machine, please install it as follows:

1. Open the windows settings (shortcut: windows key + i).
2. Search for, and select, “Add an optional feature”.
3. Click on “Add a feature”.
4. Search for, and select, “Open SSH Client”.
5. Click “Install”.
6. Restart your computer.

SSH private key with non-ASCII characters password

Even though it is possible to create an SSH key pair using a password containing non-ASCII characters, it seems like those characters are encoded differently between different operating systems.

As an SSH key might be moved to a machine with another operating system, or encoding might change with a new version, it is impossible to guess the correct encoding in any case. For this reason, we recommend not to use non-ASCII characters to protect SSH private keys.

1.7 - Authenticated Mode

Why authenticated mode?

In order to simplify the data transfer process, sett offers an authenticated mode. Once the user is authenticated, the following functionalities are offered:

• list of available DTR (data transfer request) for the logged-in user
• package verification is always enforced
• data transfer from and to a BioMedIT S3-type object storage without entering credentials

Device authorization flow explained

The authenticated mode in sett implements a so-called device authorization flow, an authorization mechanism usually used by electronic devices with internet connection that lack any input capacities. In both the CLI and GUI of sett, the user is asked to visit a specific website (an OIDC provider) to identify himself. While the user is logging in, the device (in our case: sett) is periodically polling the website, until the user has successfully logged in. sett then receives a JWT token (JSON Web Token) from the OIDC provider, which allows it to fetch additional data from Portal, e.g. the list of available data transfers for the logged-in user, as well as the credentials used for transferring data to and from the S3 object storage.

The diagram below illustrates how this device authorization flow works in sett when requesting the list of available DTRs or when up/downloading data.

Device authorization flow as implemented by sett

Glossary

• CLI: Command Line Interface. This is how sett is used in a terminal.
• GUI: Graphical User Interface. E.g. the standalone sett application that runs in its own window.
• S3 object storage: the database that (temporarily) stores all sett packages. Every node runs its own storage. S3 stands for Simple Storage Service and was originally developed by Amazon.
• SFTP: Secure File Transfer Protocol. The legacy way to transfer data.
• OIDC provider: an system that redirects a user to Switch edu-ID to authenticate himself. It can issue tokens which allow sett to fetch information on behalf of the user. It plays a crucial role in implementing the device authorization flow. OIDC stands for Open ID Connect.
• JWT token: a piece of structured data which contains information about an authenticated user. It is signed, so a third party (in our case: Portal) can verify the integrity of its content. JWT stands for JSON Web Token.

1.8 - Settings

GUI settings

The sett desktop app allows a number of options to be customized via its Settings ⚙️ page. For instance, you may change the default output directory, or enable/disable package verification before a transfer.

Each setting has a predefined default value, which is used when first running the tool or if loading the current settings fails for any reasons.

Changes made to Settings become effective immediately. Changes can be reset back to their factory default by clicking on the Reset settings button.

Settings are divided into three sections: “basic”, “advanced”, and “non-editable”.

Basic

Verify package

When enabled (the default value), the following checks are made before encrypting or transferring data:

• DTR ID is valid and the transfer is authorized.
• Sender and Recipients public OpenPGP keys are approved by the BioMedIT key validation authority.
• Recipients are approved Data Managers of the BioMedIT project for which data is being encrypted.
• The name of the data package matches the pattern <project_code>_<date_format>.zip. This ensures no sensitive information is mistakenly included in the file name.

Note that that some of the above checks require communication with the BioMedIT portal. When using sett outside of a BioMedIT project, this setting should therefore be disabled.

Default output directory

Default destination directory for operations such as encryption to the local filesystem or decryption. User’s home directory is used by default.

Advanced

Enable extra metadata

When enabled, additional metadata can be added to the package in the data encryption form.

OIDC issuer URL

URL of the OpenID Connect issuer used for authentication (BioMedIT specific).

Portal URL

URL of a BioMedIT portal instance. Portal is used for key approval verification, DTR (Data Transfer Request) validation, and retrieval of data associated with a given DTR (when sett is being used in authenticated mode). The default value of this setting is: https://portal.dcc.sib.swiss.

Public key store

Directory where public OpenPGP keys are stored.

Private key store

Directory where private OpenPGP keys are stored.

Non-editable

This section displays values of setting that cannot be modified by the user. These settings are displayed here for convenience. They can be copied to the clipboard via a dedicated “copy to clipboard” button.

Log directory

Directory where log files are stored. Location of the log directory depends on the operating system:

• Linux: ${XDG_DATA_HOME}/ch.biomedit.sett/log or $HOME/.local/share/ch.biomedit.sett/log
• macOS: $HOME/Library/Application Support/ch.biomedit.sett/log
• Windows: {FOLDERID_RoamingAppData}\ch.biomedit.sett\log

Keyserver URL

URL of the OpenPGP key server used to retrieve and publish public keys. The default value is https://keys.openpgp.org.

CLI settings

The sett-cli is stateless by design, meaning that there is no persistent configuration file where settings can be modified and stored.
Instead, settings can be set via the following shell environment variables. All settings are optional and have a default value.

SETT_OPENPGP_KEY_PWD

Password to unlock the secret OpenPGP key used to decrypt or sign data. When this environmental variable is set, sett uses its content instead of interactively asking the user to enter a password.

SETT_OPENPGP_KEY_PWD_FILE

Full path and name of a file containing the password to unlock the secret OpenPGP key used to decrypt or sign data. When this environmental variable is set, sett uses its content instead of interactively asking the user to enter a password. The file containing the password should not be encrypted.

SETT_PORTAL_URL

URL of the BioMedIT Portal instance to be used. For details see the description of the GUI Portal URL setting. This setting defaults to https://portal.dcc.sib.swiss.

SETT_OIDC_CLIENT_ID

Client ID with which sett should identify with the OpenID Connect issuer (see SETT_OIDC_ISSUER_URL). Only relevant when using sett in authenticated mode. This setting defaults to sett.

SETT_OIDC_ISSUER_URL

URL of the OpenID Connect issuer used when authenticating with the BioMedIT Portal. Only relevant when using sett in authenticated mode. This setting defaults to https://login.biomedit.ch/realms/biomedit.

SETT_KEYSTORE

Directory where private OpenPGP keys are stored. Location is platform dependent:

• Linux: ${XDG_DATA_HOME}/sequoia/keystore or $HOME/.local/share/sequoia/keystore
• MacOS: $HOME/Library/Application Support/org.Sequoia-PGP.sequoia/keystore
• Windows: {FOLDERID_RoamingAppData}\org.Sequoia-PGP.sequoia\keystore

PGP_CERT_D

Directory where public OpenPGP keys are stored. Location is platform dependent:

• Linux: ${XDG_DATA_HOME}/pgp.cert.d or $HOME/.local/share/pgp.cert.d
• MacOS: $HOME/Library/Application Support/pgp.cert.d
• Windows: {FOLDERID_RoamingAppData}\pgp.cert.d

SETT_METADATA_EXTRA

Additional metadata to be included in the data package. Extra fields must be provided in the form of key=value pairs separated by a comma and without spaces. For example: SETT_METADATA_EXTRA="foo='value 1',bar=value_2"

1.9 - Data package specification

sett compresses, encrypts, and packages files in a single .zip file whose specification is described below. Only files adhering to these specifications can be transferred or decrypted by sett, and failure to comply with the specification will generate an error.

File structure

sett .zip files have the following structure:

YYYYMMDDThhmmss.zip
├── metadata.json
├── metadata.json.sig
└── data.tar.gz.gpg
    └── data.tar.gz
        ├── content/
        |   ├── [file1]
        |   ├── [file2]
        |   └── ...
        └── checksum.sha256

metadata.json

Metadata file containing the following information:

• transfer_id: numeric ID associated to each named data transfer. In authenticated mode, or if verify package is activated, sett checks via Portal that the sender is associated with the given data transfer and verifies the data transfer is valid and authorized.
• sender: fingerprint of the sender's public OpenPGP key, a 40-character hexadecimal string.
• recipients list of fingerprints of the recipients' public OpenPGP keys, i.e. a list of 40-character hexadecimal strings.
• timestamp: point in time when a metadata file was generated. Uses the RFC 3339 format, e.g. 2024-09-03T14:06:32.675879Z
• checksum: calculated value used to verify the integrity of the data. sett uses a cryptographically safe hash algorithm which allows the receiver of the package to verify the integrity of its content before starting to decrypt and unpack it. The checksum is calculated on the compressed and encrypted data.
• checksum_algorithm: algorithm used to compute checksum. Currently SHA256 is used, where SHA stands for Secure Hash Algorithm 256 bits (64 characters). The checksum of any data package – regardless of its size – is therefore always 64 characters.
• compression_algorithm: algorithm used to compress the data before encryption in order to decrease a data package’s size and ultimately speed up the transfer time. sett uses zstandard by default, which is currently the fastest compression algorithm available. Other options are gzip and stored (no compression).
• purpose (optional): intended use for the data package. Values can be PRODUCTION, TEST, or null (optional).
• version: version of the sett package specifications (not to be confused with the version of sett itself). The sett application is designed to be backwards compatible with older sett packages.
• extra (optional): additional, custom information regarding this data package. Must be given in a key-value format (see example below). Both keys and values are strings. To be able to add additional information to a sett package, you need to [x] Enable extra metadata in the sett settings before encrypting data.

metadata.json.sig

Detached PGP signature for the metadata.json file.

data.tar.gz.gpg

A tarball encrypted using the receiver's public PGP key and signed with the sender's private key. It is compressed with compression_algorithm.

[file1], [file2]

One or several data files can be transferred. Data to be sent can be in any format, e.g. .txt, .csv, .dat.

checksum.sha256

sha256 checksum file of all files present in data.tar.gz. This is used to make sure nothing was corrupted during the encryption/transfer/decryption process.

File example

Examples of the content and structure of the metadata and checksum files.

metadata.json

{
  "transfer_id": 42,
  "sender": "AAABFBC698539AB6CE60BDBE8220117C2F906548",
  "recipients": ["D99AD936FC83C9BABDE7C33E1CF8C1A2076818C3"],
  "timestamp": "2020-01-29T15:31:42+0100",
  "checksum": "a8eb0ee5a6a53326b1c6f9bf94136da5d98a1dc6dceee21d62f694d71c4cf184",
  "checksum_algorithm": "SHA256",
  "compression_algorithm": "gzip",
  "purpose": "PRODUCTION",
  "version": "0.7",
  "extra": {
    "key1": "value1",
    "key2": "value2"
  }
}

checksum.sha256

41421f0c4a5353a5a0cdd37de3fd80a840a190ca997ad8044a67c4c1683f7b63 file1.csv
35ba157ed1c3269d731a438c466790a4f481bb49805e2d1f380df0c636792ff6 folder1/file.txt
fd9ebdbcc1a5fc35ded6e78a6b16ef658502c9d0b05dd4a2185d0f94ccf165cf folder1/folder2/file.txt

1.10 - Docker image

In addition to the desktop app and a standalone CLI tool, sett is also distributed as a Docker image, which can be used to run the tool in a containerized environment. This page provides instructions on how to use the sett Docker image.

All images are available in the GitLab container registry.

Pull an image with the following command:

docker pull registry.gitlab.com/biomedit/sett-rs/sett:5.2.0
# To simplify the subsequent commands, tag the image as `sett`
docker tag registry.gitlab.com/biomedit/sett-rs/sett:5.2.0 sett:5.2.0

Most sett commands require access to the public and private OpenPGP keys. Keys can be provided in two ways:

• As individual key files (see the --signer-path and --recipient-path flags in the encrypt and decrypt subcommands).
• As key stores.

When using individual key files, mount the directory containing these files as a volume in the container (or mount individual files):

docker run -it --rm -v ./openpgp:/openpgp:ro -v ./data:/data \
sett:5.2.0 encrypt local -S /openpgp/alice.pgp -R /openpgp/bob.pgp -o /data /data/README.md

Key stores can be mounted in a similar way. However, they must be mounted at the specific location within the container.

docker run --rm \
-v ./pub_store:/root/.local/share/pgp.cert.d \
-v ./priv_store:/root/.local/share/sequoia/keystore \
sett:5.2.0 keys list

If you want to use custom locations for the key stores, you can specify them using environment variables. For more information on key store location see CLI settings.

docker run --rm \
-e PGP_CERT_D=/certificate_store \
-e SETT_KEYSTORE=/keystore \
-v ./pub_store:/pub_store \
-v ./priv_store:/priv_store \
sett:5.2.0 keys list

Encryption and decryption subcommand might need a password for unlocking the secret OpenPGP key. This can be provided as an environment variable (see CLI settings):

# Password as an environment variable
docker run -it --rm -v ./openpgp:/openpgp:ro -v .:/data \
-e SETT_OPENPGP_KEY_PWD="secret" \
sett:5.2.0 encrypt local -S /openpgp/alice.pgp -R /openpgp/bob.pgp -o /data /data/README.md

# Password from a file
docker run -it --rm -v ./openpgp:/openpgp:ro -v .:/data \
-e SETT_OPENPGP_KEY_PWD_FILE=/pgp_key_pwd \
-v ./password_file:/pgp_key_pwd:ro \
sett:5.2.0 encrypt local -S /openpgp/alice.pgp -R /openpgp/bob.pgp -o /data /data/README.md

1.11 - Frequently asked questions and bug reports

1. Proxy

ALL_PROXY=https://host.domain:port sett-gui

2. Logging

Bug reports

To report a problem with sett or if you have a question not covered in the present documentation please open an issue on our public gitlab repo https://gitlab.com/biomedit/sett-rs/-/issues

1.12 - Source code

sett is licensed under the GPLv3 (GNU General Public License) and the source code is available at https://gitlab.com/biomedit/sett-rs

sett is developed as part of the BioMedIT project.

1.13 - Benchmarks

sett performance benchmarks

Test setup

The benchmarks were run on an Apple Silicon M3 machine with 16-Core CPUs, 128GB memory, and 2TB disk. Version 5.2.0 of sett CLI was used.

One single binary file was used, with default compression (Zstandard, level 3). Decreasing the compression level and/or using text data will influence the results.

Transfer speed is highly dependent on the infrastructure connecting the data sender and recipients. In these benchmarks, data was transferred to the same machine sending the data, therefore representing a best-case scenario in terms of transfer speed.

Results

The table and figure below present throughput speeds for different sett workflows, all values are averages of 5 runs performed with hyperfine:

• Encrypt: compress, encrypt package and transfer (S3, SFTP) data.
• Transfer: transfer already packaged data. There is no compression and encryption involved in this workflow.
• Decrypt: decrypt and decompress a data package, either locally or streamed from S3.

Throughput as function of file size: as can be seen, the curves in the figure below are linear, indicating that the throughput is constant regardless of the file size.

2 - BioMedIT Portal User Guide

In the following sections we introduce the BioMedIT portal, how it works and how to get started using it.
Please use the table of contents menu to the left or below to navigate the topics.

What is the BioMedIT Portal?

The BioMedIT Portal is the one-stop entry to the BioMedIT Network. It provides users with a single point of access to the tools and services of the BioMedIT Network in a secure web-based environment.

With the Portal:

• Project Leads have a consolidated view of their projects and resources, enabling them to manage the research team’s members and access the project space.
• Data Providers and Data Managers oversee data transfers and monitor them.
• Researchers gain access to an expanding collection of collaborative and research-oriented tools, which encompass in-house, commercial, and open-source resources.

Access is secured with a login with your SWITCH edu-ID account and two-factor authentication.

Table of Contents

2.1 - Login

To either register for the first time or login to the BioMedIT portal:

1. Go to: https://portal.dcc.sib.swiss/. The SWITCH edu-ID authentication button will appear.
2. Click on > SWITCH edu-ID.
3. Enter your email address and click on Continue. You will be asked to enter your password.
4. Your second authentication method prompt will appear (SMS, OTP). Enter your Code and click on Login.
5. You will be asked to:
   1. Enter a username of your choice.
   2. Choose which of the emails associated to your SWITCH edu-ID account to use for the BioMedIT Portal.
   3. Agree with the Portal being notified when your affiliation changes.
   4. Agree with the BioMedIT Portal Private Policy.
6. The BioMedIT Portal Home will be displayed.

2.2 - Home

As soon as you log in with your SWITCH edu-ID, the home page displays the menu bar of the left side, the Quick Access Panel, with short-cuts to central services in the middle, and the BioMedIT Feed on the right, where you may find news and announcements from the BioMedIT Team.

The Portal classifies user accounts by role, and the menu options visible for each user will depend on the role assigned. A Data Provider, a Project Leader, or a visitor without a specific role will all see different default menu options

Quick Access links

• SPHN Schema Forge
• SPHN DCC Confluence
• Terminology Service
• GitLab
• Harbor

2.3 - Profile

From the profile menu option, users can check and update their account details, and manage their OpenPGP Keys and SSH Keys.

My Profile tab

The My Profile tab shows the user’s account details.

Account details:

• Name: User’s name

• Username: User SWITCH edu-ID identifier

• Local Username: The username the user chose during their first login to Portal.

• Email: Contact email currently selected from the list retrieved from the user’s SWITCH edu-ID

• Affiliations: List of organizations (e.g., university, research institute) linked to users SWITCH edu-ID account

• IP Address: The IP address from where the user is connecting

• Flags: List of flags enabled for the user

• Groups: The list of Groups the user is a member of.

Note that personal details are retrieved from the user’s SWITCH edu-ID. For any changes to their personal data or affiliated organizations, user may update their SWITCH edu-ID via this link.

For more resources about SWITCH edu-ID, refer to SWITCH edu-ID / Quick Links.

OpenPGP Keys

From the OpenPGP Keys tab, users can see the PGP Keys they have registered in the Portal, along with their status.

Each key has the following information:

• Fingerprint: This is a unique identified for each PGP key. You should make sure sure that this value is matching with the fingerprint of your key.

• User ID: User ID associated with the PGP key. People usually have their full name (and sometimes affiliation) in this field.

• Email: Email address associated with the PGP key

• Status: The approval status of your key. The list of key statuses is as follows:

  • PENDING: a key approval request was submitted, but the key has not been approved yet. This is a manual process and can take from a few hours or up couple of days.
  • APPROVED: key is approved for usage within the BioMedIT network. Only approved keys can be used to encrypt, sign and decrypt data packages that are transiting via the BioMedIT network.
  • APPROVAL-REVOKED: approval of the key has been revoked by the BioMedIT key validation authority.
  • KEY-REVOKED: key has been revoked by its owner.
  • REJECTED: key is not trusted by the BioMedIT key validation authority.
  • DELETED: key has been removed from the keyserver by its owner.
  • UNKNOWN KEY: key has not been registered on the BioMedIT portal. If it is your own key, please register it. If it is the key of someone else, please ask them to register their key.

Registering a new PGP key

Once the above prerequisites are met, perform the following steps to register your key and request its approval:

1. Connect to the BioMedIT portal and go to Profile and OpenPGP Keys tab.

2. Click on +OPENPGP KEY.

3. Add Key dialogue box will open. Enter your key’s fingerprint: please copy-paste it from your computer to avoid typing errors. Then click on the green search icon.

4. If your key is present on the Open PGP keyserver, the user ID and email associated with your key will be retrieved and displayed in the dialog box. Please double check that the value is correct.

5. If the displayed user ID and email values are correct, click on CONFIRM.

Your key is now registered and an approval request as been automatically sent to the DCC. Please note that the verification and approval of your key by the DCC is a manual process and may take a few days.

Note that your key’s approval status will be PENDING until approved by the DCC. Once approved, the status will change to APPROVED, at which point the key can be used to encrypt and decrypt data to be transferred into the BioMedIT network.

SSH Keys

From the SSH Keys tab, users can add their public SSH key for a specific project.

Registering a new SSH key

1. Click on +SSH KEY.
2. Select the project from the drop-down list
3. Copy the content of your public ssh key
4. click on SAVE

2.4 - Groups

From the Groups menu option, Data Providers can manage users and assign them roles.

Data Provider User roles and responsibilities

The Data Provider roles are intended to provide Data Providing institutions with the ability to manage the users involved in data transfers to BioMedIT and assign permission levels according to their responsibilities.

DP Manager

The DP Manager represents the Data Provider institution within the scope of BioMedIT. A DP Manager is responsible for the institution’s internal processes related to BioMedIT and delegates responsibilities to other users in their organization by assigning/removing DP roles. In addition, a DP Manager is responsible for maintaining and keeping the roles assigned up to date, i.e., if an employee left or changed responsibilities, and informing the BioMedIT Node and the DCC accordingly. Finally, the DP Manager delegates the internal operation coordination to the DP Coordinator.

DP Managers are:

• Responsible for the ensuring data provisioning at the Data Provider institution.
• Responsible for escalations and security incidents within the Data Provider.
• Responsible to assign/revoke the DP roles to other DP users.

DP Coordinator

The DP Coordinator is responsible for internally coordinating the DP’s readiness to send data, ensuring the legal compliance, i.e., DTUA, DTPA, etc. and all technical measures required from the DP side are in place for projects in which Data Provider institution participates. The DP Coordinator must send confirmation of readiness for data transfers to the DCC when required, and act as the single point of contact for follow-up questions and/or issues in the process.

DP Coordinators are:

• Responsible for coordinating internal data transfer processes of the Data Provider institution
• Responsible for confirmation of legal readiness for data transfers
• Responsible for confirmation of technical readiness for data transfers

DP Technical Admin

The DP Technical Admin is responsible for all technical tasks required for the setup and maintenance of a secure connection from the DP organization to the BioMedIT Node e.g., network components configuration. He is also responsible to support DP Data Engineer in the onboarding process. There could be more than one DP Technical Admin.

DP Technical Admins are:

• Responsible for the configuration of the network components from the DP side to permit data transfers.
• Responsible for supporting the onboarding of the DP Data Engineer(s).

DP Data Engineer

The DP Data Engineer is responsible to prepare, encrypt and sign the data packages, and transfer them from the DP organization to the assigned BioMedIT Node. There could be more than one DP Data Engineer.

DP Data Engineers are:

• Responsible for preparing and executing the data package transfers between the Data Provider and the assigned BioMedIT Node, following the BioMedIT process for secure data transfer.

DP Security Officer

The DP Security Officer is the designated point of contact from the DP organization to receive incident and security notifications from BioMedIT such as (but not limited to) scheduled and non-scheduled downtimes in response to security incidents, emergency changes and systems upgrades.

DP Security Officers are:

• Responsible for serving as a point of contact for incident, maintenance and security notifications from BioMedIT, and distributing them internally.

DP Viewer

A DP Viewer can monitor the status of data transfers from their DP institution.

How to add a user to a Data Provider group

To add a user to a group:

1. Click on the edit button of the group.
2. In the Users field, type the user’s email address and click on +USER.
3. When no more users need to be added to the group, click on SAVE.

2.5 - Projects

In the Projects menu option:

• Project users find a consolidated view of their projects, the data transfers, and the tools and resources enabled for the project.
• Project Leaders and Permission Managers can manage project users.
• Data Managers of a project can submit a data transfer request.

Projects view

From the project’s menu option, users can see the list of their projects.

The projects displayed can be filtered by using the Search function.

By clicking over a project, users can display the project’s details:

• Project Details tab
• Data Transfers tab
• Resources tab
• Services tab
• Users tab
• Additional information tab

Project Details tab

The details tab shows the project’s general settings:

• Code: A unique identifier for the project

• Hosting Node: Name and code of the node where the project is hosted

• Archived: Yes / No

• Expiration Date: Date when the project expires.

• Legal Approval Group: Group responsible to confirm compliance with legal basis and approve production DTRs.

• Data Specification Approval Group: Group responsible to review data transfer’s Data Specifications and approve production DTRs.

• Legal Support Contact: Contact email address to engage support about ethical and legal questions.

• Enable SSH Access: Indicates if SSH access has been enabled for the project.

• Enable Resources: Indicates if resources have been enabled for the project. The resources tab will be shown/hidden accordingly.

• Enable Services: Indicates if services have been enabled for the project. The services tab will be shown/hidden accordingly.

• Identity Provider: Source user identity provider specified for the project. By default is SWITCH edu-ID, but a node-specific pre-defined identity providers may be selected.

• IP Ranges: IP range(s) from where client connections to access B-spaces is permitted.
  If the user is connecting from an IP outside of the IP range, the message “Your IP address is not within the valid IP address ranges of the project” is displayed.

Data Transfers tab

The data transfers tab displays the list of data transfer requests submitted by the project.

• From the Search box, users can search for specific data transfer requests by any content as search criteria.

• +DATA TRANSFER: Function to submit a new Data Transfer Request. Note that this option is only visible to the project’s Data Manager(s). The data transfer request list is also accessible from the Data Transfers menu option.

Users tab

Displays the list of users and their role in the project.

Authorized flag:

This badge is displayed next to a project user when they have completed the mandatory BioMedIT training modules: Information Security Awareness Training

Terms Of Use Accepted: 2.3

This version number represents the last Terms Of Use the user accepted.

Additional information tab

Notes, comments and relevant information details. Note that this field is only editable by admins (Node Managers and BioMedIT Support).

Resources tab

Displays the access links to the node-specific tools and applications configured for the project. It offers these fields:

• Name
• Location - a valid URL that points to the resource
• Contact - a valid email address whom to contact
• Description - a simple textfield

These resources are defined per project only. Because of this limitation, we added the services tab (see below) which offer more flexibility.

Services tab

Similar to the resources tab but more flexible. Displays the node-specific tools assigned to a given project as well as the tools assigned to individual members of that project. The aim is that Portal serves as a central point for node-specific information. The data field (see example below) allows the use of markdown language which will be automatically nicely rendered as rich text in Portal.

To create a new entry, node admins simply need to issue a POST request:

POST /backend/projects/<project_pk>/services/

{
    "name": "Special Tools",
    "description": "A collection of special tools for a project",
    "data": "# Title\n*   feat 1 explained\n*   feat 2 explained",  # <- Markdown language
    "state": "CREATED" # valid states: INITIAL, PROGRESS, CREATED, LOCKED, ARCHIVED
}

Where the project_pk is the primary key of a given project (not the project code). Node admins and -viewers will then see the service(s) nicely rendered in the services tab.

To view, update or delete a project service entry, use the

/backend/projects/<project_pk>/services/<service_pk>/

endpoint and the GET, PUT or DELETE http verbs. service_pk is the automatically assigned primary key of a service.

In addition, services can also be assigned to project members, by sending a POST request to this endpoint:

POST /backend/projects/<project_pk>/members/<user_pk>/services

{
    "name": "Special Tools",
    "description": "A collection of special tools for a project",
    "data": "# Title\n*   feat 1 explained\n*   feat 2 explained",
    "state": "CREATED" # valid states: INITIAL, PROGRESS, CREATED, LOCKED, ARCHIVED
}

Where the project_pk is the primary key of a given project (not the project code) and user_pk the primary key of the user. If the user is not member of the specified project, you’ll receive a 404 error.

Once a project service is assigned to a project member, users then are able to see the current state of their assigned project services in the services tab.

To view, update or delete a project service entry, use the

/backend/projects/<project_pk>/members/<user_pk>/services/<service_pk>/

endpoint and the GET, PUT or DELETE http verbs. service_pk is the automatically assigned primary key of a service.

Note 1: these services serve information purposes only, have no implicit functionality. In particular, they do not trigger any mails being sent to the user.

Note 2: the service name must be unique and a service can only be assigned once to a project or project member. The service name can be freely chosen.

Note 3: currently the action field has no meaning or implicit functionality.

Note 4: there is no GUI planned to edit service information data.

User roles

User roles are set for each project. Each authorized BioMedIT user can have several user roles and can be assigned access to several projects.

Roles and responsibilities

Project Leaders (PL):

• responsible and accountable for the project
• responsible for the data lifecycle management of the project
• acts as contact point for escalations and security incidents within the project
• responsible to discuss the setup and configuration of the project space for the project with representatives of the BioMedIT Nodes
• responsible for case-by-case authorization of Data Transfers out of the B-space
• assigned to the project in the User Role of “Permissions Manager” by default and with that is responsible to assign authorized BioMedIT users
• responsible to assign the User Role of “Permissions Manager” to authorized BioMedIT users
• responsible to ensure that at least one authorized BioMedIT user was assigned to the project space in the User Role of “Data Manager”
• can act as “Permissions Manager” and/or “Data Manager” and/or “(default) User”
• responsible to revoke access rights to the project space
• informed about every user change (add/remove) via email notification
• there can be multiple PLs per project

Permissions Manager (PM):

• accountable to the PL
• each project must have at least one project member with the User Role of “Permissions Manager”
• responsible for assigning the User Roles of “(default) User” and “Data Manager” to authorized BioMedIT users and with that granting them access to the project space
• responsible to revoke access rights to the project space
• acts as contact for the new project members to support them with registration and authorization as users of BioMedIT

Data Manager (DM):

• accountable to the PL
• each project must have at least one project member with the User Role “Data Manager”
• responsible for unpacking and decryption of data packages within the B-space
• responsible to extract data from the B-space for transfer out of the project space only following explicit authorization of the PL
• responsible to place the request(s) for Data Transfer(s) from a Data Provider to the project

User:

• accountable to the PL
• has access to one or several project spaces following authorization
• responsible for data processing within the B-space

No Role:

• accountable to the PL
• can access the project’s details in the Portal, but has no access to the B-space.
• mutually exclusive with other roles

Managing project users

When clicking on the Edit project button the top-right corner, you will be taken to edit page.

Adding users to a project

1. In the Users field, enter the user’s email address and click on +USER. Note that the user(s) must have an account in the portal.
2. You will be asked to confirm the user’s details before adding it to the project.
3. Review the details of the user, and click on ADD to confirm it.
4. You will return to the list of project users. Assign a role to the user by clicking on the corresponding checkbox(es).
5. Click on SAVE.

Removing users from a project

To remove users from the project, the Project Leader or the Permissions Manager should:

• Click on the 'x' next to the user’s name.
• After any change, click on SAVE

Submitting a Data Transfer Request

To submit a Data Transfer Request (DTR), click on +Data transfer. The data transfer creation form will be displayed.

Complete the required fields:

• Unlimited / One Package: Select if the DTR will cover one or multiple data transfers

• Data Provider: Select the Data Provider from the drop-down list

• TEST / PRODUCTION: Select TEST if mock data will be sent, or PRODUCTION if the data will contain patient real data

• Additional Data Recipients: Enter the email(s) of the data recipient(s) and click on the +USER button.
  Note that recipients can only be users with the Data Manager role in the project.

• Legal Basis: If transferring real data (purpose PRODUCTION), enter the legal agreement (i.e. DTUA, DTPA) document name

• Data Specification: If transferring real data (purpose PRODUCTION), include the link to the Data Specification
  

  Example: https://git.dcc.sib.swiss/admin/projects/project-space/example-project/data-transfer/-/tree/main/data-transfer-1?ref_type=heads

Click on SAVE.

A new DTR will be created in INITIAL status. Confirmation emails with the DTR’s details are sent to the DTR requester, recipient(s), and approvers.

DTR Approvals

When the DTR is created, the following approval requests are submitted in parallel:

1. Node(s) Approval: As data processors, the involved node(s) must confirm the presence of a pre-existing legal basis for the data transfer and ensure that the necessary technical infrastructure is in place to support it.

2. Legal Compliance Approval: For SPHN/NDS projects, facilitated by the DCC ELSI Help Desk group who verifies the existence of a legal basis (e.g., DTUA, Consortium agreement) for the data transfer. For non-SPHN projects, a dedicated local Legal Compliance group may be optionally configured.

3. Data Specification Approval: For SPHN projects, the DCC will review and approve the data specification documents referenced in the ‘Data Specification’ link. This process ensures the compliance with the SPHN Semantic Interoperability Framework.

4. Data Provider Coordinator Approval: This approval request is triggered only when the previous approvers, (1),(2) and (3) have submitted theirs.

The Data Provider Coordinator is then requested to confirm if the data delivery has been approved through their internal governance processes.

When all approvers approve the request, the DTR status changes to AUTHORIZED, and data can then be transferred. Notification emails are sent to all parties involved in the data transfer:

If any of them rejects the DTR, its status is set to UNAUTHORIZED.

Monitoring the approval status of a Data Transfer Request

The status of a DTR is displayed in the last column of the DTR list.

Approval Status

• INITIAL: The DTR was submitted, but it has not yet been approved.
• AUTHORIZED: All approvers have authorized the DTR. The Data Provider can now send the data.
• EXPIRED: The Data Provider sent the maximum allowed number of data packages, and no additional data can be sent under this DTR ID
• UNAUTHORIZED: The DTR was previously authorized but is currently unauthorized for some reason (i.e., The BioMedIT Node is offline, problems with user permissions, issues on the Data Provider's end, etc.)|

Data Transfer details

When clicking on a particular data transfer, a pop-up window shows additional information

• Details tab
• Logs tab
• Approvals tab

Details tab

Displays the attribute values of the DTR as submitted by the project’s data manager.

Logs tab

Displays the individual data packages transfer logs - routing information and whether the data has arrived to the project workspace.

Approvals tab

Displays the current approval status of a DTR by all approval groups.

2.6 - Data Transfers

In the Data Transfers menu option:

• Data Managers and Data Providers can search and view the list of submitted Data Transfers Requests (DTRs), monitor their authorization status and transfer logs
• Data Providers (DP Data Engineers) can fetch their credentials to transfer data to a research project.
• Data Provider Coordinators can approve or reject a DTR

DTR Approval status

The approval status of a DTR is displayed in the last column of the table:

• INITIAL: The DTR was submitted, but it has not yet been approved by the Data Provider and/or the BioMedIT node(s).
• AUTHORIZED: All approval groups have authorized the DTR.  The Data Provider can now send the data.
• EXPIRED: The Data Provider sent the maximum allowed number of data packages, and no additional data can be sent under this DTR ID
• UNAUTHORIZED: The DTR was previously authorized but is currently unauthorized for some reason (i.e., The BioMedIT Node is offline, problems with user permissions, issues on the Data Provider's end, etc.)

Additional details about which group approved when are shown when clicking on a specific DTR. See approval details

Data Transfer details

When clicking on a particular DTR, a pop-up window shows additional information

• Details tab
• Approvals tab
• Credentials tab
• Logs tab

DTR’s details are accessible using the following link structure: https://portal.dcc.sib.swiss/data-transfers/dtr_id

Details tab

Displays the attribute values of the DTR, as submitted by the project’s data manager:

• Project: Name of the project
• Transfer ID: Unique DTR identifier
• Data Provider: the Data Provider code
• Status: Approval status of the DTR
• Max Number of packages: 1 (single transfer) or UNLIMITED (multiple data transfers)
• Transferred Packages: Count of packages sent so far under this DTR ID
• Requestor: Name of the Data Manager that submitted the DTR
• Data Specification: link to the Data Specification
• Purpose: TEST (for mock data) / PRODUCTION (for patient real data)
• Legal Basis: Legal agreement (i.e. DTUA, DTPA) document name

Approvals tab

Displays the current approval status of a DTR.

Credentials tab

From this option, DP Data Engineers can fetch their credentials, required for authentication and authorization to transfer data when the transfer method is S3/HTTPs.

To fetch and copy their credentials, as DP Data Engineer:

• click on FETCH CREDENTIALS
• click on COPY ALL

Logs tab

Displays the individual data packages transfer logs - routing information and whether the data has arrived to the project workspace.

Approving DTRs

When the DTR is submitted by a project’s Data Manager, the groups involved in the data transfer will receive a notification by email with all the details, for their information or for their action (request for approval), depending on the case, as explained below.

Approvals

When a DTR is submitted by a Data Manager to a research project, the following approvals must be collected:

1. Node(s) Approval: As data processors, the involved node(s) must confirm the presence of a pre-existing legal basis for the data transfer and ensure that the necessary technical infrastructure is in place to support it.

2. Legal Compliance Approval: For SPHN/NDS projects, facilitated by the DCC ELSI Help Desk group who verifies the existence of a legal basis (e.g., DTUA, Consortium agreement) for the data transfer. For non-SPHN projects, a dedicated local Legal Compliance group may be optionally configured.

3. DCC Data Specification Approval: For SPHN projects, the DCC will review and approve the data specification documents referenced in the ‘Data Specification’ link. This process ensures the compliance with the SPHN Semantic Interoperability Framework.

Only when the above groups, (1), (2) and (3) confirm their approval, the DP Coordinator is requested to submit theirs:

By clicking on the link in the email above (https://portal.dcc.sib.swiss/data-transfers/<DTR_ID>), DP Coordinators will be redirected to the BioMedIT Portal to approve or rejects the request.

To approve the DTR, click on APPROVE and declare, by clicking on the checkboxes, if:

• All technical measures are in place to send the data
• There is an existing legal basis for the Data Transfer
• This data delivery has been approved through internal governance processes

Then, click on APPROVE.

To reject the DTR, click on REJECT and provide a reason for the rejecting. Then, click on REJECT.

When all approvers approve the request, the DTR status changes to AUTHORIZED, and data can then be transferred. Notification emails are sent to all parties involved in the data transfer:

If any of them rejects the DTR, its status is set to UNAUTHORIZED.

Download data transfer list

By clicking on the download icon at the top-right corner, users can download the list of DTRs in csv format.

2.7 - Terms Of Use

In the Terms Of Use menu option:

• each node has the opportunity to specify its own terms of use (TOU, a.k.a. acceptable use policy)
• each TOU has a version number
• each TOU has a version type, which is either MINOR or MAJOR
• every user that uses the services of a node must read and accept the terms of use

Terms Of Use overview

Node and portal admins

Adding new terms of use

The overview table contains a tab for every node. In every tab, a node admin can add a new Terms Of Use policy by clicking the + TERMS OF USE button. The node admin then needs to specify the node for which this TOU is valid, a version number (we recommend major.minor, e.g. 2.3) and a version type (MINOR or MAJOR) which indicates the change compared to the previous document. The actual text for the TOU must be written in Markdown format, which then will be rendered on screen. Once entered, the text cannot be altered anymore. Therefore make sure you create and review the text before you publish it here.

When a new version of the Terms Of Use is created, all the users of this node get informed by mail, indicating that a new Terms Of Use is available which they need to accept soon in order to continue using the service.

Currently there is no further mechanism implemented which would block a user, for example.

Read terms of use

By clicking on a single TOU, a new screen is shown where the details of the TOU are presented. An additional users tab shows the list of users who have already read and accepted this version of TOU.

Download TOU list

A convenient way to get a complete overview is to click on the blue download button on the TOU overview. This provides a CSV listing all nodes, all versions and all users that accepted it, including the date it has been accepted.

Project users

Every project is assigned to a specific node, and every node has its own Terms Of Use. Users of any given project need to read and accept the TOU once per node.

Read and accept terms of use

In the overview table, all users see a READ AND ACCEPT button for the Terms Of Use they have not accepted yet. The button takes them to the read and accept page, where they see the rendered Terms Of Use page(s) plus a checkbox:

[x] I have read and understood the node’s terms of use,

followed by a CONFIRM button. Once a user confirms, the node admin gets notified by email. You cannot reject the Terms Of Use once you accepted. Once you accepted, the latest version you accepted will appear in the Users tab of the Projects overview table.

By clicking on a single TOU, a new screen is shown where the details of the TOU are presented.

2.8 - Contact Us

It is the contact form to submit feature requests or support to the BioMedIT Help Desk (biomedit@sib.swiss).

2.9 - Central Services

2.9.1 - DCC GIT

1. Login to GitLab

1.1 GitLab is available via federated login from BioMedIT portal’s Quick Access link:

1.2 The following page will be displayed.

Click on Federated Login

2. Resources

• Get started with GitLab
• GitLab Learn → Watch videos and self-driven demos
• GitLab docs → Access step-by-step tutorials and guides

2.9.2 - Harbor

1. How to request access

1.1 Registry is available via federated login from the BioMedIT portal’s Quick Access links:

1.2 Click on the button: Login via OIDC provider

2. Resources

• Working with projects
• Project configuration
• Working with images
• Using the API explorer

3 - Quick links for SWITCH edu-ID

Quick Links

• Create account: how to create a new SWITCH edu-ID account

• View and modify account data: how to check, add, delete or change account data.

• Reset password: how to set a new password if you have lost or forgotten it.

• Change password: how to set another password.

• FAQs - SWITCH edu-ID - SWITCH Help

4 - SPHN-DCC Confluence

How to request access

To access SPHN-DCC Confluence, users must create an Atlassian account. Here are steps:

Access the link https://sphn-dcc.atlassian.net. The following page will display

Select the option Create an account

The following page will display.

Enter your email address and click on Sign up

Confluence will send you an email with a verification link:

Check your mailbox and open the email sent by Confluence. Open the message and click on Verify your email

After verifying your email, Confluence will redirect you to the following screen to finish setting up your account:

Enter your full name
Create a password
Click on Continue

The SPHN-DCC welcome space should display.

Note that to gain access to specific spaces, you must contact the space owner to assign you permissions.

Guides for users

• Manage your account
• Confluence Essentials

Guides for space owners

• Organize and customize your Confluence space
• Set up & manage space permissions