Data package specification

sett compresses, encrypts, and packages files in a single .zip file whose specification is described below. Only files adhering to these specifications can be transferred or decrypted by sett, and failure to comply with the specification will generate an error.

File structure

sett .zip files have the following structure:

YYYYMMDDThhmmss.zip
├── metadata.json
├── metadata.json.sig
└── data.tar.gz.gpg
    └── data.tar.gz
        ├── content/
        |   ├── [file1]
        |   ├── [file2]
        |   └── ...
        └── checksum.sha256
metadata.json

Metadata file containing the following information:

  • transfer_id: numeric ID associated to each named data transfer. In authenticated mode, or if verify package is activated, sett checks via Portal that the sender is associated with the given data transfer and verifies the data transfer is valid and authorized.
  • sender: fingerprint of the sender's public OpenPGP key, a 40-character hexadecimal string.
  • recipients list of fingerprints of the recipients' public OpenPGP keys, i.e. a list of 40-character hexadecimal strings.
  • timestamp: point in time when a metadata file was generated. Uses the RFC 3339 format, e.g. 2024-09-03T14:06:32.675879Z
  • checksum: calculated value used to verify the integrity of the data. sett uses a cryptographically safe hash algorithm which allows the receiver of the package to verify the integrity of its content before starting to decrypt and unpack it. The checksum is calculated on the compressed and encrypted data.
  • checksum_algorithm: algorithm used to compute checksum. Currently SHA256 is used, where SHA stands for Secure Hash Algorithm 256 bits (64 characters). The checksum of any data package – regardless of its size – is therefore always 64 characters.
  • compression_algorithm: algorithm used to compress the data before encryption in order to decrease a data package’s size and ultimately speed up the transfer time. sett uses zstandard by default, which is currently the fastest compression algorithm available. Other options are gzip and stored (no compression).
  • purpose (optional): intended use for the data package. Values can be PRODUCTION, TEST, or null (optional).
  • version: version of the sett package specifications (not to be confused with the version of sett itself). The sett application is designed to be backwards compatible with older sett packages.
  • extra (optional): additional, custom information regarding this data package. Must be given in a key-value format (see example below). Both keys and values are strings. To be able to add additional information to a sett package, you need to [x] Enable extra metadata in the sett settings before encrypting data.
metadata.json.sig

Detached PGP signature for the metadata.json file.

data.tar.gz.gpg

A tarball encrypted using the receiver's public PGP key and signed with the sender's private key. It is compressed with compression_algorithm.

[file1], [file2]

One or several data files can be transferred. Data to be sent can be in any format, e.g. .txt, .csv, .dat.

checksum.sha256

sha256 checksum file of all files present in data.tar.gz. This is used to make sure nothing was corrupted during the encryption/transfer/decryption process.

File example

Examples of the content and structure of the metadata and checksum files.

metadata.json

{
  "transfer_id": 42,
  "sender": "AAABFBC698539AB6CE60BDBE8220117C2F906548",
  "recipients": ["D99AD936FC83C9BABDE7C33E1CF8C1A2076818C3"],
  "timestamp": "2020-01-29T15:31:42+0100",
  "checksum": "a8eb0ee5a6a53326b1c6f9bf94136da5d98a1dc6dceee21d62f694d71c4cf184",
  "checksum_algorithm": "SHA256",
  "compression_algorithm": "gzip",
  "purpose": "PRODUCTION",
  "version": "0.7",
  "extra": {
    "key1": "value1",
    "key2": "value2"
  }
}

checksum.sha256

41421f0c4a5353a5a0cdd37de3fd80a840a190ca997ad8044a67c4c1683f7b63 file1.csv
35ba157ed1c3269d731a438c466790a4f481bb49805e2d1f380df0c636792ff6 folder1/file.txt
fd9ebdbcc1a5fc35ded6e78a6b16ef658502c9d0b05dd4a2185d0f94ccf165cf folder1/folder2/file.txt