Porting sett from Python to Rust

Tags:
Categories:

sett stands for “Secure Encryption and Transfer Tool” and is an application that facilitates and automates data packaging, encryption, and transfer in BioMedIT.

Over the past two years BIWG has ported sett from Python to a more modern technology stack, composed of Rust and Svelte. This port was performed because the Python implementation was showing some design limitations, which made the life of developers and users difficult. The list of limitations included:

  • A somewhat complicated installation process, requiring multiple steps and the usage of the command line;
  • A requirement on a local installation of gpg: a tool which is very problematic to handle programmatically, especially on some operating systems;
  • Python’s performances: good, but definitely a bottleneck when compared to the performance of a compiled language like Rust can provide;
  • A Python-based GUI where implementing complex behaviors and styling can be challenging.

Before diving deeper, it’s worth mentioning that the transition was carried out gradually. The Rust codebase was initially designed as a Rust library with Python bindings, which allowed BIWG to gradually replace Python core features—i.e. encryption, transfer, decryption, key management—with their Rust equivalents. These Rust-based core features have been running in production since months, nicely encapsulated in a Python-based shell. The last steps of this port were carried out more recently, when:

  • On 22nd April 2024, the first Rust-only version was released both for the GUI and the CLI;
  • On 30th June 2024, the Python version was discontinued.

sett-gui-architecture

Usability improvements

  • Simplified installation process. Both the CLI and the GUI can be downloaded as standalone executables, with no dependency on external software;
  • More modern looking GUI, with step-by-step user guidance (i.e. fields to fill are shown to the user as needed), and simplified CLI interface (i.e. less options);
  • sett no longer relies solely on the local gpg keyring but has now an internal key store, implemented using Sequoia PGP—a utility to migrate keys from the gpg keyring to the internal key store is provided by the tool;
  • Support for parallel compression and encryption;
  • Support for simultaneous encryption and transfer;
  • Support for simultaneous decryption and transfer;
  • Use of Zstandard as default compression algorithm. Faster than the previously used gzip algorithm, and multi-threaded;
  • Authentication with BioMedIT Portal, enabling an improved user experience—e.g. auto-retrieval of S3 credentials (no need to go to portal and copy/paste credentials anymore);
  • Auto-updater for the GUI application;
  • Code signing certificates, provided by:
    • Apple, for Mac binaries;
    • SSL.com, for Windows binaries.

Performance improvements

Performances are noticeably improved, according to BIWG’s investigation and as shown in the table that follows.

Legacy sett sett 5.2.0
Encryption 50 MB/s 249 MB/s
Decryption 122 MB/s 259 MB/s

Discontinued features

  • Support for Liquidfiles as a protocol for transferring data packages;
  • Support for persisted settings/configuration for the CLI;
  • Support for SSH jump host for transfer over SFTP.

What’s coming next ?

  • An interactive terminal user interface (TUI), alternative to the classical way of running sett CLI and especially useful in the authenticated mode;
  • Support for signing public keys, useful e.g. when the users don’t rely on a central trusted authority for key verification and approval;
  • More features based on authentication with BioMedIT Portal—e.g. better feedback to the users on the status of the BioMedIT network, improved data package tracing;
  • Support for simultaneous decryption and transfer from the GUI.

Fine details on the status of present and future developments are available in BIWG roadmap.