OTA Bundle Signing in Production with Thistle Release Helper

Thistle's OTA Update System

Thistle's Over-the-air (OTA) update system provides end-to-end security assurance, because:

• The OTA firmware bundle is digitally signed by a device maker, and verified on a device, using public-key signature systems.

• The security of the backend infrastructure where the OTA bundles are released is independent of device security: even if the server is compromised, and the released firmware is tampered with, without a valid signature the OTA bundle will be rejected by the Thistle Update Client (TUC).

The Thistle Release Helper (TRH) is a command-line tool Thistle offers to facilitate the firmware release operations for a device maker. The TRH helps a release operator prepare, sign and publish OTA bundles with a few simple commands.

OTA Bundle Signing with TRH

By default, the TRH uses Minisign, which uses the Ed25519 signature system, to sign the OTA bundle; the private Minisign signing key is loaded from a password-protected file stored on a signing machine into TRH's memory during signing.

While storing a private key in a computer's filesystem is simple for development and testing, there are many reasons it may not meet production requirements. More specifically,

• The private key file can be easily copied, and subsequently leaked, by an entity, e.g., careless or malicious personnel, a backup program, or malware running on the computer.

• Even if the private key is password protected, the password can be brute-forced offline.

• It isn't a straightforward task to securely back up a private key file. On the other hand, not backing up a private key file used for production signing can easily be a production continuity risk - imagine the consequences of a disk failure.

It's not impossible, but usually difficult and costly to create and maintain a release process that can mitigate the above risks associated with private key management. Hence, a proper key management system is desired for OTA bundle signing.

Signing in Production

Thistle's OTA update platform seamlessly supports two key management systems, with private signing keys backed by YubiKeys and Cloud Key Management Services (KMS). Thistle provides signing infrastructures, trh-y and trh-k, respectively, to facilitate these two key management cases for strong private key protection, while ensuring the device maker's complete ownership of their signing keys.

• trh-y: Thistle Release Helper with YubiKey Signing

  • It's a low-cost, high-security-assurance solution. Only off-the-shelf YubiKeys are needed for key management.

  • It leverages a YubiKey signing tool called pivit for OTA bundle signing

  • It uses the ECDSA P-384 signature scheme and PKCS#7 signature format

  • Key pairs are generated inside YubiKey hardware. Private keys cannot be exported: private keys are never loaded into TRH memory; no one (be a human being or a software program) has access to the private keys.

  • Because YubiKeys as hardware devices are subject to eventual failures, the user needs to come up with a key rotation plan and ensure sufficient key redundancy.

  • Instructions for using trh-y can be found in trh-y: Thistle Release Helper with YubiKey Signing.

• trh-k: Thistle Release Helper with KMS Signing

  • It's a medium-cost, high-security-assurance solution. The user needs to have a Cloud account with the KMS feature enabled.

  • It currently supports GCP KMS. We will add support for other Cloud providers, e.g., AWS KMS in the near future. Stay tuned!

  • It uses the ECDSA P-256 signature scheme.

  • Key pairs are generated inside the Cloud KMS. Private keys cannot be exported: private keys are never loaded into TRH memory; no one (be a human being or a software program) has access to the private keys.

  • Cloud KMS is reliable and highly available (cf. this reference about AWS KMS). Key availability thus reduces to account availability.

  • Instructions for using trh-k can be found in trh-k: Thistle Release Helper with KMS Signing.

Signature Verification in TUC

The Thistle Update Client supports signatures produced by Minisign, trh-y, and trh-k. In TUC's configuration file, multiple public keys/certificates can be specified in the public_keys array as verification keys to provide redundancy on the client side. Moreover, public_keys can contain public keys/certificates corresponding to different signature schemes to maximize interoperability. For example, one can put two YubiKey-generated public keys and one GCP KMS-generated public key in the public_keys array in TUC's configuration file, using the two YubiKey-backed keys as the primary signers for regular signing, and the GCP KMS-backed key for emergency signing.