Obnam → blog → 2025 → Obnam 3: client key storage and management
  1. Goal
  2. Plan
  3. Notes
  4. Summary
  5. Comments?

Goal

I have implemented encrypted chunks, and chunk storage. Next I want to add storing and management of the encryption keys for chunks.

  • A client should be able to have any number of chunk keys. This opens up the possibility of sharing some of the keys with other clients, so that they can also share chunk and de-duplicate across clients.
  • Chunk keys should be randomly generated, not chosen by humans. Random keys are more secure.
  • All the chunk keys known to a client should be stored in an encrypted chunk, using a "client key". The user should only need to provide the client key. (Later, not even that, see my plans for "credential chunks".)

Plan

  • Implement a type for the client chunk.
  • Add subcommand obnam client init to create a client chunk.
  • Add subcommand obnam client list to list IDs of client chunks.
  • Add subcommand obnam client show to decrypt a client chunk and show its contents.
  • Add subcommand obnam client key generate to generate a new chunk key.
  • Change obnam chnunk subcommands to be able to use keys from the client chunk.

Notes

On terminology

  • I'm using "client" to mean "a host or device that gets backed up and the specific configuration to back up to a specific backup repository". If I have both a laptop and a desktop computer, those would be different clients, because they might contain distinct sets of data. They might also share some data, so de-duplication would be nice. However, I might also want to have each to back up to two different repositories, with different configurations. My laptop media archive might be backed up only to my desktop, but to an online service.
  • If I use an online service that I pay for, I am a client of that service, but both my laptop and desktop run backup clients. This can get confusing. To avoid confusion, I will not refer to the person who pays for the service as the client. When it's time, I'll come up with a better term for that.

Storing and finding the client chunk

  • The client chunks are stored as chunks with the label client.
  • Because chunks are never modified, when a client chunk is updated, a new chunk is created, and the IDs of all previously existing client chunks for the same client are stored in the new client chunk. Then all the old ones are deleted.
  • To find the current client chunk for a client, we cannot assume we know the ID of client chunk. We may have just experiences a catastrophe and you have nothing, except you remember the passphrase for the credential for you client.
  • The credential is for the future, for we assume we have the client key.
  • Thus, to find the current chunk for a client, we search for all client chunks (based on label), and try to decrypt all of them. From the ones we decrypt, we pick the one that is not referred to by any other client chunk.
  • There might be many client chunks we can decrypt, if deleting old ones after an update fails, for whatever reason.

Client chunk content

  • The client chunk most importantly has a set of chunk encryption keys.
  • Each key should have a nick name that the user can use to refer to the key.
  • A list of IDs of older versions of the client chunk for the same client.
  • The client chunk should also have some metadata of the client. To start with, this is a name, which will default to the system hostname.

Command line interface design

  • I will introduce a global option --client-key so the user can set that.
  • I expect to add or change the following commands:
    • obnam client init [--name] to create a new client chunk with a name but no keys
    • obnam client generate KEYNAME to generate a new client key, and store it in the client chunk with the given key name, then store the new chunk and remove any old ones
    • obnam client list lists IDs of all client chunks for the current client: the ones we can decrypt that also have the expected name; optionally, all the ones we can decrypt; optionally, show contents of all in JSON format
    • obnam client show decrypts and shows the contents of a client chunk
    • obnam chunk encrypt and obnam chunk decrypt will get a new option --key-name that identifies the key to use from the client chunk, to augment the --key option they already have
  • When showing the content of a client chunk, by default the actual key is not shown, only its name. The key can also be shown by using an option for that purpose.
  • I will extend the subplot to verify this new functionality.

Actual changes

  • First, run test suite to make sure it still passes. It does.
  • Make a branch, client-chunk.
  • Added a module for the client chunk data type.
  • Added global option --client-key.
  • I've change my mind: I'll make the user always specify the name of the client, so that I can skip looking up the hostname today. Getting the hostname is not fundamental to having client chunks.
  • The client chunk is its own data type, not a Chunk. To get a Chunk, I need to serialize the client chunk. I'll do that with postcard. Should I version the client chunk? Maybe, but I won't now, to save time today. I can always use the magic cookie for the chunk to version the client chunk, if nothing else.
  • Implemented the ClientChunk serialization into Plaintext, and back.
  • Implemented initial version of obnam client init.
  • Implemented initial version of obnam client find, but due to mishap while testing manually noticed that init can create two client chunks with the same name. Will need to fix that.

Summary

I had set a goal too ambitious for a single session, so didn't finish everything I had hoped. But I made progress and this is a good place to continue from next time.

Comments?

If you have feedback on this development session, please use the following fediverse thread: https://toot.liw.fi/@liw/114686202997824166.