Obnam → blog → 2026 → Obnam: terminology, server persistence

Goal

Today's goal is to make chunk storage persistent in the server.

At the moment chunks are stored in memory in the server. This was easy to do and is sufficient for light testing. It's time to make things persistent. I already have code in the client for this, and I can reuse that in the server.

Plan

  • Make terminology consistent: refer to where chunks are stored as "the backup repository" instead of "chunk store".

  • Change the server to use code from the client for the backup repository.

  • Add verification scenario for server persistence.

Notes

  • Lots of tedious grepping and renaming to make sure "backup repository" or just "repository" is uses consistently and "chunk store" or "store" is no longer used at all as a synonym. Could not automate this with sed, because each place where "store" is used needed review to make sure it needed changing. For example, "Store chunk in repository" is OK.

  • To make the backup repository in the server persistent, I'm going to reuse code from the client. In the long run, I'm sure both the client and server need to access backup repositories stored on local disk. It make sense for them to share the code.

  • I may want to refactor my Obnam crates so that the server doesn't depend on the client, but that's a decision I'm not ready to do yet. I'll want to mull over options before I choose one, and today is a bad day for that.

  • So I'll make the server crate depend on the client crate, for now. Adding the dependency is easy enough. However, using the client repository code needs a little bit of thinking.

  • Initialing the server currently only creates a key pair for creating and validating API tokens. It needs to also create the directory for the backup repository. That repository then needs to be opened so that HTTP API calls can use it.

  • Opening the repository takes a few steps: open an SQLite database and create tables, if necessary. This might be too heavy to do on every HTTP API call, but I won't care about that now. If it's not a problem, there's no point in spending effort on it now, and if it is a problem, I'll need to make a benchmark to make sure I've solved the problem. That's too much work today. The goal of today is to achieve persistence, but performance is not part of that.

  • So for now, I'll open the backup repository on every API call.

  • Bummer. That won't work as-is, either. The client backup repository code wants to deal in chunks, constructed by encrypting plaintext data. There's no way to construct Chunk value from already encrypted data (plus associated metadata).

  • Or maybe there is. Of course there is, as the client also needs to do that. There is Chunk::parse. This does not require decryption, so it's perfect.

  • I can avoid all of that, however, by changing obnam::repository::BackupRepository to handle chunks as blobs, with metadata provided separately. I'll do that, but today I ran out of time, so I'll do that next time.

Summary

I made terminology more consistent and tried to make backup storage persistent in the server, but ran out of time.

Comments?

If you have feedback on this, please use the following fediverse thread: https://toot.liw.fi/@liw/116430436359061282.

If you'd like to fund Obnam development, see my funding page.