Goal

The previous eight weeks of developing encryption support has not resulted in success. Last week I came to the conclusion that I was building an internal API for this that was much too complicated.

• the cipher module needs to have an enum for cipher engines, and engine-independent ways to handle keys and errors, or otherwise hide the different implementations; the API caller should not need to deal with the details
• there's no real point in having two kinds of plain text encoding forms, JSON and Postcard; I'm not really ever going to use anything but the Postcard, and carrying the superfluous JSON encoding is just more work for now and makes the API harder to use
• having separate plain text and encrypted encoding methods also makes API harder to use
• having a way to produce plain text chunks isn't really useful, either; I'm never going to want that in production code

Today, I'll start over from a new branch. I'll reuse what code I can from the past eight weeks, but I'll start from a new, higher level API for encrypting chunks.

Plan

• Make a new branch off main.
• Make sure the test suite passes. If not, fix it.
• Change the API for the obnam::chunk module so it's easy to use. Hide, or throw away, the plain text chunk variants, as they should never be used outside the chunk module.
• Add a new module obnam::cipher by copying the relevant code from last week's branch.
• Implement chunk encryption with the cipher module.
• Fix up the chunk sub-commands to only do encryption, and make sure they work. Adjust acceptance scenarios to verify they work.

Notes

Start over

• Make new branch, encrypton.
• Run make to verify everything works. It does.

New obnam::chunk API

• For the new API, I'll want at minimum:
  • a type for chunk metadata: the id and the label, with additional types for those
  • a type for an encrypted chunk, with methods to create one and to decrypt the data: I'll call this just Chunk
• Started over with a new chunk module, except I'm calling it chunk2 so that I don't have to update the sub-commands that use the old module.
• Added types Id, Label, and Metadata, with some tests.

Cipher engine

• Before I add a type for the chunk, I'll want to have a cipher engine. I can re-use most of the code from last week for this, but I'll change the API.
• I'll want a generic key type, not specific to the kind of cipher engine in use. This makes it easier to deal with keys without having to special case for each kind.
• My initial new API is:

pub enum EngineKind {
    Aead,
}

pub struct Engine {
    engine: ActualEngine,
}

impl Engine {
    pub fn new(kind: EngineKind, key: Key) -> Self {
        Self {
            engine: match kind {
                EngineKind::Aead => ActualEngine::Aead(AeadEngine::new(key)),
            },
        }
    }

    pub fn encrypt(&self, data: &[u8], ad: &[u8]) -> Result<Vec<u8>, CipherError> {
        unimplemented!()
    }

    pub fn decrypt(&self, ciphertext: &[u8], ad: &[u8]) -> Result<Vec<u8>, CipherError> {
        unimplemented!()
    }
}

enum ActualEngine {
    Aead(AeadEngine),
}

struct AeadEngine {
    key: Key,
}

impl AeadEngine {
    fn new(key: Key) -> Self {
        Self { key }
    }
}

• I don't want to expose the AEAD engine. Nothing outside this module should need it. If it turns out this is wrong, I can expose it later. None of this affects the on-disk persistent form of chunks, which is the only really important thing to provide longevity for.
• Made Key generate a random key by default. There is no need for users to choose chunk encryption keys.
• Got a round trip test for AEAD encryption to work.

Back to chunks

• Added a chunk type based on my previous thinking about a nice API.
• Refactored to have a very tidy branch

Using new chunk module for subcommands.

• Change src/bin/cmd/chunk.rs to import from chunk2, the new module.
• This results in many errors, because new module doesn't even try to be backwards compatible.
• Drop subcommands encode and decode since they only work for plain text chunks. Keep inspect as I'll want to change that to work for encrypted chunks, but commented out.
• Add a new subcommand encrypt.
• Add a new subcommand decryt.
• Had forgotten that chunks, not just ciphertext, need to be serialized. Fixed that.
• Round trip works.

Bringing back inspection

• Hacked up together a poor version of chunk inspection. Can get back to this later.

Merged

I decided that I'm happy enough with this code, and merged it to main.

Summary

I got there, finally. Chunks can be encrypted and decrypted.

Phew. This has been a long process, and I'm sure I'll need to come back to this code, but it's good enough for now. I hope.

Comments?

If you have feedback on this development session, please use the following fediverse thread: https://toot.liw.fi/@liw/114488795517084418,