Obnam prior to version 1.6.1, and Larch prior to version 1.20131130, had serious, but somewhat rare bugs that could result in an error that said "Missing node" or "KeyError". These bugs would cause a corruption in the backup repository by either causing a file to be deleted, when it shouldn't have been, or the reference count for a B-tree node to be wrong. As a result, Obnam would become unable to make a backup or, in some cases, to restore everything from a backup (some files could be restored).

The corruption is not necessarily noticeable on all backup runs. It depends on the exact files that get modified during a backup generation. Worse, obnam fsck isn't able to fix the corruption, though it would find it.

As a result, a corrupted repository may be unrecoverable. Sometimes you can recover at least some of the repository, with some loss of data:

  • If the corruption is only in a "per-client B-tree", recovery can happen by removing or renaming that B-tree, but that results in all backup generations for that client being lost. However, file content will remain in the repository, so a fresh full backup should be reasonably fast. The per-client B-tree is a directory that has a large number as its name. The directory needs to be removed manully, not using Obnam. Instead of removing it, you can also rename it.

  • If the corruption is in the chunksums or chunklist B-trees, you can delete or rename them both. The next backup run will create new ones. This will reduce the possibility of de-duplication, but otherwise everything should work.

If the corruption is any other B-tree, no recovery is possible.

When in doubt, avoid using a repository that has ever been used with Obnam prior to version 1.6.1, or Larch prior to version 1.20131130. Instead, start over with a new, empty repository.

Lars apologises for this.