Expanded backup functionality

[hestia_hacker]

I wasn’t able to find a thread about the backup / restore process other than the small references in Documentation initiative I know it was something @rlkoshak was looking into and I wanted to start this thread to discuss what the requirements should be for this feature. The first objective of this thread is to get agreement that this is actually the feature we want.

I’d like to see a backup system which backs up and can restore an entire device (WiFi settings, static IP address, configuration files outside of OpenHAB, TLS certs, any needed cron jobs, etc.). The use case for this is for getting back up and running quickly when data is lost (e.g., power surge damages the hardware, SD card stops working). This feature would also help supports the use case of wanting to be able to update. The process could be: flash a new O/S onto the spare pi to get all the latest code, then restore from a backup to set everything.

In my opinion, it may be reasonable to make this separate from the existing backup/restore feature. The current backup/restore process makes it easy to share configuration files and NOT get the other settings. This supports the use case of a multi-zone system where there used to be 3 thermostats and these have since been replaced by 3 Hestia Pis. The user would likely want to just get one Hestia to start with, get it set up, and make sure they’re happy with it before buying subsequent units. It would be very nice to be able to restore the openhab2 configuration and have both of those units automatically know about all the things.

In terms of implementation, it seems like a directory (e.g., /backup) which contains a copy of all the config files that should be backed up, the script to back them up, and a script to restore them would be a nice clean option. It contains all of this new backup system (maybe call it “system backup” to differentiate it from the existing “backup”) in a single directory. A cron job could run the backup (e.g. weekly) and I propose no versioning happen on the pi.

I propose versioning be handled outside of this system. For example, from a backup server, scp the /backup directory from the pi to the server every week and handle versioning on the backup server side of things (and the files will be small enough that a full backup can probably be taken and then just eliminate files with duplicate content). Or maybe the user wants to put all their configs in a git repo so they scp them over to some system, and have a script to add all files, commit, and push up to a repo (and in most cases the “add”, “commit”, and “push” steps would have no visible effect.

[HestiaPi]

I can totally see your point. The idea of a script to restore selectively from the list of all backups sounds good. Maybe start an issue on GitHub as a placeholder so we don’t forget.

[rlkoshak]

@hestia_hacker

The process could be: flash a new O/S onto the spare pi to get all the latest code, then restore from a backup to set everything.

I’m not sure that would work in practice. For example, there are already some massive changes made to the openHAB configs on the ONE branch on GitHub. Doing an update like this will wipe out all of that, at which point you really haven’t done much of an upgrade at all. So much of what makes HestiaPi is configuration. Blindly restoring old configs over new configs will wipe out a lot of the upgrade and might even break things.

Where possible, users should keep their customizations separate but that’s not always feasible in practice. For example, I can have a separate rules file to implement hysteresis for heating/cooling. There has to be some changes made to the default.rules file to make that happen. And on an upgrade that means losing any improvements made to the rules to use your modified version of default.rules, or losing your modifications and forcing you to reintegrate your changes into the new rules.

I think this needs a good deal of thought on how we can manage updates for customized systems. I’m not sure we have a solution that really works.

As for implementation, I think we need to pay special attention to the audience. Based on my experience with openHABian, which uses Amanda for incremental backups, for many users the task of mounting an external USB thumb drive was too much, let along a network share. So the versioning approach needs to take heed of the intended audience and built and documented accordingly.

Beyond that, I really like the approach and think it should be pursued. Even if it’s use during upgrade isn’t as direct, the ability to backup everything I might have done to the machine, or at least the HestiaPi related stuff so it’s preserved would be incredibly useful. I like having it done automatically also. I don’t want to have to remember to make the backups. But then again, there shouldn’t be much of anything changing to require backing up. I wonder how hard it would be to make the script smart enough to detect when there has been a change and only backup then? It could keep a simple timestamp for the last time it ran and check all the files against that timestamp. Anything that changed more recently than that timestamp means we need to backup.