question

gainestr avatar image
gainestr asked

System wide backup using SFTP

What is the best way to backup the system? I know that I can simply clone the flash drive but that requires me to have the system down for the process. Is all of the systems configurable and user data stored in the /data folder? Can I simply copy that folder remotely via SFTP for a quick backup? In the event of a system failure, all I'd have to do is install a cloned flash drive and install a copy of the /data folder and I would be back in business right? It would be easy enough for me to copy the /data folder to a zip file and have that automated using a script and then transfer the zip file via SFTP on a daily basis if I wanted but I'm wondering if it is as simple as that or if there are other folders/files I need to include in a backup. Also, I know that GuiMods has an option to backup the system but it requires you to install a USB/Flash drive and I'm more interested in doing this remotely via SFTP. So, as a feature request, I'd like to see a way to backup pertinent system and user data to a zip file to be either copied over to removable storage or transferred remotely.


feature request
2 |3000
Toggle Comment visibility. Current Visibility: Viewable by all users

Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

4 Answers
mr-manuel avatar image
mr-manuel answered ·

Hello,

I wrote a script that backup the whole SD card to a SMB/CIFS share. It's a lot more data (depending on you SD card size) than backing up only the /data folder.

If you are able to mount a SFTP folder to the local system, than you can use my script also with SFTP.

In case of restore you just have to flash the image to the SD card and you are good.

Here is the link: https://github.com/mr-manuel/raspberry-pi-backup

2 |3000
Toggle Comment visibility. Current Visibility: Viewable by all users

Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

gainestr avatar image
gainestr answered ·

There are a couple of small issues with your script.

Firstly, It doesn't copy the mount.cifs file correctly into the /sbin folder under VenusOS. Looks like it isn't looking into the correct folder.

error:

cp: can't stat '/data/etc/ext/mount.cifs': No such file or directory


Once I manually did that and changed the attributes, I was able to mount the shared folder correctly. Secondly, I had to remove the '/' at the end of the BACKUP_SUBFOLDER="Backups/" subfolder in the script because it was trying to add it in twice. Once I got that sorted out, your script started working pretty much as expected but, I am getting another error that says: Unable to find suitable address

However, I can manually run the commands and get things to work.

Thank you and much appreciated. I am now able to do a backup remotely to my system without having to remove the micro SD card in the raspberry pi to my NAS. This will be super helpful if/when I have a system failure without having to do a complete system install from scratch.

For some reason the backup is extremely slow but I'm thinking that it has more to do with the built it wifi of the raspberry pi than it does with anything else. I'm only getting speeds of 239 kB/s for some reason.

4 comments
2 |3000
Toggle Comment visibility. Current Visibility: Viewable by all users

Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

mr-manuel avatar image mr-manuel commented ·
Thats for the feedback. I will check the script, on my end it was running fine. What Venus OS version do you have?
0 Likes 0 ·
gainestr avatar image gainestr mr-manuel commented ·
currently running v3.00~21. the systems dd does not support status=progress apparently.
0 Likes 0 ·
mr-manuel avatar image mr-manuel gainestr commented ·

I fixed the errors.

status=progress is not supported on the dd version in Busybox on Venus OS. Therefore it should run the dd in the ext folder. I changed the logic and this should also work now.


0 Likes 0 ·
gainestr avatar image gainestr mr-manuel commented ·

For some reason the script still doesn't mount the shared drive but I can run the mount command manually and it works. When I get time I'll dig into it a little more but I can confirm that the script does seem to work once the drive is mounted. Currently backing up to my NAS now. Thanks for your work on this. Couldn't have done it without your insight and help! This works for me and now all I have to do is compress the .img file to save a little space on the NAS.

1 Like 1 ·
Kevin Windrem avatar image
Kevin Windrem answered ·

Yes, /data stores all configuration items. In fact in it's stock configuration /root is read only and not changed during operation. Backing up all of /data will probably work as long as you are restoring to the same system with the same I/O configurations as when the backup was made.

There are also some things modified in /u-boot (the /boot partition) that you'd need to restore for complete functionality: config.txt, cmdline.txt and any overlays added to the system

You would not want to replace /data in a working system however, so your concept of doing the restore with the SD card on another system is a good approach.

The GuiMods backup/restore does not save settings that are based on serial numbers, device names or other things that might change. Victron experimented with a mechanism to backup the entire settings database but had problems, I assume in this same area.

2 |3000
Toggle Comment visibility. Current Visibility: Viewable by all users

Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

Jons Collasius avatar image
Jons Collasius answered ·

This is for the linux machine you will back TO, not for the GX device! You will pull a backup, not push!

Save this to /etc/cron.daily/rsync-gx and chmod 755 the file

#!/bin/sh
sshpass -p "CerboPassword" rsync -avL -e "ssh -o StrictHostKeyChecking=no" root@10.0.0.50:/data/ /var/backups/einstein/

Replace CerboPassword with the ssh password of your GX device
Replace 10.0.0.50 with the IP of your GX device
Replace /data/ with any directory you like to sync
Replace /var/backups/einstein/ with a local directory you want to sync to

Now every day you get a backup on to the device the cronjob runs on.

2 |3000
Toggle Comment visibility. Current Visibility: Viewable by all users

Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

Related Resources

The feature requests topic tag can help Victron R&D find feature suggestions for products. 

There are many factors that go into the Victron product roadmap, and feedback from end-users can help.