Switching from Windows to Linux: Sharing My Experience and Lessons Learned

I am looking for some incremental file backup software.

My rather flaky ZFS volume which had an incident yesterday falsely claiming that several of my SSD's had failed with thousands of errors. My ZFS warning message popup script worked great! A not so quick reboot and suddenly there are no errors at all.

I want to make incremental backups of my Virtual Machines one of which has a 1.5TB disk image to my NAS and some other files on a daily basis. I am remembering the old adage that RAID is not a Backup! I was lucky this time..

I think Timeshift would have to copy the entire disk image every time the contents change which would take too long.

Is there a Linux programme that can do this or am I better creating an image within the Virtual Machines and using Timeshift for the rest?

Edit: Thinking about it I don't think incremental backup of a file works like I thought. I doubt the software wood look for difference within a file and only record the new data.

Edit 2: I have installed Paragon Backup free to take care of the 2 Windows Virtual Machines. Timeshift won't save to a network share unless I am missing something. Would Rsync be suitable? I need to be able to role back files to previous versions of files in the event of corruption.

Edit 3: Paragon Backup turned out to be a disappointment, just as well as it was free. Macrium Reflect free seems to be much better.
 
Last edited:
Aren't incremental backups (snapshots) one of the key features of zfs? Or are you thinking in addition to that, if so then rsync the snapshots perhaps. I went basic with rsync and old school par2 in addition to that.
 
santa said:
Aren't incremental backups (snapshots) one of the key features of zfs? Or are you thinking in addition to that, if so then rsync the snapshots perhaps. I went basic with rsync and old school par2 in addition to that.
Click to expand...
Yes snapshots should be enough and that's one of the reasons why I am using ZFS. its supposed to be my Fort Knox of storage for me, however I was faced with a situation where my ZFS wrongly claimed several of my drives had failed all at once on a RAIDZ2 which if it was true there wouldn't be enough redundancy to rebuild. The incident made me rethink my backup strategy or lack of it, as I had only prepared a bunch of SSD's to replace any failed ones that might crop up now and then. This lack of forsite could have cost me all the data.

Rsyncing the snap shots could be the answer. I have not heard of par2 and will look into that. I need to have a long think about ways to mitergate the problem should it happen again.
 
If you lost the whole array do you have a backup plan?

RAIDZ2 is what I use but mostly for redundancy and convenience in the case of drive loss or errors; it doesn’t factor into how I ensure my data isn’t lost. I have all my key data backed up onsite again, plus semi-regular updates to a secure cloud storage provider for offsite. So I could start again if needed.

In terms of how to sync/backup I’m using samba shares with the zpool as well as running Time Machine for macs and then my cloud provider has a cli application that I can run to sync. For my onsite backup it’s manual ‘cp’. Maybe I could automate both those but it doesn’t save much time, just easier if I forget I guess. I usually remove the secondary backup anyway just in case I leave it connected and they both get lost. So there’s still a manual step anyway.
 
alex24 said:
If you lost the whole array do you have a backup plan?

RAIDZ2 is what I use but mostly for redundancy and convenience in the case of drive loss or errors; it doesn’t factor into how I ensure my data isn’t lost. I have all my key data backed up onsite again, plus semi-regular updates to a secure cloud storage provider for offsite. So I could start again if needed.

In terms of how to sync/backup I’m using samba shares with the zpool as well as running Time Machine for macs and then my cloud provider has a cli application that I can run to sync. For my onsite backup it’s manual ‘cp’. Maybe I could automate both those but it doesn’t save much time, just easier if I forget I guess. I usually remove the secondary backup anyway just in case I leave it connected and they both get lost. So there’s still a manual step anyway.
Click to expand...

I have irregular manual backups of some of my files on my NAS so it wouldn't be a total loss. Off site, I only have 1TB Onedrive for some of the data. I tend to mostly save copies of my Onedrive locally in case something goes wrong. I am accually trying to move my files away from Onedrive as I think its a big target for a hack and is probably subjected to data mining by MS, Its another tie I need to cut in my quest to divest from MS..

In a perfect world I would be able to have a offsite backup for my NAS but I think that would cost a lot as Its half full about 15TB. I suppose I could get an external USB HDD to back up the NAS.
 
Par2 are old school parity files (for a given large file you can generate a small file that contains parity for it), ZFS/btrfs/bcachefs have a similar thing built into the FS itself to detect and correct errors, probably not worth you trying par2. It's just a little extra layer to a full backup of my film collection, handy in that case as you can have a USB stick full of par2 files for a hard drive's worth of films.
 
You must log in or register to reply here.
Back
Top Bottom