hi peeps.
Im early on my selfhosting journey and managed to setup some things as a test in docker. the main service im looking to run is ERPNext. after a while I got it up and running so now I want to back this up including all the customization I did in the app.
where to start with this? any intuitive guide?
I backup the whole / with borg
It has insane deduplication and compression and it creates a structure kind of like git, so you can have lots of incrimental versions without using much space.
Borg is a solid choice, but over the last couple years I’ve transitioned to Restic which prefer slightly. It seems a lot faster, has better usability/ergonomics, and easier to configure a write-only setup (so your server can write snapshots, but is incapable of deleting and such).
Check the app’s own docs first, there is something here about automating backups:
https://docs.frappe.io/erpnext/user/manual/en/download-backup
Oh this is smart! Thanks
Modern filesystems like ZFS have snapshots and the ability to incrementally copy those around even over the network. I’d suggest considering using those instead of something that operates on top of the filesystem.
Uuh what? Im not sure I follow. this feels more complicated rather that using some sort of service
A snapshot is like copying a the files in the snapshot to another location, while the original files remain as is and can be written to or read from.
As an example, if you have a folder, let’s say
/important/secret/stuff, and you want to take a backup of it. You have to make sure that nothing is writing to those files while the backup is running, otherwise the backup risk being corrupted. (This is because if the backup starts to read the file, and halfway through the first part of the file changes, you now have a backuped file that half the old and half the new file).With a snapshop, you can “copy”
/important/secret/stuffto some other location, run a backup reading from this other location and then remove it. Any changes to the original files will not affect the backup since the backup is reading from the “copied” files (snapshot).So what you do is take a snapshot before running the backup, have the backup read from the snapshot and when it is complete, remove the snapshot.
Well there’s software that can make it easier. Just an option. For ZFS specifically there’s zfs_autobackup which I’ve got experience with and I haven’t tried this but just found out about a Web UI that works with it: https://github.com/natankeddem/bale
In addition to daily backups, once a month I image the drive. I wrote a simple script triggered by a cron job to image the drive to a NAS backup. The daily backups go to 3 different offsite storage facilities, and two go to separate NAS drives. All drive images are kept both local and off premise as well. So, for individual files, etc, I can restore them from the daily backups. If the wheels fall off, I can restore the whole drive from an image. Might be a bit over engineered but I’ve been caught a few times and so I just decided that won’t happen again.
Interesting.
Yep, I agree there’s 2 types of backups:
- data
- OS image
Out of curiosity, how are you doing the drive imaging?
Out of curiosity, how are you doing the drive imaging?
I wrote a script that is fired by a cron job once a month:
sudo nano /usr/local/bin/backup_drive.shspoiler
# Directory to store backups BACKUP_DIR="/mnt/myhdd" # Drive to backup DRIVE="/dev/sdX" # Create a timestamp TIMESTAMP=$(date +"%Y%m%d") # Filename for the new image NEW_IMAGE="$BACKUP_DIR/my_drive_image_$TIMESTAMP.img" # Create the image sudo dd if=$DRIVE of=$NEW_IMAGE bs=4M status=progress # Keep the last 3 images ls -tp $BACKUP_DIR/my_drive_image_*.img | grep -v '/$' | tail -n +4 | xargs -I {} rm -- {}Make the script executable:
sudo chmod +x /usr/local/bin/backup_drive.shCreate Cron:
sudo crontab -e0 3 1 * * /usr/local/bin/backup_drive.shVerify cron:
sudo crontab -lI’m sure it could be written differently or that there is a better way to do the imaging, but that is the extent of my notes, and it works for me. As with any code you find on the internet, it is wise to evaluate before dropping on a production server.
Ah, good old
ddWhen you have some spare time, take a look at
partclone- clonezilla uses it because it only backs up used blocks, not free space, so more efficient.Thanks for the tip. I’ll definitely check it out
I started selfhosting month ago. I’ve came across Borg Web Ui. Its borg, but wrapped in nice web ui. I managed to setup daily local backups to external drive without struggling in the CLI. Borg web ui provides more dumbed down documentation (which is good for me). I’ve also setup to send notification to my phone when the backup is successful. I used NTFY for that


