Sunday, April 18

Blog

The Prodigal Server Returns

Yes, Akane is back - our shiny Ryzen 3700X, with its 64GB of ECC RAM and enterprise NVMe storage - in a shade under two weeks.

Doing a full offsite backup, followed by software updates, then we can return to something approaching normality.



Meanwhile I'm getting errors on the backup drive I had them swap into the new server at my day job, which is annoying because I'll need to re-take or re-verify 11TB of backups, but nothing is actually down which is refreshing.

Update: Backups of our server are complete.  That server is so fricking fast compared to this one.  I'd like to replace it with a 5900X but it's already really fast.

Backups of the backups of the work server are ongoing, since there's 60x as much data over there.  (11TB vs. 180GB.)

Oh, and the HoloEN Minecraft server is back too.


Update: One of the things that worried me about the main server being down for so long was that I didn't have an up-to-date offsite backup.  I did have offsite backups, just nothing less than a month old.

That server is running LXD virtualisation on ZFS.  This gives you two ways to do backups:
  1. lxc snapshot which is simple and instantaneous and uses minimal disk space but is stored on the local drive
  2. lxc export which gives you a complete portable backup in a single file but by default backs up your everything straight into your root filesystem

So if your container is larger than the available space in your root filesystem, splat.

You can configure it not to do that, but it's not very well documented, and by not very well documented I mean have fun trawling through Stack Overflow, sucker.

Anyway, since right now everyone is on this server and that server is free, I thought I'd try updating the software and configuring proper backups with lxc export.

Tried it on a small container - around 1GB - with pigz (parallelised GZip) compression, and it completed in 15 seconds.  Great!

Tried it on the main blog container, which is around 70GB of active data, and so far it's been running for two and a half hours, has used 470GB of temporary space, and hasn't even started writing the backup file yet.

Oh, and it doesn't give you any progress information, not even Microsoft level where the indicator sometimes runs backwards.

And you can't stop a running backup.

This is garbage.

Update: Always use --instance-only to avoid the snapshot explosion, and to cancel a running export, kill the pigz; the backup process will abort cleanly.  

Meanwhile I'm watching Kanata building and tuning an array of Minecraft note blocks and redstone delay thingies to play Hololive songs by ear.  She doesn't even count, she just goes wapwapwapwapwap on one block after another and it comes out right.  Gen 4's own little Beethoven.

Posted by: Pixy Misa at 01:09 AM | No Comments | Add Comment | Trackbacks (Suck)
Post contains 480 words, total size 4 kb.




Apple pies are delicious. But never mind apple pies. What colour is a green orange?




44kb generated in CPU 0.15, elapsed 0.44 seconds.
56 queries taking 0.31 seconds, 256 records returned.
Powered by Minx 1.1.6c-pink.