Dear Santa, thank you for the dolls and pencils and the fish. It's Easter now, so I hope I didn't wake you but... honest, it is an emergency. There's a crack in my wall. Aunt Sharon says it's just an ordinary crack, but I know its not cause at night there's voices so... please please can you send someone to fix it? Or a policeman, or...
Back in a moment.
Thank you Santa.

Sunday, September 16

Blog

Performance Glitches

I've noticed some performance hiccups on the site recently, and users might have as well.  I think it's tied to web spiders over-enthusiastically indexing our sites.  

I've updated our logging to try to trap the culprits; I'm not planning to throttle Google or Bing, but if it's Yandex or Baidu causing problems I will stomp on them.

Posted by: Pixy Misa at 12:34 AM | No Comments | Add Comment | Trackbacks (Suck)
Post contains 59 words, total size 1 kb.

Friday, September 07

Blog

Speaking Of Servers

I cancelled the server I ordered by mistake and the new server is up and running.  It has the same basic specs with two differences: Instead of 8 x 1TB disk drives on the old server (I misread the specs and thought it was SSDs) it has 6 x 2TB SSDs.  Really real SSDs this time; I've tested the array at 300,000 IOPS, the equivalent of 2500 regular 7200 RPM disk drives.

And instead of 200TB of monthly bandwidth, it's 1Gbit unmetered.  Which doesn't actually make much difference, because 200TB is close to saturating 1Gbit outbound and I don't do much inbound traffic.

Oh, and it's software RAID rather than hardware.

Reinstalling it now, configuring RAID-5 and LVM, so I can take consistent snapshots of the entire server without having to worry about managing clean database dumps of MySQL and MongoDB and Elasticsearch and and and...

Then I install KVM and LXC, then I start migrating systems across into their own neat little virtualised containers.

Update 1: Manually configured RAID-1 for boot and RAID-5 for LVM, splotted swap volumes everywhere, and installing Ubuntu 18.04.1 right now.  The auto-install script unhelpfully assigns 100% of the default volume group to / meaning you have no room left to take snapshots.  I hope I got it right, but at least the partitioning is right so it will be a lot easier if I have to reinstall again.

Update 2: The secret is to use the autoinstall to bring up the server quickly, then use fdisk to create your custom partitioning scheme, then use the Ubuntu expert install mode to install on those partitions.  Much much quicker than fiddling about with the partitioning tool in the installer.

Also, don't install to a huge software RAID-5 or 6 array, even on fast SSDs.  It takes at least five times longer than normal because the sync will be running the entire time.  Create a RAID-1 array for the OS and you'll be done that much faster.

Update 3: Whee!  That was so much faster.  Let's see if the network config works this time...

Update 4: Yep, that worked perfectly.

https://ai.mee.nu/images/Mari.PNG?size=640x&q=95

Update 5: Well, I messed that up slightly.  I think I'll just go with LXC here, and leave KVM alone.

Posted by: Pixy Misa at 06:12 PM | No Comments | Add Comment | Trackbacks (Suck)
Post contains 376 words, total size 3 kb.

<< Page 1 of 1 >>
37kb generated in CPU 0.02, elapsed 0.1547 seconds.
49 queries taking 0.1379 seconds, 170 records returned.
Powered by Minx 1.1.6c-pink.
Using https / https://ai.mee.nu / 168