Nextwave*

* Not the Warren Ellis comic.  Sorry.

The current mu.nu / mee.nu server conglomeration consists of Aoi and Midori (blue and green), two Xeon 3220 systems with 8GB of memory and, well, a bunch of disk drives, and Sakura (pink), a Core 2 Duo 6500 system with 8GB of memory and a bunch of disk drives.

When Sakura went wobbly with a sprained disk a couple of weeks ago, I had to move some sites back to Midori, which then also went wobbly, albeit not because it was out of memory or CPU, but because it was out of Apache threads.  That caused me a few sleepless nights until I could get Sakura back in action.

There's plenty of free resources on Aoi, the mee.nu server, because Minx is so memory and CPU-efficient, but I can't easily drop random sites on there because the whole server is configured to run Minx.

That's why I got Sakura in the first place, or one of the reasons, the other being off-site backup in case something horrible happens.

Particularly around election time, I've wished that I could just upgrade the memory in Midori and install OpenVZ; that would solve a lot of problems.  And while I could install OpenVZ, upgrading the memory is not so easy: The motherboard supports a maximum of 8GB, which is kind of what we have already.

So I've been waiting, more-or-less patiently, for Nehalem-based servers to become available.  You see, the current Nehalem processors (Bloomfield and Gainestown) support three memory channels, and at least two DIMMs per channel, which means a single-processor Nehalem system can go to at least 12GB of memory.

And if the motherboard supports registered memory, you can get three DIMMs per channel and 4GB DIMMs, for a total of 36GB.

So at least 12, maybe 24 or even 36.  Any of which is an improvement on 8.

The server version of Nehalem - the Xeon 3500 (single processor) and 5500 (dual processor) ranges - are reportedly due out in exactly a week - March 29.  Intel haven't confirmed this so far as I know, but given that Apple is already shipping them in the new Mac Pro, the launch can't be too far off.

Apart from the improved memory support (up to 4.5x the capacity and 3x the bandwidth) over our current processors, the Nehalem delivers a sizeable performance boost.  On SpecInt a 2.66GHz Nehalem scores 28.6 vs 20.3 for the 2.4GHz 3220, a 40% increase, and on SpecIntRate, the multi-threaded version of the benchmark, it delivers 109 vs 59.2, more than an 80% increase.

The Nehalem supports hyperthreading, just like he good old Pentium 4, only this time it provides a much bigger performance boost.  A single 2.66GHz Nehalem scores about the same in multi-threaded benchmarks as a dual 2.33GHz Xeon of the previous model.  (On multi-threaded floating-point it does even better, about the equivalent of a dual 3GHz previous-model Xeon.  But I'm less interested in floating-point.)

So, that's memory and CPU covered.  What's next?

Well, last November we got an ioDrive in for testing at my day job.  These things are incredible; they deliver 120,000 random IOs per second.  Unfortunately, they're also incredibly expensive; the cheapest card costs $3000.

But SoftLayer now sells Intel's X25-E SSDs.  They're not as fast as an ioDrive, delivering "only" 30,000 read IOPS and 3,000 write IOPS, but that's still 25 to 250 times as fast as the disks we have right now.  We have one in our development server at my day job - I just finished setting it up - and it really seems to work as advertised.

Best part?  Only $50 per month.

So, 2.66GHz Nehalem, 12GB of RAM, 32GB X25-E.  What's next?

Next is RAID.  We're not running RAID at the moment, because when I set up the current servers I simply couldn't afford it - I have to pay for the RAID controller itself, and for twice the number of disks.  For each server.  Instead I have the servers doing a daily rsync to each other - which we'd still need to do, but with RAID we'd only need that for operator or software error, not for simple drive failures.

The good thing there is that SoftLayer's disk pricing is effectively half what it was when I set up the current servers, so I only have to pay for the controller.  So, hardware RAID it is, probably either 1TB RAID-1 or 1.5TB RAID-5.  RAID-5 is no good for databases, but the databases will be on the SSD, so it doesn't matter.

The one big variable is virtualisation.  I'd like to go Virtuozzo, because it's efficient and elegant (it's based on OpenVZ, but has some extra features and a nice control panel), and it integrates cleanly with Plesk.  (And offers substantial discounts on Plesk licenses - about 66%.)

The problem is, a 10-VM Virtuozzo license, the smallest practical size,
costs $100 per month.  Per server.  For that much money I could use Citrix XenServer (which is now free), add 6GB of memory to each server (to make up for Xen's less memory-efficient virtualisation model), and still save enough to pay for the more expensive Plesk license if I wanted it.

So, meh.  We'll see.

There's some other things I'm looking at - hardware firewalls (we sort of have one, but it's just there as a freebie and I don't have access to configure it); hardware load balancers (we use a software load balancer at mee.nu, which works very nicely as long as the server itself doesn't hiccup, but nothing at mu.nu right now); a content delivery network (which I already have, but need to configure); backups to iSCSI or CIFS storage (why don't they offer NFS? ).  They all cost money, though, so we'll see.

As to when?  First week of April.

Posted by: Pixy Misa at 11:29 AM | No Comments | Add Comment | Trackbacks (Suck)
Post contains 971 words, total size 6 kb.