I'm in the future. Like hundreds of years in the future. I've been dead for centuries.
Oh, lovely, you're a cheery one aren't you?

Self-Similar Loads And The Deaths Of Cloud Computing

The recent collapse of not one, but multiple entire Amazon Availability Blooples* into a smoking crater caused a certain amount of buzz in the webosphere.  It would have caused more of a buzz if it hadn't reduced a fair chunk of the webosphere to a smoking crater along with it.

What happened?

Well, someone at Amazon threw the wrong switch during a network upgrade.  Effectively, instead of rerouting traffic onto a carefully planned detour, they rerouted traffic onto the sidewalk.

This did not go over terribly well with all the servers trying to send data to their storage pigs* further along the sidewalk.  Since there's a significant variability in performance of Amazon storage pigs* many servers were set up to take any slowdown as an indication of a bad pig* and automatically try to set up a new pig* to replace it.  To do this, the data had to be replicated....

Along the sidewalk, which was already jammed beyond capacity.

To say that the problem snowballed at this point would be to waste a perfectly good video involving mousetraps and ping-ping balls.



You see, the idea of setting up a huge hosting cloud thingy like Amazon has done is that most servers run mostly idle most of the time.  (Ours, for example, has 12 cores and uses, on average, slightly less than one.)

So if you aggregate a whole lot of servers together into one huge bloople* you can get far more sites running on far less hardware and make a huge amount of money in the process.  Until someone drops a ping-pong ball; once that happens there's no way to stop the process.  It's far too big and complicated to control manually.  The entire bloople* is set to burn down, fall over, and sink into the swamp and all you can do is watch.


All you can do is watch...

Because traffic (and hence load) doesn't neatly average out when you aggregate lots of different services together.  Instead, it piles up.  Internet activity levels are self-similar - everything everywhere tending to follow the same pattern of spikes and dips at the same time. 

When one service spikes, it's likely that everything else is spiking at exactly the same moment.  And since cloud computing gains efficiency by eliminating the huge amount of headroom you would traditionally plan into a dedicated server (or server farm, depending on how many shoestrings you have to throw around), this leads to everyone looking for extra capacity at the same moment.  And that puts more strain on everything right when it's at its busiest, and....

Splat.*

In Amazon's case, the splat* was triggered by someong dropping a ping-pong ball.  But that's just the proximate cause.  People drop ping-pong balls every day.  It's only a drama if you happen to have covered every level surface of your home including the ceiling with fully-armed spring-loaded ping-pong ball launchers.

But that's what every cloud provider, almost without exception, has done.  That's the entire business model.  It is cheap, but it's intrinsically flaky.


Intrinsically flaky.

It's no accident either that the piece of the puzzle - uh, bloople* - that flaked out in this was the flakiest flake of all, the network-attached storage.  Amazon's EBS gives you disks attached across a network. 

Disks suck.  There's no gentler way to put it.  At my day job, we have SSDs all over the place, because we'd be dead without them.  (We know, because we tried that at the start.  We died.  Then we went out and bought a bunch of SSDs and tried again.)  Disk access is on the order of ten million times slower than CPUs, and modern servers typically have more CPUs than disks.

Even so, when your disks are right there in your server, at least you can see how busy they are (too busy) and who's using them (you).  When the disks are abstracted away to free-roaming data pigs*, all you have is an end result.  Pig* too slow?  Don't try to investigate the problem.  You can't investigate the problem; it's been abstracted to such a degree that there's simply no information available.  People tried mounting new pigs* because that was the only thing they could do.  They were throwing gasoline onto a bonfire, but when you build a bonfire and hand everyone a free can of gasoline, you really shouldn't be surprised at the result.

So, how do we fix this?

Well, first, everyone everywhere who has anything to do with anything at all should be nailed to the floor and forced to read J. B. S. Haldane's On Being the Right Size.

Second, anyone planning to deploy a new server with disks used for anything other than backups and log files should be lightly shot.

Third, watch Ano Hana.

* The technical term.

Pictures from A Channel and Ano Hana via RandomC.

Posted by: Pixy Misa at 10:11 PM | Comments (3) | Add Comment | Trackbacks (Suck)
Post contains 815 words, total size 6 kb.

Ranting Aside

When and how did the weakest anime season in recent years turn into one of the best?

Whatever; please keeping doing this!

Posted by: Pixy Misa at 01:24 AM | No Comments | Add Comment | Trackbacks (Suck)
Post contains 24 words, total size 1 kb.

Great Art, Tiny Minds

So, I was just catching up on Ano Hana - which continues to bravely walk the tightrope of awesome over the great chasm of suck - and thought to myself, self, why not hop over to Youtube and find the opening and ending themes and plip them into your one-line reviewlet so that people can get an idea of what they're missing?

The answer to that question is, of course, Sony Fucking Music.*  The only clips available are mirrored, distorted, or both, because Sony F. Music have taken down all the rest.

They're also the reason I no longer have a Youtube account.  At least, I think so; Youtube won't actually tell me.

So, just trust me on this, okay?  Ano Hi Mita Hana no Namae o Bokutachi wa Mada Shiranai.  Watch it.  It's not for everyone, but it's sure worth taking a look.

* The Fucking is silent.

Posted by: Pixy Misa at 12:48 AM | Comments (2) | Add Comment | Trackbacks (Suck)
Post contains 153 words, total size 1 kb.

Today's Unpaid Steamvertising...

Is brought to you courtesy of Kalypso, who are offering Tropico Reloaded (consisting of Tropico, Tropico: Paradise Island, and Tropico 2: Pirate Cove) for $1.74.

The only possible downside to this is that this bundle doesn't let you play the original version of Tropico, only the version with Paradise Island installed.  Which means you get the second, less, um, tropical theme song.

Also available at GOG completely DRM-free - and with the soundtrack as a downloadable extra - but $9.99.

I never liked Pirate Cove that much, but the original Tropico was a delight.  If you've ever wanted to run your own little Caribbean island nation, this is your chance.  You can play as a benevolent dictator, as a not-so-benevolent dictator, as an elected president, as an "elected" president, as a client state of either the US or the SU, however you like (though maintaining power while having free and fair elections on a regular basis is hard).  Very recommend.

Steam also has Tropico 3 Gold going for $4.99.  I haven't played that one yet, but it seems to be pretty well-regarded.

Posted by: Pixy Misa at 03:52 AM | No Comments | Add Comment | Trackbacks (Suck)
Post contains 185 words, total size 2 kb.

Railcars In Anime Update

The girls in A Channel are riding the Enoshima Electric Railway:



This may be the spot where Run starts chanting Umi!  Umi!



And this (Koshigoe Station in Kamakura) could be where they find themselves in the middle of the road:



The first one is an obvious perfect match that I just stumbled upon based on this post on AnimeSuki; the latter two are guesswork from what I can find on the net.  All three pictures care of Wikimedia Commons.

Update: Um, a few posts further down someone did my detective work for me.  Oh well.

Update 2: Ah, now I realise they're starting from the Kamakura end of the line, so I don't know if that sequence of photos makes sense.  But it's definitely the same train at the same station.

Posted by: Pixy Misa at 02:33 AM | Comments (1) | Add Comment | Trackbacks (Suck)
Post contains 136 words, total size 1 kb.

In The Spring

A young man's fancy lightly turns to thoughts of...

Railcars?

A Channel



Ano Hana



Denpa Onna



Iroha



I have no idea why, but the Spring season is definitely railcar season.  All the anime I've watched so far has included railcars.  In fact, when I saw the setting of the first episode of Ano Hana, I was just waiting for the railcar to show up, and was quite disappointed when it didn't.  But then in the second episode my patience was rewarded.

Also, hell's bells, the background artists in some of these shows have been putting in the overtime.  Even simply-animated productions like A Channel have some gorgeous background art.

Posted by: Pixy Misa at 04:38 AM | No Comments | Add Comment | Trackbacks (Suck)
Post contains 112 words, total size 2 kb.

Ano Hi Mita Hana No Namae O Bokutachi Wa Mada Shiranai

The name is a mouthful, but the anime is very, very good.

Score!  Four out of four railcars.

Posted by: Pixy Misa at 04:10 AM | Comments (4) | Add Comment | Trackbacks (Suck)
Post contains 29 words, total size 1 kb.

One Day Only!

Mass Effect is $5 on Steam today.

Mass Effect 2 is $9.99.

Don't miss out!

Sadly, Mass Effect 3 has been delayed until next year.  And as it turns out, I'm not feeling Fallout 3 and Bioshock makes me queasy, leaving me with a sudden outbreak of free time.  Maybe I'll get some work done...

Posted by: Pixy Misa at 03:47 AM | Comments (1) | Add Comment | Trackbacks (Suck)
Post contains 58 words, total size 1 kb.

Quick Precis Time

Dragon Age: Origins

The best recent computer role-playing game.  Superb storytelling and characters, and the fact that the entire country is the colour of mud only enhances the realism.

Four giant undying demon dragon gods out of four.


Dragon Age II

Takes everything that was good about the original game and breaks it in the name of capturing a wider audience.

Zero teleporting darkspawn out of four.


Mass Effect

Superbly crafted story brought down only by some dull and overused settings.  (Not all of them, by any means, but some.)  Great science fiction action/RPG.

Three and a half exploding geth snipers out of four.


Mass Effect II

Superbly crafted setting brought down only by some dull and overused stories.  (Not all of them, by any means, but some.)  Great science fiction action game with just enough RPG elements to keep me happy.

Three and a half apologetic security droids out of four.

Posted by: Pixy Misa at 01:09 PM | No Comments | Add Comment | Trackbacks (Suck)
Post contains 155 words, total size 1 kb.

The Name Of The Wind

By Patrick Rothfuss

The words are pretty, but I have no interest in the main character whatsoever, and 70-odd pages in there are no other characters, only cardboard cutouts.

Taking the broken hero and winding him up and setting him on his way again, Curse of Chalion-style, could have worked well.  But a two-thousand-page flashback?  No.  Just no.

Zero silences out of four.

Posted by: Pixy Misa at 12:57 PM | No Comments | Add Comment | Trackbacks (Suck)
Post contains 68 words, total size 1 kb.