Minxulations

Some of you might remember, back about 15 months ago, when we were on the old old server, it used to take something like 30 seconds to leave a comment. That was before we moved to the new old server, and before we moved from Berkeley DB (boo hiss!) to MySQL (yay!)

Back then, I started work on a blogging program I called Minx. Minx was written in Python and used a database called Metakit, and it was really really fast, at least until you started getting a lot of posts and comments, whereupon it ate all your memory and died.

So much for Minx 0.1, which bit the dust around January 2004.

Minx 0.2 re-arranged the database a bit, and came along in April 2004. However, by that time we were on the new server and things were relatively speedy again, so there was much less urgency to Minx (and I had a lot of work to do with the migration, and a swarm of new MuNus who joined us around that time). And as it turned out, Minx 0.2 also, once you got past a certain point, ate all your memory and died.

Minx 0.3 came along in December, just before my brain got eaten by hackers and spammers, something that lasted through January and February. March I basically went to bed and played Final Fantasy X and Rumble Roses.* This month I dug out Minx and started playing with it. Minx 0.3 was the best yet, but after a certain point, it ate all your memory and died.

The problem in all three cases was the Metakit database, which is extremely fast and flexible, but has a nasty habit - if you push it too far - of eating all your memory and dying.

And so, a couple of weeks ago, I came up with this startling insight: Minx might work better if I didn't store all the data in Metakit.**

And so Minx 0.4 was born. And it's even faster and better than before, and it still works when you throw - What was it? Right. - when you throw 4000 blogs with a total of 4 million posts (and 20 million comments) at it. At that point, my home machine was starting to struggle a little, but certainly hadn't eaten its memory and died. Minx 0.4 uses Berkeley DB (boo - I mean yay!) only it uses it completely differently to Movable Type, so that it actually works.

Soon - maybe even very soon - the Minx Dev Blog and Ambient Irony will be making the Great Leap Sideways and switching to Minx. You'll know it's happened when trackbacks suddenly stop working. Well, unless they come back five minutes later, in which case it's just Fluffy the anti-spam watchdog at work.

Oh yes, there was a point to all this. Munuvians, if you're interested in the future of your blogging platform™, hop on over to the Minx Dev Blog and take a look around. If that makes your brain hurt, wait for the test release, coming soon to a server near you.

* I'm not a sexist pig, I just play one on TV. Um, literally. At least it's better than Dead or Alive: Extreme Beach Volleyball.
** There's slightly more to it than that. It involved a full understanding of Metakits on-disk and in-memory data structures, and realising that there was no work-around for the problems I was encountering. No matter how I split up the data, as long as I stored it all in Metakit, it would eat up all my memory and die.

[Also posted at Munuviana]

Posted by: Pixy Misa at 02:00 AM | Comments (3) | Add Comment | Trackbacks (Suck)
Post contains 603 words, total size 3 kb.