Thursday, April 28
Some of you might remember, back about 15 months ago, when we were on the old old server, it used to take something like 30 seconds to leave a comment. That was before we moved to the new old server, and before we moved from Berkeley DB (boo hiss!) to MySQL (yay!)
Back then, I started work on a blogging program I called Minx. Minx was written in Python and used a database called Metakit, and it was really really fast, at least until you started getting a lot of posts and comments, whereupon it ate all your memory and died.
So much for Minx 0.1, which bit the dust around January 2004.
Minx 0.2 re-arranged the database a bit, and came along in April 2004. However, by that time we were on the new server and things were relatively speedy again, so there was much less urgency to Minx (and I had a lot of work to do with the migration, and a swarm of new MuNus who joined us around that time). And as it turned out, Minx 0.2 also, once you got past a certain point, ate all your memory and died.
Minx 0.3 came along in December, just before my brain got eaten by hackers and spammers, something that lasted through January and February. March I basically went to bed and played Final Fantasy X and Rumble Roses.* This month I dug out Minx and started playing with it. Minx 0.3 was the best yet, but after a certain point, it ate all your memory and died.
The problem in all three cases was the Metakit database, which is extremely fast and flexible, but has a nasty habit - if you push it too far - of eating all your memory and dying.
And so, a couple of weeks ago, I came up with this startling insight: Minx might work better if I didn't store all the data in Metakit.**
And so Minx 0.4 was born. And it's even faster and better than before, and it still works when you throw - What was it? Right. - when you throw 4000 blogs with a total of 4 million posts (and 20 million comments) at it. At that point, my home machine was starting to struggle a little, but certainly hadn't eaten its memory and died. Minx 0.4 uses Berkeley DB (boo - I mean yay!) only it uses it completely differently to Movable Type, so that it actually works.
Soon - maybe even very soon - the Minx Dev Blog and Ambient Irony will be making the Great Leap Sideways and switching to Minx. You'll know it's happened when trackbacks suddenly stop working. Well, unless they come back five minutes later, in which case it's just Fluffy the anti-spam watchdog at work.
Oh yes, there was a point to all this. Munuvians, if you're interested in the future of your blogging platform™, hop on over to the Minx Dev Blog and take a look around. If that makes your brain hurt, wait for the test release, coming soon to a server near you.
* I'm not a sexist pig, I just play one on TV. Um, literally. At least it's better than Dead or Alive: Extreme Beach Volleyball.
** There's slightly more to it than that. It involved a full understanding of Metakits on-disk and in-memory data structures, and realising that there was no work-around for the problems I was encountering. No matter how I split up the data, as long as I stored it all in Metakit, it would eat up all my memory and die.
[Also posted at Munuviana]
Sill waiting for the video card for my new computer, which will be named, let's see... Naga. I could get an X800XT tomorrow, but that's more than I wanted to spend on a video card for a Linux box.
Oops, phone ringing... I can get an Abit X600XT by Friday. Sounds good to me.
Meanwhile, if you're in a geeky frame of mind, I've been posting over at the Minx dev blog. Minx is the new integrated blogging & forum system I'm developing for MuNu. The project was stalled for about a year, but recently got under way again and is now close to its first alpha release. It's blazingly fast, and should run even faster on an Athlon 64 3200+. Zoom zoom!
Monday, April 25
Psyco is a compiler for Python. Once you've installed it (which consists of typing the command
python setup.py install), you can use it by adding two lines to your Python program:
How well does it work? Well, I'm currently working on Minx, an integrated blog/forum/wiki/coffee-and-donuts application, which happens just by chance to be written in Python.
Let's see how long it takes Minx to generate 10,000 pages of 20 posts, first with regular Python:
real 1m28.206sAnd now with new improved Psyco:
real 0m56.130sPsyco chops more than a third off the time, giving almost 60% more pages per second. Not bad for a free compiler that requires no code changes! Particularly with this sort of application, which is mostly database accesses and text manipulation, things compilers generally can't help you with.
Note 1: These timings were run against Minx 0.4.03X, not the current 0.4.02A. My tests with Psyco uncovered a performance problem which I will have to fix in the main codebase, which was costing whole milliseconds for each page generated. Which isn't significant, but it would have got worse as the database size grew. Fortunately, there is a fairly straightforward fix.
Saturday, April 02
Daniel Henninger has an article in today's Opinion Journal (an online offshoot of the Wall Street Journal) discussing the Metro-Goldwyn-Mayer v. Grokster case currently before the Supreme Court. The article is called Wonder Land.
In case you haven't heard of Grokster, it's just one of the new crop of file-sharing systems. The all work the same way, basically: You put files on your computer, and you share them with other people.
Which is fine and wonderful and exactly what the internet was designed for (all this email and web stuff was just a hack added on later). The problem is copyright, or more specifically, that people are sharing files that they don't have permission to share.
The music industry (represented most notably by the RIAA) and to a lesser extent the film and television industry (represented by the MPAA) have been doing their level best to stamp this practice out. They've been trying for years, and they have made some small progress; at the least, they have splintered the file-sharing networks into many small segments, and discouraged businesses from trying to make a profit out of aiding and abetting copyright infringement.
I'm a writer, a programmer, and a musician, so my income depends to at least some degree on the protections of copyright law. (These days I work as an in-house programmer and not for a software publisher, so it's more a matter of trade secrets than copyright, but we'll leave that point for the moment.) So as you'd expect, I'm firmly on the side of the file-sharers.
The reason being, whatever the technical and legal merits of the case, the recording industry is a corporate cancer devoted to ripping off everybody: Musicians, songwriters, the public. Well, not politicians or "music industry" execs, but all the people who actually matter in the process. They have been paranoid about file-sharing and savage in their attacks (verbal and legal) precisely because they no longer add any value. Their time is done, and they are merely forestalling the day the doors are closed for good so they can keep looting the coffers for a little while longer.
This does not apply quite so much to the film and television industry: The big studios are still necessary, they don't always screw everybody, and they have been more circumspect in their approach to file-sharing (though still fundamentally opposed to it).
Further, most downloaders would likely concede that in a royalty-free world the incentives for the next Dylan diminish. Even writers gotta eat. But this means one has to buy into the validity of eeeek, "profit." I would push this even further; it requires a moral or at least philosophical commitment to the legitimacy of profit. Absent that, there's no hope.Most downloaders (not all, I admit) accept and indeed approve of profit. What they don't approve of is being ripped off. A CD costs rather less than a dollar to produce; we know that the musician is not (with very few exceptions) getting rich of the CD sales. We now, in fact, where the music is going, and we know that the recording industry has set itself up with a tightly ruled little monopoly, an empire of greed. And we don't appreciate that.
I've got about 600 CDs here, about 500 DVDs, and as many more VHS tapes and laser discs, so I'm not exactly a freeloader. But when unit costs are going down and retail prices are going up, people object.
What a weird ethic. Some who will spend hundreds of dollars for iPods and home theater systems won't pay one thin dime for a song or movie. So Steve Jobs and the Silicon Valley geeks get richer while the new-music artists sweating through three sets in dim clubs get to live on Red Bull. Where's the justice in that?Ask Janis Ian. Even with CD prices the way they are and file-sharing stamped out for good, most new-music artists would still be sweating through three sets in dim clubs for a percentage of the bar tab. Thus it ever was. And we can accept that. What we don't accept is a recording industry that is still structured as though it were the 1950s. If there were any actual competition, most of the record companies would have gone bankrupt years ago.
But Henninger also says this:
For starters, if "the people" don't solve this problem themselves, Congress will, and you won't like the solution--unless you enjoy the tax code. Try Googling "Chapter 17 Federal Code Copyrights." Then click on any of its 13 chapters or any of Sections 101 through 1332. It can get worse.He has no idea how much worse.
The only way to really block file sharing is to ban everything that can be used for file sharing. That includes - just for starters - every computer, every tape recorder, every VCR, every video camera, every microphone in existence today. Every last one has to be seized and destroyed; every new product sold has to be designed from the ground up so that it cannot possibly be used to share files. The requirements for enforcing this are so odious that they'd make the tax code look like Dr Seuss.
There is a way around it.
First, stop screwing around with DRM (digital rights management). Until every electronic device in the world has been destroyed under the new RIAA World Government, DRM can always be broken. All it does is annoy people because they have to go and break the DRM before they can do what they want with the music (or film) they have just bought. Like, in many cases, listen to it.
Second, stop charging those absurd prices. A buck a track? You're dreaming. Your unit cost was less than that when you were selling CDs; your cost now is zero. A buck an album. Two bucks for a feature film. Maybe five bucks for a season of a TV show.
The rental places will scream at the latter, but hey, they're dinosaurs too. They're not parasites like the recording industry, it's just that they found a niche and now it's disappearing. Nothing in the constitution* guarantees you profits in perpetuity.
Make it quick and easy and convenient to buy the things. Offer subscriptions so that people can have the latest episodes of their favourite shows delivered straight to their digital home entertainment centre. Make it so that it's just not worth the bother of going out and finding a stolen copy.
Because most people want to be honest. They're just not willing to be robbed. And stop trying to force people to buy the same thing twice.
* Yours or mine.
53 queries taking 0.3618 seconds, 318 records returned.
Powered by Minx 1.1.6c-pink.