What happened?
Twelve years!
You hit me with a cricket bat!
Ha! Twelve years!

Monday, April 26

Geek

OurSQL

Upgraded the MuNu MT database to MySQL today, something I should have done long ago. If you're going to run Movable Type, it's definitely worth your time to use MySQL or PostgreSQL (or I think SQLite is also supported). For a brand new blog there isn't much difference, but on a very large installation like MuNu, a SQL* database can be as much as 10 times faster.

With this on top of the Minxification of Ambient Irony, I now have some of the fastest MT comments around. Leave me a comment, and marvel in the fastness of it! It's almost... adequate!

* I say "a SQL database" rather than "an SQL database" because in my head I pronounce SQL as "skwuhl".

Posted by: Pixy Misa at 07:56 AM | Comments (21) | Add Comment | Trackbacks (Suck)
Post contains 123 words, total size 1 kb.

Tuesday, April 20

Geek

Who Let The Brap Out?

I can't for the life of me work out why the mt-db2sql.cgi program is so slow. I put both the Berkeley DB (the old database) and the MySQL DB (the new database) in a ram disk, and it behaved exactly the same. Moofleglerp.

I'm familiar with Berkeley DB, and I've never seen it behave like that. I'm less familiar with MySQL, but I've never seen MySQL behave like that either. Right now I'm trying it with PostgreSQL, and any moment now... Crunch! As soon as it finishes with the comments and starts transferring the entries, performance drops to zero. It's not CPU bound, it doesn't seem to be I/O bound, it just sucks.

Well, I'll let it finish anyway, and then I'll have more numbers for comparison.

Posted by: Pixy Misa at 06:06 AM | Comments (1) | Add Comment | Trackbacks (Suck)
Post contains 137 words, total size 1 kb.

Sunday, April 18

Geek

Who Writes This Brap?

(Yeah, that's a typo, but I decided to leave it.)

As I've noted before, Movable Type is robust and rich in features, but really really slow.

I've wondered if it's something to do with using Berkeley DB. Some people on the MT support forum have claimed this is the case, but they have not impressed me as particularly knowledgable. Berkeley itself is very fast anyway, so it would have to be a problem with MT's use of it rather than Berkeley itself - though that is quite possible.

Anyway, there's a CGI script available in MT to convert from a Berkeley DB to MySQL or PostgreSQL. Two things to note: It only comes with the upgrade package, and not with the "full" version (minor oops), and it doesn't work (big oops).

Well, if you run it as the docs suggest, as a CGI program from your web browser, it will happily create the necessary tables in your SQL database and report that its work is done. Without, mind you, copying any of the data across.

If instead you run it from the command line, it will copy all your data, only very very slowly. It zips through the first part - I'm not sure exactly what that consisted of, but at least it was quick - but when it starts processing the entries it slows to a crawl. Crawwwwwwwwwl.

Once it's finished, which should be within the next 10 or 12 hours (seriously!), I'll rerun my little stress-test.

Oh, yes: I've written a template which exports your entire Movable Type system in a nice convenient format. Just the ticket if you want to move off MT and onto a more modern and efficient sytem. (Cough Minx cough.)

It takes two hours to run on mu.nu. That's on a lightly-loaded Athlon XP 2500+ with 1GB of memory.

Pfft. But at least it's better than mt-db2sql.cgi.

Posted by: Pixy Misa at 06:24 AM | Comments (16) | Add Comment | Trackbacks (Suck)
Post contains 322 words, total size 2 kb.

Wednesday, April 07

Geek

Open Sourceme

If you're looking for an open-source CMS - that's content management system, which means a program to manage your web site - you could do a lot worse than to browse around opensourceCMS. They have (counts) 33 portals, 7 blogging systems, 3 e-commerce packages, 7 groupware systems, 5 forums, 3 "e-learning" packages and 4 other thingies all set up ready for you to test drive.

Unfortunately, they only have packages written in PHP, which is not my favourite langauge (and far from my favourite programming paradigm), but if you're looking for something useful rather than something elegant, it's a darn good place to do a bit of tyre-kicking.

I'm looking for a package that integrates blogging, forum, and portal functions and doesn't suck. So far I haven't found one, which is why I'm writing Minx. Although to be honest recently it's been more of a case of not writing Minx. Sigh.

Update: A number of the packages seem to have had their tyres kicked rather too thoroughly and aren't working right now. They get refreshed (says the site) every hour, so maybe they'll come back soon.

Posted by: Pixy Misa at 11:52 AM | No Comments | Add Comment | Trackbacks (Suck)
Post contains 190 words, total size 1 kb.

Tuesday, April 06

Geek

MT Zoom

I spent a couple of hours today seeing if I could take one of the tricks I've used in the blogging system I'm developing, Minx, and apply it to MT. It does work, but it's not the solution to all life's problems.

MT is robust and has lots of good features, but it's as slow as a dead dog. For example, on an Athlon XP 2800+, with both the code and the database stored in a ram disk, it takes around 28 seconds of CPU time to add a three-word post to Ambient Irony. I haven't yet had time to profile MT to work out what the hell it's doing with all that time. Part of the problem is MT's insistence on static files and its brute-force approach: Whenever you update a post or add a comment, MT will rebuild not only the individual entry, but also the main index, the archive for whatever categories that post is in, and the archives for that day, week and month, and the archives for the previous day, week and month, and, if they exist, the archives for the following day, week and month.

And MT is slow at just rebuilding a single entry, so when it has to run around rebuilding everything in sight, it's really slow.

Without fixing the code, the only way I can see to cure this is to give it less work to do. One way to do this is to disable any archives you don't use. If you don't use categories, make sure that category archives are turned off.

The next thing you can do is to make the templates simpler. And here's where my little trick comes in.

If you have your blog set up like me, the appearance of the posts in the main index is exactly the same as in the monthly archives and the category archives. You could use a sub-template to keep the common data, but there's an even sneakier thing you can do:

Step 1.

Create a new template. Call it something like "Individual Include Template". Copy the section of your main template that deals with the individual entries into this new template. If you are using the default MT template, the result will be this:


<div class="blogbody">

<a name="<$MTEntryID pad="1"$>"></a>
<h3 class="title"><$MTEntryTitle$></h3>

<$MTEntryBody$>

<MTEntryIfExtended>
<span class="extended"><a href="<$MTEntryPermalink$>#more">Continue reading "<$MTEntryTitle$>"</a></span><br />
</MTEntryIfExtended>

<div class="posted">Posted by <$MTEntryAuthor$> at <a href="<$MTEntryPermalink$>"><$MTEntryDate format="%X"$></a>
<MTEntryIfAllowComments>
| <a href="<$MTCGIPath$><$MTCommentScript$>?entry_id=<$MTEntryID$>" onclick="OpenComments(this.href); return false">Comments (<$MTEntryCommentCount$>)</a>
</MTEntryIfAllowComments>
<MTEntryIfAllowPings>
| <a href="<$MTCGIPath$><$MTTrackbackScript$>?__mode=view&entry_id=<$MTEntryID$>" onclick="OpenTrackback(this.href); return false">TrackBack (<$MTEntryTrackbackCount$>)</a>
</MTEntryIfAllowPings>
</div>

</div>
Save the new template.
Step 2.

Now go into the Archiving page in Weblog Config. Click on the Add New button at the bottom, and create a new Individual Entry archive using the template you just created.

You will need to give the files created by this template a new and unique name. This is what the Archive File Template field is for. You will need to use MT tags here to generate the name. A good, simple name might be

<$MTEntryID>.i

Now, save your archive settings and do a full rebuild of your blog to generate all those little .i files, because in the next step you're going to need them.

Step 3.

Go into your Main Index template and (after making a backup copy), cut out all the code you put into your new "Individual Include Template" and replace it with this:

<?php  include 'archives/<$MTEntryID>.i'; ?>

(Assuming that your archive directory is called "archives", which it probably is.)

Save your changes, but don't rebuild yet. Go into the Preferences page in Weblog Config, and scroll down to where it says File extension for archive files This is probably set to html. You will need to change it to php.

And in the Templates page, select your Main Index template and change the output file to index.php.

Save the changes, and now go and rebuild your main index. If you got it right, it will now rebuild without having to process the details of the individual entries. The entries will instead be read automatically from their individual .i files whenever someone visits your blog.

Repeat step 3 for your daily, weekly, monthly and category archives, whichever ones you may be using, except that for these you will not need to specify the "archives" directory as it will look there automatically.

Step 4. (Optional)

Somewhere near the top of your main index, add the following code:

<?php

$timeparts = explode(' ',microtime());
$starttime = $timeparts[1].substr($timeparts[0],1); ?>
And in a suitable place near the end, add this:
<div class="sidetitle">
Render Time
</div>

<div class="side">
<?php
$timeparts = explode(' ',microtime());
$endtime = $timeparts[1].substr($timeparts[0],1);
echo bcsub($endtime,$starttime,6)?> seconds<br>
</div>

This will show you how long PHP takes to put all the individual entries together to build your page. A fairly typical result for Ambient Irony, which has 25 entries on the main page, is 0.005888 seconds. Which I must say is satisfyingly quick.

Having done all of that, is MT now zoomily fast? Alas, no. It will probably halve the time required to add a post or leave a comment, but it's still dog slow compared to what it should be capable of.

ExpressionEngine is quite a bit faster at processing posts and comments - at least ten times faster - but it's between thirty and a hundred times slower at delivering the pages. That's the main reason I decided not to go down that route.

This is basically how Minx works, except that the Minx template engine is about a hundred times faster than Movable Type's. It's just a pity that it's not finished yet. sad

Update:

Here's how to turn off the rebuilding of the index and archive files when someone leaves a comment.

The program that processes comments is called Comments.pm, and it lives in the lib/MT/App directory under whatever directory you have installed MT in.

Before making any changes, copy Comments.pm to Comments.bak.

In Comments.pm, look for the line
sub post {
This is the start of the posting routing that we will need to modify.

Now scroll down from there until you reach three line that read

$app->rebuild_indexes( Blog => $blog )
    or return $app->error($app->translate(
        "Rebuild failed: [_1]", $app->errstr));
Comment out these three lines by putting a # at the start of each line.

That's it.

If you are running MT-Blacklist, you will need to modify the code there instead. The file you want in this case is extlib/jayallen/Blacklist.pm. The code change required is identical.

If you only want to disable this rebuilding function on one blog, you can do it like this:

if ($comment->blog_id != 1) {
    $app->rebuild_indexes( Blog => $blog )
        or return $app->error($app->translate(
            "Rebuild failed: [_1]", $app->errstr));
}
This will cause comments to rebuild indexes and archives on all blogs except for blog number 1.

Posted by: Pixy Misa at 08:12 AM | Comments (2) | Add Comment | Trackbacks (Suck)
Post contains 1121 words, total size 8 kb.

<< Page 1 of 1 >>
69kb generated in CPU 0.06, elapsed 0.1686 seconds.
54 queries taking 0.1321 seconds, 304 records returned.
Powered by Minx 1.1.6c-pink.