I just rewrote it.

I tested it on Ace's blog, by searching for "Bush". It's currently set to only search the last 500 entries, but for my test I set it to scan the last 5000.

It takes one second. And 19MB of memory. About 350ms for the SQL query and 650ms for the program itself.

It sure ain't optimised. It selects the last 5000 entries, sorts them (because God forbid there should be a useful compound index), yanks the entire result set into a list, scans them for each of the search terms, uses an SGML parser to remove HTML tags, and kicks the result out.

For more reasonable searches, like searching for "test" on the last 500 entries at Munuviana, it takes 35ms for the query and 130ms for the program.

About 120ms of that is start-up: Launching the Python interpreter and loading the seven or eight libraries involved.

Now, I'm not using a template system for this. Still, one-tenth the memory, one hundredth(?) the processing time. I can't be sure about the processing difference, because I can't run the original script right now. I configured Apache with a 100MB memory limit for CGI scripts.

For Minx, the 120ms start-up time wil disappear because the application runs as a multi-threaded server itself, not as individual CGI (or PHP or ASP) scripts. Can only do so much about the query time, but I'll play around with it. And I'll pre-store the excerpts rather than create them on the fly. Well, probably. I might be able to live with an average search time of 45ms.

Hmm.

What if I get MySQL to do the matching? Let's see...

Okay, not good. Hmm.

Ah, there we go. Don't use regexp's unless you need them. "LIKE" is nice and brisk. With a 500-result limit, the Ace/Bush search is 125ms for the SQL query and 300ms for the program. And doing it that way, I could actually page it, so 50 results at a time. Hmm. And when I'm taking the 50-word excerpts, rather than whitespace-split the entire entry, I'll just look at the first 400 bytes.

And let's go back and add that compound index while we're at it...

Okay, now we're cooking. 7ms for the query, 100ms for the search script. Since the resolution of the timer seems to be 10ms, and the search script takes 100ms if you feed it an invalid blog id, that's less than 10ms or so for the actual work.

That'll do.

Posted by: Pixy Misa at 03:47 AM | Comments (1) | Add Comment | Trackbacks (Suck)
Post contains 461 words, total size 3 kb.

Okay, the problem(s) this time:

If portsentry is running, you cannot start NFS. Solution: Stop portsentry, start portmap and nfsd, restart portsentry.

If sunrpc is not available as a kernel module, rpc.idmapd will not start. Without rpc.idmapd, you can't use NFSv4. Solution: Don't use NFSv4.

Everything is set up correctly, your mount requests are being accepted according to the syslog, and yet the client always sees "Permission denied". Solution: Add the line

none /proc/fs/nfs nfsd noauto,defaults 0 0

This, of course, is all explained in great detail in the man pages... err, the online howto's... err, in the kernel mailing list.

Posted by: Pixy Misa at 12:06 AM | Comments (6) | Add Comment | Trackbacks (Suck)
Post contains 128 words, total size 1 kb.

BUT I STILL GET SPAMMED.

Posted by: Pixy Misa at 02:04 AM | Comments (4) | Add Comment | Trackbacks (Suck)
Post contains 25 words, total size 1 kb.

They did not include migrating the whole of munu from one server to another to dodge a DoS attack, much less recovering from a hard disk failure during that migration.

So guess what I actually am doing.

Posted by: Pixy Misa at 10:39 PM | Comments (11) | Add Comment | Trackbacks (Suck)
Post contains 76 words, total size 1 kb.