It Just Worked Out That Way

Just to clear one thing up - munu is not a commercial service. It's not just non-profit, it's non-revenue. That may change in the future, but at the moment we have a gross profit of zilch on an annual turnover of nothing.

Now, on the subject of web spiders: By default, Movable Type 2.6 creates CGI links for the comments and trackbacks for every post.

We have over 160,000 posts here at munu. (Just in MT 2.6.)

Back in November when we moved servers, we had about 140,000 posts. As it happens, when you dump and reload a MySQL database it loses all its indexing statistics. And with no statistics, it simply doesn't use the indexes.

So along came a spider, and decided to index all 280,000 CGI pages against a database with no working indexes causing a near meltdown of a pair of dual-processor 2.8GHz servers.

Web spiders are now banned from our CGI directory.

Posted by: Pixy Misa at 08:10 PM | Comments (1) | Add Comment | Trackbacks (Suck)
Post contains 168 words, total size 1 kb.