Hah

I just finished replacing the SQL-based page caching in Minx with memcached. Which was actually dead easy, and works, and speeds things up not at all... Though memcached is putting about one-third the load on the server as MySQL in the same test.

Currently I'm seeing a time per page of 7.0 ms and 87 pages per second. The system load is about 60% Python, 2.3% memcached, 1.1% MySQL, and 33% idle. I can't seem to push it much higher than that with my current test framework.

Which tells me that:

1. It's plenty fast.
2. memcached uses no CPU, in the broad scheme of things.

The live deployment will have multiple instances of Minx per server, and load-balance across them as well as across the servers themselves. And the servers will be about 3.5 times as fast (comparing the Xeon 3060 to the Athlon XP 2800+).

Multiply it out and I'll get, hmm, about 465 pages per second.

As for the 7ms of overhead, well, even cached pages get passed through the template engine (because there are template tags that take effect after the page is generated) and it still has to check your login against the database (to see if you have any preference settings that would require a custom page to be presented) and so on. I'll try to keep the overhead down, of course, but that's not too terrible, considering.

Now I'll leave it to soak. Python is using 85MB at the moment (with 50 threads), but seems to be creeping up a bit. One odd thing with CherryPy is that the virtual space is much larger - currently 650MB. Which limits me to about 300 threads total per instance.

The other thing I noticed is that memcached was developed by the people at LiveJournal. LiveJournal has since been acquired by SixApart. SixApart are the Movable Type people. I originally started writing Minx just so I could get off Movable Type.

Well, I was amused.

Update: Hmm. 96MB now. Admittedly, it reached 70MB with 10 threads after a long soak, and I was expecting it to be on the order of 350MB this time.

As long as it stops growing eventually.

Update: 121MB and 8.2ms per page at the 130,000 page mark. That smells like a leak, but a pretty small one.

Posted by: Pixy Misa at 05:42 AM | Comments (1) | Add Comment | Trackbacks (Suck)
Post contains 388 words, total size 2 kb.