Saturday, January 20


Need More Fast!

I've been running some more benchmarks on Minx.

The page you see here takes around 40ms to produce.

The same page on my test server, on essentially the same version of Minx, takes around 60ms.

The same page on the new version of Minx takes around 95ms.

One interesting point there is that although Nabiki (the server) and Martina (my test box) deliver the same results in trivial Python benchmarks, the actual application runs significantly faster on Nabiki. Those same trivial benchmarks indicate that the Xeon 3060 servers I'm planning to deploy on will be 60% faster than Nabiki. I'm very curious now to see how that translates to the real application.

On a more depressing note, jumping from 23 fields per entry to 118 (I think those counts are correct) has a real and noticeable impact on performance. And I still have five tables to add to that join...

The performance hit doesn't seem to be on the MySQL side. At least, once something is in the query cache, it coughs up the data more or less instantly. Rather, it's the Python DB library handling all the fields. It takes time. On my test box, Python can retrieve from MySQL and stash into native data structures about 300,000 fields per second. That puts a hard limit on how fast Minx can run; the time taken to run the queries on New Minx on Martina is about 85% of the total time taken to generate the page on Old Minx on Nabiki. I'm going to keep on working to improve performance, but it looks like I'm fighting Amdahl's Law here.

My goal was to deliver complex pages in under 100ms and simple ones in 10; that's still achievable unless I really break something, but I'm going to have to borrow some of that performance from the new hardware.

Posted by: Pixy Misa at 11:26 AM | Comments (5) | Add Comment | Trackbacks (Suck)
Post contains 314 words, total size 2 kb.

1 Have you looked at valgrind to help in debugging performance issues ?

Just got back from myself. Lots of informative talks and lots of fun.

Posted by: Andrew at Monday, January 22 2007 07:18 AM (81C4m)

2 I hadn't thought of trying valgrind; I always thought of it as a C debugging tool.  They say it works with any language, so I'll give it a whirl and see what it says.

Posted by: Pixy Misa at Monday, January 22 2007 12:50 PM (GaSFI)

3 Well, there you go.

It segfaults.

Posted by: Pixy Misa at Monday, January 22 2007 01:01 PM (GaSFI)

4 Funny. It worked in the demo. :)

It segfaulted with you ran it ?

Posted by: Andrew at Monday, January 22 2007 10:26 PM (7Q1Mu)

5 Yeah, it started fine with Minx and then dropped dead on the first page request.

It's almost certainly because I'm running with Psyco, which is a JIT compiler.  But if I turn Psyco off, all the timings change.  Psyco speeds up loops and numeric processing by as much as 100x, and string processing only about 2x, so it makes a huge difference when you're working out what to optimise.

Posted by: Pixy Misa at Monday, January 22 2007 10:39 PM (GaSFI)

Hide Comments | Add Comment

Comments are disabled. Post is locked.
47kb generated in CPU 0.0207, elapsed 0.3948 seconds.
56 queries taking 0.3802 seconds, 344 records returned.
Powered by Minx 1.1.6c-pink.