Oh, lovely, you're a cheery one aren't you?
Tuesday, January 30
Got about sixty miles out of Melbourne when I realised that I'd forgotten to re-pack my toiletries bag. I'd checked and double-checked the important stuff: wallet, keys, iPod, phone, camera. The laptop is a too big and heavy to easily forget.
So I stopped at the supermarket on the way home and got a hairbrush and a razor; everything else I have spares of.
And then I finally got home and dumped out my backpack and there it was. Okay, I forgot the washcloth, so I'm not totally bewildered, but I somehow remembered everything else despite the best efforts of my two older nephews (three and five; the youngest was blissfully asleep with mummy.)
In a word: Back.
Thursday, January 25
I'm heading down to Melbourne for the Australia Day long weekend to visit my family. As usual, I have about three days worth of stuff I need to do before I leave, and I have to catch a train in, oh, five hours.
I expect to sleep most of the way down...
Now where did I put those Sugar DVDs?
Tuesday, January 23
Saw this over at Ace's, and thought That needs to be set to The Blue Danube. Ask, and apparently you will receive:
Got New Minx up and running on Kasumi:
Processing 0.05, elapsed 0.0557 seconds.Cf. Martina:
42 queries taking 0.0192 seconds, 138 records returned.
Page size 98 kb.
Powered by Minx 0.81.19a.
Processing 0.09, elapsed 0.1028 seconds.So, not quite twice as fast, and hence not quite back to the speeds of Old Minx on Nabiki.
42 queries taking 0.0383 seconds, 138 records returned.
Page size 98 kb.
Powered by Minx 0.81.19a.
I do have a solution for this: Replace the template interpreter with a compiler that custom-builds all the database queries. And I might even do that at some point, but not today. 55 milliseconds is something I can live with.
Oh, and the perennial favourite:
Retrieved from cache, processing 0.0, elapsed 0.0019 seconds.I always like seeing that one.
Page size 98 kb.
Powered by Minx 0.81.19a.
Monday, January 22
This is updated from my previous post. I now have three new servers to play with, so let's see how they go.
LinuxKasumi, Ukyo and Shampoo are Xeon 3060s: dual-core 64-bit 2.4GHz
Akane and Nabiki are Opteron 170s: dual-core 64-bit 2.0GHz.
Ranma is an Athlon XP 3000+: single-core 32-bit 2.16GHz.
Martina is an Athlon XP 2800+: single-core 32-bit 2.08GHz.
Naga is an Athlon 64 3200+: single-core 64-bit 2.0GHz.
Namo was a P4 Celeron: single-core 32-bit 1.7GHz
WindowsLina: Pentium 4 2.6GHz
Amelia: Core Duo 1.66GHz
Haruhi: Core 2 Duo 2.4GHz
|Lina||Pentium 4||2.6GHz||2.5 (Win)||2.038||5.058||0.875||7.971|
|Haruhi||Core 2 Duo||2.4GHz||2.5 (Win)||0.644||1.933||0.477||3.053|
|Amelia||Core Duo||1.66GHz||2.5 (Win)||1.243||3.158||1.033||5.434|
|Haruhi||Core 2||2.4GHz||2.5 (Win)+Psyco||0.012||0.273||0.554||0.839|
I'm not sure where the difference between the Linux and Windows versions comes from; I'm guessing that Psyco would still be using Python's string libraries, and they're compiled using a different (better) compiler on Windows, perhaps Intel's. I'm still using GCC 3.4.6 (which is what CentOS installs); I might be able to do better with GCC 4.1, and I'll probably try that at some point.
I can't easily scan that inside page, but she's on the back cover as well. Just not quite so much of her...
Sunday, January 21
Had lunch today with Pete Zaitcev, who's been in Sydney the past few days for Linux Conference Australia. (And who I see made it to a computer before I did. I went grocery shopping on the way home, then my Coca-Cola exploded*, so I was distracted for a bit.)
Pete very generously presented me with a DVD set of Serial Experiments Lain, which I have had on VHS tape since it first came out, but have never watched. Thanks Pete!
Then we hit Kinokuniya, where we wandered over to the Japanese language section. Usually I don't go past the art books, because I can't read a word of Japanese, but since Pete can, over we wandered.
And it's a good thing for my wallet that I can't read Japanese, because they had all 40 volumes of 3x3 Eyes.**
I picked up a couple of things: the latest volume of Battle Club, from the creator of Ikki Tousen, and the Crest/Banner of the Stars manga Pete mentions.
Which look like this:
Is that not how Lafiel normally looks, then?
* It was supercooled - I guess my fridge is turned down a bit too low - so when I opened it about a quarter of it froze instantly and another quarter squirted out all over the kitchen.
** How do you pronounce that, anyway?
SoftLayer are running a sale this weekend, and it looks like I'll be picking up three Xeon 3060 servers, with a total of 10GB of memory and 2.5TB of disk.
That's a whole lot of fast. It's equivalent in terms of CPU, memory and disk alike to five more of the Opteron servers we have right now.
I'll be bringing forward the commissioning of Kasumi by two months, but the deal I'm getting works out equal to two months free over the first year, so effectively I get a development and beta-test box for two months at no extra charge.
Saturday, January 20
I've been running some more benchmarks on Minx.
The page you see here takes around 40ms to produce.
The same page on my test server, on essentially the same version of Minx, takes around 60ms.
The same page on the new version of Minx takes around 95ms.
One interesting point there is that although Nabiki (the server) and Martina (my test box) deliver the same results in trivial Python benchmarks, the actual application runs significantly faster on Nabiki. Those same trivial benchmarks indicate that the Xeon 3060 servers I'm planning to deploy on will be 60% faster than Nabiki. I'm very curious now to see how that translates to the real application.
On a more depressing note, jumping from 23 fields per entry to 118 (I think those counts are correct) has a real and noticeable impact on performance. And I still have five tables to add to that join...
The performance hit doesn't seem to be on the MySQL side. At least, once something is in the query cache, it coughs up the data more or less instantly. Rather, it's the Python DB library handling all the fields. It takes time. On my test box, Python can retrieve from MySQL and stash into native data structures about 300,000 fields per second. That puts a hard limit on how fast Minx can run; the time taken to run the queries on New Minx on Martina is about 85% of the total time taken to generate the page on Old Minx on Nabiki. I'm going to keep on working to improve performance, but it looks like I'm fighting Amdahl's Law here.
My goal was to deliver complex pages in under 100ms and simple ones in 10; that's still achievable unless I really break something, but I'm going to have to borrow some of that performance from the new hardware.
Thursday, January 18
Processing 0.02 seconds.(Does happy dance.)
7 queries taking 0.0122 seconds, 29 records returned.
Page size 22 kb.
Powered by Minx 0.80 alpha.
The production version of Minx is up and running.
As I was slogging through it, and through the enormous lists of fields that the new version provides, I was thinking this is not going to be fast.
Well, a page of 1000 posts comes up in 220 milliseconds (with Psyco; 300ms without). That doesn't have inline comments, because I've broken the comment handler, but I would hope you aren't using inline comments if you have 1000 posts per page...
That's not slow.
Update: Except it didn't actually include 1000 posts. Bleh. Bug hunt time.
Update: Ah, it's a not-a-bug. It's returning 255 posts. Hint hint. Okay, I'll just tweak that...
Processing 0.71, elapsed 0.8462 seconds.Hmm. Anyway...
7 queries taking 0.4891 seconds, 1004 records returned.
Page size 1452 kb.
Powered by Minx 0.81.18 alpha.
Approximately 20ms for a normal page of 25 posts. You can see that this blog takes rather longer than that; that's the effect of the inline comments, which are dynamically sanitised in the old version. The new version does static sanitisation, which should eliminate much of that performance penalty. (Also, I'm not using Psyco on the server, because I ran into a memory leak when I tried it; that's since resolved.)
So I'm through restructuring the code and data.* Now the fun begins!
* Mostly. Still lots of tweaking to do, but the heavy lifting is done.
58 queries taking 0.5875 seconds, 378 records returned.
Powered by Minx 1.1.6c-pink.