Tuesday, January 30
Got about sixty miles out of Melbourne when I realised that I'd forgotten to re-pack my toiletries bag. I'd checked and double-checked the important stuff: wallet, keys, iPod, phone, camera. The laptop is a too big and heavy to easily forget.
So I stopped at the supermarket on the way home and got a hairbrush and a razor; everything else I have spares of.
And then I finally got home and dumped out my backpack and there it was. Okay, I forgot the washcloth, so I'm not totally bewildered, but I somehow remembered everything else despite the best efforts of my two older nephews (three and five; the youngest was blissfully asleep with mummy.)
In a word: Back.
Thursday, January 25
I'm heading down to Melbourne for the Australia Day long weekend to visit my family. As usual, I have about three days worth of stuff I need to do before I leave, and I have to catch a train in, oh, five hours.
I expect to sleep most of the way down...
Now where did I put those Sugar DVDs?
Tuesday, January 23
Saw this over at Ace's, and thought That needs to be set to The Blue Danube. Ask, and apparently you will receive:
Got New Minx up and running on Kasumi:
Processing 0.05, elapsed 0.0557 seconds.Cf. Martina:
42 queries taking 0.0192 seconds, 138 records returned.
Page size 98 kb.
Powered by Minx 0.81.19a.
Processing 0.09, elapsed 0.1028 seconds.So, not quite twice as fast, and hence not quite back to the speeds of Old Minx on Nabiki.
42 queries taking 0.0383 seconds, 138 records returned.
Page size 98 kb.
Powered by Minx 0.81.19a.
I do have a solution for this: Replace the template interpreter with a compiler that custom-builds all the database queries. And I might even do that at some point, but not today. 55 milliseconds is something I can live with.
Oh, and the perennial favourite:
Retrieved from cache, processing 0.0, elapsed 0.0019 seconds.I always like seeing that one.
Page size 98 kb.
Powered by Minx 0.81.19a.
Monday, January 22
This is updated from my previous post. I now have three new servers to play with, so let's see how they go.
LinuxKasumi, Ukyo and Shampoo are Xeon 3060s: dual-core 64-bit 2.4GHz
Akane and Nabiki are Opteron 170s: dual-core 64-bit 2.0GHz.
Ranma is an Athlon XP 3000+: single-core 32-bit 2.16GHz.
Martina is an Athlon XP 2800+: single-core 32-bit 2.08GHz.
Naga is an Athlon 64 3200+: single-core 64-bit 2.0GHz.
Namo was a P4 Celeron: single-core 32-bit 1.7GHz
WindowsLina: Pentium 4 2.6GHz
Amelia: Core Duo 1.66GHz
Haruhi: Core 2 Duo 2.4GHz
|Lina||Pentium 4||2.6GHz||2.5 (Win)||2.038||5.058||0.875||7.971|
|Haruhi||Core 2 Duo||2.4GHz||2.5 (Win)||0.644||1.933||0.477||3.053|
|Amelia||Core Duo||1.66GHz||2.5 (Win)||1.243||3.158||1.033||5.434|
|Haruhi||Core 2||2.4GHz||2.5 (Win)+Psyco||0.012||0.273||0.554||0.839|
I'm not sure where the difference between the Linux and Windows versions comes from; I'm guessing that Psyco would still be using Python's string libraries, and they're compiled using a different (better) compiler on Windows, perhaps Intel's. I'm still using GCC 3.4.6 (which is what CentOS installs); I might be able to do better with GCC 4.1, and I'll probably try that at some point.
I can't easily scan that inside page, but she's on the back cover as well. Just not quite so much of her...
Sunday, January 21
Had lunch today with Pete Zaitcev, who's been in Sydney the past few days for Linux Conference Australia. (And who I see made it to a computer before I did. I went grocery shopping on the way home, then my Coca-Cola exploded*, so I was distracted for a bit.)
Pete very generously presented me with a DVD set of Serial Experiments Lain, which I have had on VHS tape since it first came out, but have never watched. Thanks Pete!
Then we hit Kinokuniya, where we wandered over to the Japanese language section. Usually I don't go past the art books, because I can't read a word of Japanese, but since Pete can, over we wandered.
And it's a good thing for my wallet that I can't read Japanese, because they had all 40 volumes of 3x3 Eyes.**
I picked up a couple of things: the latest volume of Battle Club, from the creator of Ikki Tousen, and the Crest/Banner of the Stars manga Pete mentions.
Which look like this:
Is that not how Lafiel normally looks, then?
* It was supercooled - I guess my fridge is turned down a bit too low - so when I opened it about a quarter of it froze instantly and another quarter squirted out all over the kitchen.
** How do you pronounce that, anyway?
SoftLayer are running a sale this weekend, and it looks like I'll be picking up three Xeon 3060 servers, with a total of 10GB of memory and 2.5TB of disk.
That's a whole lot of fast. It's equivalent in terms of CPU, memory and disk alike to five more of the Opteron servers we have right now.
I'll be bringing forward the commissioning of Kasumi by two months, but the deal I'm getting works out equal to two months free over the first year, so effectively I get a development and beta-test box for two months at no extra charge.
Saturday, January 20
I've been running some more benchmarks on Minx.
The page you see here takes around 40ms to produce.
The same page on my test server, on essentially the same version of Minx, takes around 60ms.
The same page on the new version of Minx takes around 95ms.
One interesting point there is that although Nabiki (the server) and Martina (my test box) deliver the same results in trivial Python benchmarks, the actual application runs significantly faster on Nabiki. Those same trivial benchmarks indicate that the Xeon 3060 servers I'm planning to deploy on will be 60% faster than Nabiki. I'm very curious now to see how that translates to the real application.
On a more depressing note, jumping from 23 fields per entry to 118 (I think those counts are correct) has a real and noticeable impact on performance. And I still have five tables to add to that join...
The performance hit doesn't seem to be on the MySQL side. At least, once something is in the query cache, it coughs up the data more or less instantly. Rather, it's the Python DB library handling all the fields. It takes time. On my test box, Python can retrieve from MySQL and stash into native data structures about 300,000 fields per second. That puts a hard limit on how fast Minx can run; the time taken to run the queries on New Minx on Martina is about 85% of the total time taken to generate the page on Old Minx on Nabiki. I'm going to keep on working to improve performance, but it looks like I'm fighting Amdahl's Law here.
My goal was to deliver complex pages in under 100ms and simple ones in 10; that's still achievable unless I really break something, but I'm going to have to borrow some of that performance from the new hardware.
Thursday, January 18
Processing 0.02 seconds.(Does happy dance.)
7 queries taking 0.0122 seconds, 29 records returned.
Page size 22 kb.
Powered by Minx 0.80 alpha.
The production version of Minx is up and running.
As I was slogging through it, and through the enormous lists of fields that the new version provides, I was thinking this is not going to be fast.
Well, a page of 1000 posts comes up in 220 milliseconds (with Psyco; 300ms without). That doesn't have inline comments, because I've broken the comment handler, but I would hope you aren't using inline comments if you have 1000 posts per page...
That's not slow.
Update: Except it didn't actually include 1000 posts. Bleh. Bug hunt time.
Update: Ah, it's a not-a-bug. It's returning 255 posts. Hint hint. Okay, I'll just tweak that...
Processing 0.71, elapsed 0.8462 seconds.Hmm. Anyway...
7 queries taking 0.4891 seconds, 1004 records returned.
Page size 1452 kb.
Powered by Minx 0.81.18 alpha.
Approximately 20ms for a normal page of 25 posts. You can see that this blog takes rather longer than that; that's the effect of the inline comments, which are dynamically sanitised in the old version. The new version does static sanitisation, which should eliminate much of that performance penalty. (Also, I'm not using Psyco on the server, because I ran into a memory leak when I tried it; that's since resolved.)
So I'm through restructuring the code and data.* Now the fun begins!
* Mostly. Still lots of tweaking to do, but the heavy lifting is done.
Tuesday, January 16
Now, weave in our recent thread about consumer eugenics and designer babies. If consumer eugenics becomes cheap and ubiquitous, as I suspect it will, won't religious people want their offsprings' genes tweaked to make them religious, too? With the result, if those differential birthrates hold up, that the world will become more religious generation by generation?I can think of few more things more damaging to religion that the discovery that it is even partly genetic in origin and readily manipulable. If your belief system can be set by simple genetic tweaking, as Derbyshire would have it, then it seems to be of no more significance than blue eyes or brown.
I'm not greatly concerned by this, but I think that those in favour of religion should be.
Sunday, January 14
I have 120 Apache threads running non-stop and bandwidth up 100% over normal, and who do I have to thank for it?
Idiots searching for pictures of Jessica Alba and Elisha Cuthbert and ending up at the Jawas.
Saturday, January 13
It's just a phone.
I've been down with a cold all week, which is why I've been less active than normal. I'm trying to finish translating Minx from the proof-of-concept model to the production design, but that's hard to do when your head is all squishy.
Worse, I think the cough that took me out for three weeks last October is coming back. Well, it's nothing like as bad yet, but it feels similar. Or maybe it's just a cough; I don't know.
I'd go back to bed except that it's nearly 30 degrees here at Pixy Central even with the air conditioner running non-stop.
I'd eat some ice cream except that seems to aggravate the cough.
I'd drink some of my nice iced tea except I already drank it all.
I'd go to the shop and buy some more except that it's even hotter outside, and as I said, my head is all squishy.
Meanwhile, I was doing some testing, converting all the munu blogs to Minx using my automagical converter program. Two problems cropped up:
First, it took 50 minutes to convert everything, which seemed a bit slow. It's less than 15 seconds per blog, and that involves pulling the data out of the old database as well as putting it into the new one (this is a direct db-to-db transfer rather than an export/import function) - but that's still slower than I'd hoped.
Second, the resulting database was nearly twice the size of the original Movable Type one. I know I have a lot of new fields to account for the extra functionality, but I was very careful about how and when and where they were added, so I was extremely disappointed to see that they caused that much bloat. Essentially, that would mean I'd need twice the memory on the database servers to handle the same number of users, which makes for a lot more expense.
And then I discovered a leetle bug in the conversion program. That caused it to copy each blog twice. Which would account for 50% of the slow and 100% of the bloat.
My head feels a bit better now.
Update: Well, it fixed the size problem. The speed problem, not so much. The code isn't the issue; 45 minute run time vs. 4 minutes CPU. I'll move on to the import routine and see if it's the pull/push nature that's slowing it down.
Update: The dread cough of doom seems to be in hiding. Let's hope it stays that way. It's also set to be a lot cooler the next few days, with predicted maximums of 23 to 23 degrees, where today it was as I said 30 degrees indoors with the air conditioner running...
Now if I could just lose the headache, I'd be almost happy.
Friday, January 12
Steven: Some of the issues you raise are major plot points. Some of them aren't, but are nonetheless resolved satisfactorily by the end of the series. Some of them are ignored as the action heats up, but some of them come to the fore just as the action heats up.
And some are never really answered.
Just so you know.
62 queries taking 0.1006 seconds, 317 records returned.
Powered by Minx 1.1.6c-pink.