No boom today. Boom tomorrow. There's always a boom tomorrow.
Saturday, March 31
Always In The Last Place You Look
I've been scratching my head trying to work out why the main query that drives Minx blogs is so inefficient. I tweak indexes and I tweak selection criteria and I tweak sorting and nothing seems to improve the query time reported at the bottom of the page. Now, normally it's only 20ms or so, but if you catch it after an update - a new post or comment - it blows out to 400ms or so because the query cache just got flushed.
So I tweaked and I twiddled and I fiddled and fretted, and I took the query and plugged in some parameters and ran it outside of Minx, and it turned out to be, in fact, reasonably quick.
Then I tried disabling the sidebar to see what effect that had on the total query time.
19ms cached; 28ms uncached. Perfectly reasonable for selecting and sorting and joining multiple fairly complex (albeit small) subsets of 13,000 records.
It's those darn recent comments again!
Update: And now I remember why recent comments are slow. When you list the recent comments, you're listing the last 20 (or whatever) comments in a folder. But comments aren't in folders, they're in threads, so you have to look at the threads in the folder, and then find the most recent comments from among the comments in those threads. Big slow join before the sort.
There's a straightforward way to optimise the common case - use either multiple queries or subqueries to restrict MySQL to only inspect the 20 most recently updated threads. That gets messy for the general case, though, where you want page 17 of recent comments.
Need to spend a bit more time thinking about this one.
Update: Gargh! MySQL 5.0.27 doesn't support LIMIT in subqueries anyway.
Update: Well, it doesn't if you're using IN to restrict the main search to a subset of threads selected by the subquery, but if you use the subquery to return the update datetime of the nth most recently updated thread and then restrict the search to threads updated since then, you're swimming in gravy.
...
Which would be pretty unpleasant, come to think of it.
I've been scratching my head trying to work out why the main query that drives Minx blogs is so inefficient. I tweak indexes and I tweak selection criteria and I tweak sorting and nothing seems to improve the query time reported at the bottom of the page. Now, normally it's only 20ms or so, but if you catch it after an update - a new post or comment - it blows out to 400ms or so because the query cache just got flushed.
So I tweaked and I twiddled and I fiddled and fretted, and I took the query and plugged in some parameters and ran it outside of Minx, and it turned out to be, in fact, reasonably quick.
Then I tried disabling the sidebar to see what effect that had on the total query time.
19ms cached; 28ms uncached. Perfectly reasonable for selecting and sorting and joining multiple fairly complex (albeit small) subsets of 13,000 records.
It's those darn recent comments again!
Update: And now I remember why recent comments are slow. When you list the recent comments, you're listing the last 20 (or whatever) comments in a folder. But comments aren't in folders, they're in threads, so you have to look at the threads in the folder, and then find the most recent comments from among the comments in those threads. Big slow join before the sort.
There's a straightforward way to optimise the common case - use either multiple queries or subqueries to restrict MySQL to only inspect the 20 most recently updated threads. That gets messy for the general case, though, where you want page 17 of recent comments.
Need to spend a bit more time thinking about this one.
Update: Gargh! MySQL 5.0.27 doesn't support LIMIT in subqueries anyway.
Update: Well, it doesn't if you're using IN to restrict the main search to a subset of threads selected by the subquery, but if you use the subquery to return the update datetime of the nth most recently updated thread and then restrict the search to threads updated since then, you're swimming in gravy.
...
Which would be pretty unpleasant, come to think of it.
Posted by: Pixy Misa at
06:13 AM
| Comments (19)
| Add Comment
| Trackbacks (Suck)
Post contains 367 words, total size 2 kb.
Catagrees
Categories. On the right. On the category icons. In the trees. They're everywhere! Eeeeee!!!
...
Sorry.
Fixed a couple of problems. The MT->Minx transfer marked posts as unpublished under their individual categories (sort of, it's complicated), so they showed on the main blog page, but not on the category pages. And a bug in the recent comments query meant that recent comments didn't show up at all on category pages.
Now, back to being driven crazy by this darn editor.
Categories. On the right. On the category icons. In the trees. They're everywhere! Eeeeee!!!
...
Sorry.
Fixed a couple of problems. The MT->Minx transfer marked posts as unpublished under their individual categories (sort of, it's complicated), so they showed on the main blog page, but not on the category pages. And a bug in the recent comments query meant that recent comments didn't show up at all on category pages.
Now, back to being driven crazy by this darn editor.
Posted by: Pixy Misa at
03:57 AM
| No Comments
| Add Comment
| Trackbacks (Suck)
Post contains 81 words, total size 1 kb.
Friday, March 30
Bad Surprises
I was back at my old job today, preparing for what I hope is one of the last phone company bill runs I ever have to do, when suddenly, at about 5:45 on a Friday afternoon,* the billing server dropped dead.
My first thought was Do we have backups? Yes, we have backups.** Okay then, let's haul the server out, plug it into a monitor (the computer room is not terribly well organised), and see what we get.
After a certain amount of fiddling I managed to get a BIOS screen up, and select the boot menu, whereupon the server spontaneously rebooted. I tried this several times with identical results; in fact, leaving the server alone would result in a boot-crash-boot-crash cycle just like my unlamented Compaq notebook.
There are four lights on the front of the server that indicate a POST code, so I hit Dell's website for the user manual and looked it up. Running down the list: CPU failure, memory failure, hard drive failure.***
USB failure.
Swapped the keyboard to a different USB port and it worked perfectly.
* There's some sort of physical law that requires this.
** It would take a couple of days to rebuild the system and restore the backups, but at least we had backups. From the previous day, I should add.
*** This is the mission-critical server that was spec'd with only one disk drive, so my first thought was exactly that.
I was back at my old job today, preparing for what I hope is one of the last phone company bill runs I ever have to do, when suddenly, at about 5:45 on a Friday afternoon,* the billing server dropped dead.
My first thought was Do we have backups? Yes, we have backups.** Okay then, let's haul the server out, plug it into a monitor (the computer room is not terribly well organised), and see what we get.
After a certain amount of fiddling I managed to get a BIOS screen up, and select the boot menu, whereupon the server spontaneously rebooted. I tried this several times with identical results; in fact, leaving the server alone would result in a boot-crash-boot-crash cycle just like my unlamented Compaq notebook.
There are four lights on the front of the server that indicate a POST code, so I hit Dell's website for the user manual and looked it up. Running down the list: CPU failure, memory failure, hard drive failure.***
USB failure.
Swapped the keyboard to a different USB port and it worked perfectly.
* There's some sort of physical law that requires this.
** It would take a couple of days to rebuild the system and restore the backups, but at least we had backups. From the previous day, I should add.
*** This is the mission-critical server that was spec'd with only one disk drive, so my first thought was exactly that.
Posted by: Pixy Misa at
08:42 PM
| Comments (1)
| Add Comment
| Trackbacks (Suck)
Post contains 243 words, total size 2 kb.
All Purpose Cultural Shopping Arcade Abenobashi
The music is good too.
The music is good too.
Posted by: Pixy Misa at
05:09 PM
| Comments (2)
| Add Comment
| Trackbacks (Suck)
Post contains 11 words, total size 1 kb.
Timing Is Everything
70ms for the main page without comments.
150ms with comments.
96 comments displayed (including the Recent Comments thingy).
Hmm. A bit disappointing, considering this is the fastest hardware I'm likely to get this year.
I just hope that Ace, who has 200+ comments on individual posts, doesn't try to do inline comments.
Update: Well, it's not broken, just a bit poky at certain tasks. An individual entry page takes 5.7ms (elapsed time) if you remove the Recent Comments. 2.9ms of that is the SQL queries. The forum page sans sidebar takes 54ms to list 20 posts; 6ms for the queries. (The query times are for cached queries; uncached takes longer in elapsed time, but that doesn't come into CPU efficiency of the application code.)
If I tweak the forum listing to include the body of the post, it takes exactly the same amount of time to within the limits of precision of the built-in timer. Which shouldn't be surprising I guess; all the data processing is done whether the template needs it or not, and Python's string processing is very efficient. It does point to where I need to concentrate my attention, though.
Update: Adding the planned [topics] selector and the associated [topic] data tags reduces the time for a forum page sans sidebar to 13ms. The sidebar adds another 13ms. Extending it from 20 to 50 topics pushes the time out to 42ms: about 500μs per topic, compared to about 2ms per entry.
70ms for the main page without comments.
150ms with comments.
96 comments displayed (including the Recent Comments thingy).
Hmm. A bit disappointing, considering this is the fastest hardware I'm likely to get this year.
I just hope that Ace, who has 200+ comments on individual posts, doesn't try to do inline comments.
Update: Well, it's not broken, just a bit poky at certain tasks. An individual entry page takes 5.7ms (elapsed time) if you remove the Recent Comments. 2.9ms of that is the SQL queries. The forum page sans sidebar takes 54ms to list 20 posts; 6ms for the queries. (The query times are for cached queries; uncached takes longer in elapsed time, but that doesn't come into CPU efficiency of the application code.)
If I tweak the forum listing to include the body of the post, it takes exactly the same amount of time to within the limits of precision of the built-in timer. Which shouldn't be surprising I guess; all the data processing is done whether the template needs it or not, and Python's string processing is very efficient. It does point to where I need to concentrate my attention, though.
Update: Adding the planned [topics] selector and the associated [topic] data tags reduces the time for a forum page sans sidebar to 13ms. The sidebar adds another 13ms. Extending it from 20 to 50 topics pushes the time out to 42ms: about 500μs per topic, compared to about 2ms per entry.
Posted by: Pixy Misa at
12:06 PM
| Comments (7)
| Add Comment
| Trackbacks (Suck)
Post contains 245 words, total size 2 kb.
Wednesday, March 28
Last Tags For Now
Simple, but neat:
[me] Pixy Misa
[you] you
[you=du] du
[me] picks up the author of the current post or comment. [you] gets your username if you're logged in, or the name you used on your most recent comment if it's still retained by the session code, or the text specified, or if all else fails, "you".
Simple, but neat:
[me] Pixy Misa
[you] you
[you=du] du
[me] picks up the author of the current post or comment. [you] gets your username if you're logged in, or the name you used on your most recent comment if it's still retained by the session code, or the text specified, or if all else fails, "you".
Posted by: Pixy Misa at
10:48 PM
| Comments (16)
| Add Comment
| Trackbacks (Suck)
Post contains 52 words, total size 1 kb.
Eeeevil!
Windows Update just croaked both my desktop and my notebook. I've spent the past hour in the Windows XP Repair thingy.
That little yellow blippy can sit there forever as far as I'm concerned. In fact, hey, c'mere. Automatic Updates Off. Little red blippy Always Hide.
Much better.
Windows Update just croaked both my desktop and my notebook. I've spent the past hour in the Windows XP Repair thingy.
That little yellow blippy can sit there forever as far as I'm concerned. In fact, hey, c'mere. Automatic Updates Off. Little red blippy Always Hide.
Much better.
Posted by: Pixy Misa at
03:43 PM
| Comments (8)
| Add Comment
| Trackbacks (Suck)
Post contains 49 words, total size 1 kb.
More BBCodes
Let's give these ones a whirl:
more...
Let's give these ones a whirl:
more...
Posted by: Pixy Misa at
10:05 AM
| No Comments
| Add Comment
| Trackbacks (Suck)
Post contains 79 words, total size 4 kb.
Haruhi-sama!
First, as Steven says, 2006 was a pretty blah year for anime, and Haruhi is a good show, with a solid story, strong cast, and high production values.
But second, and specific to Haruhi Suzumiya, is the effect of the re-ordering of the episodes. It's effective when you watch the episodes one after the other, but it's crazy-making when you watch them a week apart.
BBCode update one for the morning: You can now add links to the attribution in [quote] tags. Just separate the author name and the link with a pipe character, |. So in this case, the code was [quote="Steven|http://denbeste.nu"], producing
Steven wrote:Context is key, in two ways.Though it's good, it doesn't strike me as being that outstanding. I suspect that what I'm missing is context. What I suspect I'm missing is that the hooplah was from people who were comparing it to all the other shows that came out at the same time, and by comparison to the normal run-of-the-dreck series from the last year, it really did seem like a lightning strike.
First, as Steven says, 2006 was a pretty blah year for anime, and Haruhi is a good show, with a solid story, strong cast, and high production values.
But second, and specific to Haruhi Suzumiya, is the effect of the re-ordering of the episodes. It's effective when you watch the episodes one after the other, but it's crazy-making when you watch them a week apart.
BBCode update one for the morning: You can now add links to the attribution in [quote] tags. Just separate the author name and the link with a pipe character, |. So in this case, the code was [quote="Steven|http://denbeste.nu"], producing
Steven wrote:
Posted by: Pixy Misa at
09:17 AM
| Comments (5)
| Add Comment
| Trackbacks (Suck)
Post contains 176 words, total size 1 kb.
Return Of The Sticky
So that's why my honeypot wasn't catching any flies. I hadn't properly updated the code, and it was still expecting the Page object to be a global - which hasn't been true since the Great Rewrite back in January.
Without the Page object, it didn't have access to... well, much of anything, really. Including the database. So it never actually created any records.
Let's see what it does now...
Hmm. Still empty, but it hasn't been fixed very long.
Guess I should pretend to be a spammer and actually test it.
So that's why my honeypot wasn't catching any flies. I hadn't properly updated the code, and it was still expecting the Page object to be a global - which hasn't been true since the Great Rewrite back in January.
Without the Page object, it didn't have access to... well, much of anything, really. Including the database. So it never actually created any records.
Let's see what it does now...
Hmm. Still empty, but it hasn't been fixed very long.
Guess I should pretend to be a spammer and actually test it.
Posted by: Pixy Misa at
12:45 AM
| Comments (5)
| Add Comment
| Trackbacks (Suck)
Post contains 95 words, total size 1 kb.
Tuesday, March 27
BBCodarama
One of the features I've built in to Minx is BBCode support. You can use this in both posts and comments. At least, you should be able to use it in comments. We'll soon see.
more...
One of the features I've built in to Minx is BBCode support. You can use this in both posts and comments. At least, you should be able to use it in comments. We'll soon see.
more...
Posted by: Pixy Misa at
10:07 PM
| Comments (3)
| Add Comment
| Trackbacks (Suck)
Post contains 117 words, total size 5 kb.
Monday, March 26
How Much For Just The Planet?
Bugger.
I knew that Niven-style Ringworlds required materials with impossible tensile strengths (beyond anything known by quite a few orders of magnitude), but I wasn't sure about Banks-style orbitals. Orbitals, as well as being much smaller, have the attraction that they are in orbit about their sun and don't exhibit the dynamic instability of ringworlds.
So how big can we build an orbital? The answer is, not very.
Micro-scale samples of carbon nanotubes have apparently been tested with a tensile strength of 62GPa. That means they can withstand a tensile force of 62GN/m2* before breaking.
The density of said carbon nanotubes is 1.34g/cm3. Assuming we want an acceleration of 1G - let's round it up to 10m/s2 for simplicity - this gives us a maximum circumference of 62GPa / 13.4kN/m3 = 4600km.**
The theoretical maximum tensile strength for carbon nanotubes is 300GPa, which would give us 23,000km. But I'd want to have a safety factor of 5, which would eat that gain, or reduce our circumference to about 900km for current materials.
That last figure also means that you couldn't rely on gravity to hold your atmosphere in; for any reasonable atomosphere retention you'd need walls rising pretty much the radius of the structure. So it would have to be a sealed environment.
An orbital that size*** with a population density of, say, The Netherlands, would support 18 million people. Its mass is kind of arbitrary, but if we make it 100m thick, we get about 6 trillion tons. The asteroid 10 Hygiea† would provide enough materials to build 10,000 of these, giving us room for 180 billion people.****
Just in case you were looking for an investment property...
* Hey, I have super/subscript in the comments editor, but not in the post editor. I really need to fix that!
** I think I got the units right there. I'm pretty sure I at least got the scalar quantities right.
*** Let's give it a width of 50km for the sake of getting a number here.
**** If we had theoretical-limit materials and scaled everything up linearly - including the thickness - we'd get 80 habitats capable of holding 450 million people each.
† A test of my new Wikipedia linky tags, [wp] and [wikipedia]. It's a generalised bit of code that will allow me to extend it to making easy links to any site. It doesn't have the smarts I added for the [youtube] tag, though; that one can take the video ID itself, but given a full Youtube URL it will extract the ID before putting it into the template.
Bugger.
I knew that Niven-style Ringworlds required materials with impossible tensile strengths (beyond anything known by quite a few orders of magnitude), but I wasn't sure about Banks-style orbitals. Orbitals, as well as being much smaller, have the attraction that they are in orbit about their sun and don't exhibit the dynamic instability of ringworlds.
So how big can we build an orbital? The answer is, not very.
Micro-scale samples of carbon nanotubes have apparently been tested with a tensile strength of 62GPa. That means they can withstand a tensile force of 62GN/m2* before breaking.
The density of said carbon nanotubes is 1.34g/cm3. Assuming we want an acceleration of 1G - let's round it up to 10m/s2 for simplicity - this gives us a maximum circumference of 62GPa / 13.4kN/m3 = 4600km.**
The theoretical maximum tensile strength for carbon nanotubes is 300GPa, which would give us 23,000km. But I'd want to have a safety factor of 5, which would eat that gain, or reduce our circumference to about 900km for current materials.
That last figure also means that you couldn't rely on gravity to hold your atmosphere in; for any reasonable atomosphere retention you'd need walls rising pretty much the radius of the structure. So it would have to be a sealed environment.
An orbital that size*** with a population density of, say, The Netherlands, would support 18 million people. Its mass is kind of arbitrary, but if we make it 100m thick, we get about 6 trillion tons. The asteroid 10 Hygiea† would provide enough materials to build 10,000 of these, giving us room for 180 billion people.****
Just in case you were looking for an investment property...
* Hey, I have super/subscript in the comments editor, but not in the post editor. I really need to fix that!
** I think I got the units right there. I'm pretty sure I at least got the scalar quantities right.
*** Let's give it a width of 50km for the sake of getting a number here.
**** If we had theoretical-limit materials and scaled everything up linearly - including the thickness - we'd get 80 habitats capable of holding 450 million people each.
† A test of my new Wikipedia linky tags, [wp] and [wikipedia]. It's a generalised bit of code that will allow me to extend it to making easy links to any site. It doesn't have the smarts I added for the [youtube] tag, though; that one can take the video ID itself, but given a full Youtube URL it will extract the ID before putting it into the template.
Posted by: Pixy Misa at
11:14 PM
| Comments (16)
| Add Comment
| Trackbacks (Suck)
Post contains 435 words, total size 3 kb.
That's Just Wrong
Invader Zim slash fanfic?
Eww.
Invader Zim slash fanfic?
Eww.
Posted by: Pixy Misa at
12:30 AM
| No Comments
| Add Comment
| Trackbacks (Suck)
Post contains 8 words, total size 1 kb.
Sunday, March 25
Some Day There Will Be Content
In the meantime, here's a video I found at LGF:
more...
In the meantime, here's a video I found at LGF:
more...
Posted by: Pixy Misa at
01:29 PM
| No Comments
| Add Comment
| Trackbacks (Suck)
Post contains 28 words, total size 1 kb.
Saturday, March 24
Hidden Spoilers Not Safe For Work
Secret message:
Spoiler for any random anime series:
This message brought to you by the tags [hide], [spoiler] and [nsfw].
Secret message:
Spoiler for any random anime series:
This message brought to you by the tags [hide], [spoiler] and [nsfw].
Posted by: Pixy Misa at
07:31 PM
| Comments (4)
| Add Comment
| Trackbacks (Suck)
Post contains 22 words, total size 1 kb.
<< Page 1 of 3 >>
103kb generated in 0.2396 seconds; 61 queries returned 322 records.
Powered by Minx 1.1.4-pink.
Powered by Minx 1.1.4-pink.
