Well that's good. Fantastic. That gives us 20 minutes to save the world and I've got a post office. And it's shut!
Saturday, March 31
Always In The Last Place You Look
I've been scratching my head trying to work out why the main query that drives Minx blogs is so inefficient. I tweak indexes and I tweak selection criteria and I tweak sorting and nothing seems to improve the query time reported at the bottom of the page. Now, normally it's only 20ms or so, but if you catch it after an update - a new post or comment - it blows out to 400ms or so because the query cache just got flushed.
So I tweaked and I twiddled and I fiddled and fretted, and I took the query and plugged in some parameters and ran it outside of Minx, and it turned out to be, in fact, reasonably quick.
Then I tried disabling the sidebar to see what effect that had on the total query time.
19ms cached; 28ms uncached. Perfectly reasonable for selecting and sorting and joining multiple fairly complex (albeit small) subsets of 13,000 records.
It's those darn recent comments again!
Update: And now I remember why recent comments are slow. When you list the recent comments, you're listing the last 20 (or whatever) comments in a folder. But comments aren't in folders, they're in threads, so you have to look at the threads in the folder, and then find the most recent comments from among the comments in those threads. Big slow join before the sort.
There's a straightforward way to optimise the common case - use either multiple queries or subqueries to restrict MySQL to only inspect the 20 most recently updated threads. That gets messy for the general case, though, where you want page 17 of recent comments.
Need to spend a bit more time thinking about this one.
Update: Gargh! MySQL 5.0.27 doesn't support LIMIT in subqueries anyway.
Update: Well, it doesn't if you're using IN to restrict the main search to a subset of threads selected by the subquery, but if you use the subquery to return the update datetime of the nth most recently updated thread and then restrict the search to threads updated since then, you're swimming in gravy.
...
Which would be pretty unpleasant, come to think of it.
I've been scratching my head trying to work out why the main query that drives Minx blogs is so inefficient. I tweak indexes and I tweak selection criteria and I tweak sorting and nothing seems to improve the query time reported at the bottom of the page. Now, normally it's only 20ms or so, but if you catch it after an update - a new post or comment - it blows out to 400ms or so because the query cache just got flushed.
So I tweaked and I twiddled and I fiddled and fretted, and I took the query and plugged in some parameters and ran it outside of Minx, and it turned out to be, in fact, reasonably quick.
Then I tried disabling the sidebar to see what effect that had on the total query time.
19ms cached; 28ms uncached. Perfectly reasonable for selecting and sorting and joining multiple fairly complex (albeit small) subsets of 13,000 records.
It's those darn recent comments again!
Update: And now I remember why recent comments are slow. When you list the recent comments, you're listing the last 20 (or whatever) comments in a folder. But comments aren't in folders, they're in threads, so you have to look at the threads in the folder, and then find the most recent comments from among the comments in those threads. Big slow join before the sort.
There's a straightforward way to optimise the common case - use either multiple queries or subqueries to restrict MySQL to only inspect the 20 most recently updated threads. That gets messy for the general case, though, where you want page 17 of recent comments.
Need to spend a bit more time thinking about this one.
Update: Gargh! MySQL 5.0.27 doesn't support LIMIT in subqueries anyway.
Update: Well, it doesn't if you're using IN to restrict the main search to a subset of threads selected by the subquery, but if you use the subquery to return the update datetime of the nth most recently updated thread and then restrict the search to threads updated since then, you're swimming in gravy.
...
Which would be pretty unpleasant, come to think of it.
Posted by: Pixy Misa at
06:13 AM
| Comments (19)
| Add Comment
| Trackbacks (Suck)
Post contains 367 words, total size 2 kb.
Friday, March 30
Bad Surprises
I was back at my old job today, preparing for what I hope is one of the last phone company bill runs I ever have to do, when suddenly, at about 5:45 on a Friday afternoon,* the billing server dropped dead.
My first thought was Do we have backups? Yes, we have backups.** Okay then, let's haul the server out, plug it into a monitor (the computer room is not terribly well organised), and see what we get.
After a certain amount of fiddling I managed to get a BIOS screen up, and select the boot menu, whereupon the server spontaneously rebooted. I tried this several times with identical results; in fact, leaving the server alone would result in a boot-crash-boot-crash cycle just like my unlamented Compaq notebook.
There are four lights on the front of the server that indicate a POST code, so I hit Dell's website for the user manual and looked it up. Running down the list: CPU failure, memory failure, hard drive failure.***
USB failure.
Swapped the keyboard to a different USB port and it worked perfectly.
* There's some sort of physical law that requires this.
** It would take a couple of days to rebuild the system and restore the backups, but at least we had backups. From the previous day, I should add.
*** This is the mission-critical server that was spec'd with only one disk drive, so my first thought was exactly that.
I was back at my old job today, preparing for what I hope is one of the last phone company bill runs I ever have to do, when suddenly, at about 5:45 on a Friday afternoon,* the billing server dropped dead.
My first thought was Do we have backups? Yes, we have backups.** Okay then, let's haul the server out, plug it into a monitor (the computer room is not terribly well organised), and see what we get.
After a certain amount of fiddling I managed to get a BIOS screen up, and select the boot menu, whereupon the server spontaneously rebooted. I tried this several times with identical results; in fact, leaving the server alone would result in a boot-crash-boot-crash cycle just like my unlamented Compaq notebook.
There are four lights on the front of the server that indicate a POST code, so I hit Dell's website for the user manual and looked it up. Running down the list: CPU failure, memory failure, hard drive failure.***
USB failure.
Swapped the keyboard to a different USB port and it worked perfectly.
* There's some sort of physical law that requires this.
** It would take a couple of days to rebuild the system and restore the backups, but at least we had backups. From the previous day, I should add.
*** This is the mission-critical server that was spec'd with only one disk drive, so my first thought was exactly that.
Posted by: Pixy Misa at
08:42 PM
| Comments (1)
| Add Comment
| Trackbacks (Suck)
Post contains 243 words, total size 2 kb.
Wednesday, March 28
Return Of The Sticky
So that's why my honeypot wasn't catching any flies. I hadn't properly updated the code, and it was still expecting the Page object to be a global - which hasn't been true since the Great Rewrite back in January.
Without the Page object, it didn't have access to... well, much of anything, really. Including the database. So it never actually created any records.
Let's see what it does now...
Hmm. Still empty, but it hasn't been fixed very long.
Guess I should pretend to be a spammer and actually test it.
So that's why my honeypot wasn't catching any flies. I hadn't properly updated the code, and it was still expecting the Page object to be a global - which hasn't been true since the Great Rewrite back in January.
Without the Page object, it didn't have access to... well, much of anything, really. Including the database. So it never actually created any records.
Let's see what it does now...
Hmm. Still empty, but it hasn't been fixed very long.
Guess I should pretend to be a spammer and actually test it.
Posted by: Pixy Misa at
12:45 AM
| Comments (5)
| Add Comment
| Trackbacks (Suck)
Post contains 95 words, total size 1 kb.
Monday, March 26
How Much For Just The Planet?
Bugger.
I knew that Niven-style Ringworlds required materials with impossible tensile strengths (beyond anything known by quite a few orders of magnitude), but I wasn't sure about Banks-style orbitals. Orbitals, as well as being much smaller, have the attraction that they are in orbit about their sun and don't exhibit the dynamic instability of ringworlds.
So how big can we build an orbital? The answer is, not very.
Micro-scale samples of carbon nanotubes have apparently been tested with a tensile strength of 62GPa. That means they can withstand a tensile force of 62GN/m2* before breaking.
The density of said carbon nanotubes is 1.34g/cm3. Assuming we want an acceleration of 1G - let's round it up to 10m/s2 for simplicity - this gives us a maximum circumference of 62GPa / 13.4kN/m3 = 4600km.**
The theoretical maximum tensile strength for carbon nanotubes is 300GPa, which would give us 23,000km. But I'd want to have a safety factor of 5, which would eat that gain, or reduce our circumference to about 900km for current materials.
That last figure also means that you couldn't rely on gravity to hold your atmosphere in; for any reasonable atomosphere retention you'd need walls rising pretty much the radius of the structure. So it would have to be a sealed environment.
An orbital that size*** with a population density of, say, The Netherlands, would support 18 million people. Its mass is kind of arbitrary, but if we make it 100m thick, we get about 6 trillion tons. The asteroid 10 Hygiea†would provide enough materials to build 10,000 of these, giving us room for 180 billion people.****
Just in case you were looking for an investment property...
* Hey, I have super/subscript in the comments editor, but not in the post editor. I really need to fix that!
** I think I got the units right there. I'm pretty sure I at least got the scalar quantities right.
*** Let's give it a width of 50km for the sake of getting a number here.
**** If we had theoretical-limit materials and scaled everything up linearly - including the thickness - we'd get 80 habitats capable of holding 450 million people each.
†A test of my new Wikipedia linky tags, [wp] and [wikipedia]. It's a generalised bit of code that will allow me to extend it to making easy links to any site. It doesn't have the smarts I added for the [youtube] tag, though; that one can take the video ID itself, but given a full Youtube URL it will extract the ID before putting it into the template.
Bugger.
I knew that Niven-style Ringworlds required materials with impossible tensile strengths (beyond anything known by quite a few orders of magnitude), but I wasn't sure about Banks-style orbitals. Orbitals, as well as being much smaller, have the attraction that they are in orbit about their sun and don't exhibit the dynamic instability of ringworlds.
So how big can we build an orbital? The answer is, not very.
Micro-scale samples of carbon nanotubes have apparently been tested with a tensile strength of 62GPa. That means they can withstand a tensile force of 62GN/m2* before breaking.
The density of said carbon nanotubes is 1.34g/cm3. Assuming we want an acceleration of 1G - let's round it up to 10m/s2 for simplicity - this gives us a maximum circumference of 62GPa / 13.4kN/m3 = 4600km.**
The theoretical maximum tensile strength for carbon nanotubes is 300GPa, which would give us 23,000km. But I'd want to have a safety factor of 5, which would eat that gain, or reduce our circumference to about 900km for current materials.
That last figure also means that you couldn't rely on gravity to hold your atmosphere in; for any reasonable atomosphere retention you'd need walls rising pretty much the radius of the structure. So it would have to be a sealed environment.
An orbital that size*** with a population density of, say, The Netherlands, would support 18 million people. Its mass is kind of arbitrary, but if we make it 100m thick, we get about 6 trillion tons. The asteroid 10 Hygiea†would provide enough materials to build 10,000 of these, giving us room for 180 billion people.****
Just in case you were looking for an investment property...
* Hey, I have super/subscript in the comments editor, but not in the post editor. I really need to fix that!
** I think I got the units right there. I'm pretty sure I at least got the scalar quantities right.
*** Let's give it a width of 50km for the sake of getting a number here.
**** If we had theoretical-limit materials and scaled everything up linearly - including the thickness - we'd get 80 habitats capable of holding 450 million people each.
†A test of my new Wikipedia linky tags, [wp] and [wikipedia]. It's a generalised bit of code that will allow me to extend it to making easy links to any site. It doesn't have the smarts I added for the [youtube] tag, though; that one can take the video ID itself, but given a full Youtube URL it will extract the ID before putting it into the template.
Posted by: Pixy Misa at
11:14 PM
| Comments (16)
| Add Comment
| Trackbacks (Suck)
Post contains 435 words, total size 3 kb.
Friday, March 16
Hot Skynet-on-Skynet Action!
When we wake up one day to find that our computers have taken over the world, it won't be a top-secret military project behind it.
It will be a spam filter.
When we wake up one day to find that our computers have taken over the world, it won't be a top-secret military project behind it.
It will be a spam filter.
Posted by: Pixy Misa at
09:37 PM
| Comments (6)
| Add Comment
| Trackbacks (Suck)
Post contains 34 words, total size 1 kb.
Friday, March 09
Reboot, Reinstall... CentOS?
I've been having some strange issues with Shampoo, one of the two main servers for mee.nu. I decided that I'd better get Softlayer (our hosting company) to do a reinstall, because these issues now include not being able to start Apache.
So I opened a support ticket, saying that the server was screwed up, and could they do a memory test and if that doesn't show anything, reinstall Linux.
And they responded saying that this will cost $3, and I said, well, fine, and they said, hey, your server is screwed up, you'll need to reinstall Linux.
And I said, huh?
And they said, there's a button in our management site. Choose the server, and you can repartition your drives and reinstall the operating system.
And I said, cooool.
And did just that.
Update: It still doesn't work, but hey, I got to reinstall the operating system from the other side of the planet.
Update: The network ports weren't autonegotiating properly, which was causing collisions and errors, which was causing problems with automatic RPM downloads, which was screwing up the install process. Fixed.
I've been having some strange issues with Shampoo, one of the two main servers for mee.nu. I decided that I'd better get Softlayer (our hosting company) to do a reinstall, because these issues now include not being able to start Apache.
So I opened a support ticket, saying that the server was screwed up, and could they do a memory test and if that doesn't show anything, reinstall Linux.
And they responded saying that this will cost $3, and I said, well, fine, and they said, hey, your server is screwed up, you'll need to reinstall Linux.
And I said, huh?
And they said, there's a button in our management site. Choose the server, and you can repartition your drives and reinstall the operating system.
And I said, cooool.
And did just that.
Update: It still doesn't work, but hey, I got to reinstall the operating system from the other side of the planet.
Update: The network ports weren't autonegotiating properly, which was causing collisions and errors, which was causing problems with automatic RPM downloads, which was screwing up the install process. Fixed.
Posted by: Pixy Misa at
11:52 AM
| Comments (1)
| Add Comment
| Trackbacks (Suck)
Post contains 186 words, total size 1 kb.
Saturday, March 03
No Joy In Screenville
I just acquired a compact, quiet, elegantly-styled home-theatre PC.
I just acquired a compact, quiet, elegantly-styled home-theatre PC.
Yesterday it was a notebook.
Posted by: Pixy Misa at
08:48 AM
| Comments (5)
| Add Comment
| Trackbacks (Suck)
Post contains 22 words, total size 1 kb.
<< Page 1 of 1 >>
83kb generated in CPU 0.0548, elapsed 0.2991 seconds.
57 queries taking 0.2836 seconds, 392 records returned.
Powered by Minx 1.1.6c-pink.
57 queries taking 0.2836 seconds, 392 records returned.
Powered by Minx 1.1.6c-pink.
