Now? You want to do this now?
I have a right to know! I'm getting married in four hundred and thirty years!
Saturday, March 31
Always In The Last Place You Look
I've been scratching my head trying to work out why the main query that drives Minx blogs is so inefficient. I tweak indexes and I tweak selection criteria and I tweak sorting and nothing seems to improve the query time reported at the bottom of the page. Now, normally it's only 20ms or so, but if you catch it after an update - a new post or comment - it blows out to 400ms or so because the query cache just got flushed.
So I tweaked and I twiddled and I fiddled and fretted, and I took the query and plugged in some parameters and ran it outside of Minx, and it turned out to be, in fact, reasonably quick.
Then I tried disabling the sidebar to see what effect that had on the total query time.
19ms cached; 28ms uncached. Perfectly reasonable for selecting and sorting and joining multiple fairly complex (albeit small) subsets of 13,000 records.
It's those darn recent comments again!
Update: And now I remember why recent comments are slow. When you list the recent comments, you're listing the last 20 (or whatever) comments in a folder. But comments aren't in folders, they're in threads, so you have to look at the threads in the folder, and then find the most recent comments from among the comments in those threads. Big slow join before the sort.
There's a straightforward way to optimise the common case - use either multiple queries or subqueries to restrict MySQL to only inspect the 20 most recently updated threads. That gets messy for the general case, though, where you want page 17 of recent comments.
Need to spend a bit more time thinking about this one.
Update: Gargh! MySQL 5.0.27 doesn't support
LIMIT in subqueries anyway.
Update: Well, it doesn't if you're using
IN to restrict the main search to a subset of threads selected by the subquery, but if you use the subquery to return the update datetime of the
nth most recently updated thread and then restrict the search to threads updated since then, you're swimming in gravy.
...
Which would be pretty unpleasant, come to think of it.
Posted by: Pixy Misa at
06:13 AM
| Comments (19)
| Add Comment
| Trackbacks (Suck)
Post contains 367 words, total size 2 kb.
1
Of
course it's the last place you look, because once you find it,
you stop looking.
Posted by: Steven Den Beste at Saturday, March 31 2007 05:18 PM (+rSRq)
2
Rule only applies to normal people.
Posted by: Pixy Misa at Saturday, March 31 2007 05:19 PM (PiXy!)
3
New query is in place. Seems not to have broken things too badly...
Posted by: Pixy Misa at Monday, April 02 2007 03:28 AM (PiXy!)
4
Seems to be roughly 20x faster on the common worst case (query cache flushed, showing most recent comments) and no slower on the best case (query in cache) or absolute worst case (cache flushed, page n of n).
Posted by: Pixy Misa at Monday, April 02 2007 03:32 AM (PiXy!)
5
Except that it
didn't work!Well, the way it worked was that if you wanted, say, the 20 most recent comments, it found the date of the 20th most recently updated post, and only scanned the comments of posts updated at least as recently as that.
If you didn't have 20 posts, that kind of went bang.
Testing a modified version now.
Posted by: Pixy Misa at Monday, April 02 2007 12:25 PM (PiXy!)
6
Yeah, that seems to be okay too. Take the max() of the updated datetime of the N most recent posts instead.
Posted by: Pixy Misa at Monday, April 02 2007 12:26 PM (PiXy!)
7
Okay, I'm not sure that the new version actually
works, performance-wise. Test test...
Posted by: Pixy Misa at Monday, April 02 2007 12:32 PM (PiXy!)
Posted by: Pixy Misa at Monday, April 02 2007 12:32 PM (PiXy!)
9
What if I tweak the query a little to use this new index?
Posted by: Pixy Misa at Monday, April 02 2007 12:35 PM (PiXy!)
Posted by: Pixy Misa at Monday, April 02 2007 12:35 PM (PiXy!)
11
Fnordles. The new subquery works just fine as a stand-alone query, but plug it into the main query and it all sucks. Some kind of limitation or bug in the MySQL query optimiser? Dunno.
But since an extra query only adds a few hundred microseconds, and I've already spent hours trying to make this work, I'm gonna go for the easy way out.
Posted by: Pixy Misa at Monday, April 02 2007 12:56 PM (PiXy!)
12
Well, that's not working either. But now that I can see the results from the subquery, I know why it's not working.
I don't know how to make it work, but I know why it's not working.
Posted by: Pixy Misa at Monday, April 02 2007 01:15 PM (PiXy!)
13
Trying again, this time with nested subqueries...
Posted by: Pixy Misa at Monday, April 02 2007 01:46 PM (PiXy!)
Posted by: Pixy Misa at Monday, April 02 2007 01:46 PM (PiXy!)
Posted by: Pixy Misa at Monday, April 02 2007 01:46 PM (PiXy!)
16
Hey, Webmaster! There's some loon using the name "Pixy" who's spamming your blog. If I were you, I'd ban him. (He can't possibly have anything interesting to say.)
Posted by: Steven Den Beste at Monday, April 02 2007 02:05 PM (+rSRq)
17
Thanks for the tip, Steven.
By the way, I think your network cable is plugged in upside down!
Posted by: Pixy Misa at Monday, April 02 2007 02:21 PM (PiXy!)
18
Could be; I'll check it out!
Posted by: Steven Den Beste at Monday, April 02 2007 02:31 PM (+rSRq)
19
I hope that's all it is... I don't want to have to turn my monitor upside down.
Posted by: Wonderduck at Monday, April 02 2007 04:15 PM (R/R0E)
Hide Comments
| Add Comment
Friday, March 30
Bad Surprises
I was back at my old job today, preparing for what I hope is one of the last phone company bill runs I ever have to do, when suddenly, at about 5:45 on a Friday afternoon,* the billing server dropped dead.
My first thought was
Do we have backups? Yes, we have backups.** Okay then, let's haul the server out, plug it into a monitor (the computer room is not terribly well organised), and see what we get.
After a certain amount of fiddling I managed to get a BIOS screen up, and select the boot menu, whereupon the server spontaneously rebooted. I tried this several times with identical results; in fact, leaving the server alone would result in a boot-crash-boot-crash cycle just like my unlamented Compaq notebook.
There are four lights on the front of the server that indicate a POST code, so I hit Dell's website for the user manual and looked it up. Running down the list: CPU failure, memory failure, hard drive failure.***
USB failure.
Swapped the keyboard to a different USB port and it worked perfectly.
* There's some sort of physical law that requires this.
** It would take a couple of days to rebuild the system and restore the backups, but at least we had backups. From the previous day, I should add.
*** This is the mission-critical server that was spec'd with only one disk drive, so my first thought was exactly that.
Posted by: Pixy Misa at
08:42 PM
| Comments (1)
| Add Comment
| Trackbacks (Suck)
Post contains 243 words, total size 2 kb.
1
Yes. I can see how USB is key to a computer, let alone a server, operating.
Posted by: Andrew at Monday, April 02 2007 08:19 PM (/uGTr)
Hide Comments
| Add Comment
Wednesday, March 28
Return Of The Sticky
So that's why my honeypot wasn't catching any flies. I hadn't properly updated the code, and it was still expecting the Page object to be a global - which hasn't been true since the Great Rewrite back in January.
Without the Page object, it didn't have access to... well, much of anything, really. Including the database. So it never actually created any records.
Let's see what it does now...
Hmm. Still empty, but it hasn't been fixed very long.
Guess I should pretend to be a spammer and actually test it.
Posted by: Pixy Misa at
12:45 AM
| Comments (5)
| Add Comment
| Trackbacks (Suck)
Post contains 95 words, total size 1 kb.
1
Let's see now.
Yep, banned myself nicely.
Posted by: Pixy Misa at Wednesday, March 28 2007 12:00 PM (PiXy!)
2
Okay, updated the standard form (and made a mess the first time), so let's see if comments still work.
Posted by: Pixy Misa at Wednesday, March 28 2007 12:06 PM (PiXy!)
3
One time I stumbled onto J Greely's honeypot by accident and had to send him email to ask to be unbanned. When you said you were going to test your own honeypot I was worried for a moment that you might find yourself completely locked out of your development system.
Posted by: Steven Den Beste at Wednesday, March 28 2007 04:51 PM (+rSRq)
4
Usually you only do that once, and then you whitelist one IP address to make sure you can get back in and undo it. I was very careful to set mine up this way, since the machine was 75 miles away and didn't have a remote console. :-)
In this case, he's probably only blacklisting at the application level, not changing his firewall config to drop all packets the way I do when someone visits the /banmyipaddress/ URL on my site.
-j
Posted by: J Greely at Wednesday, March 28 2007 07:16 PM (9Nz6c)
5
Yep. I'd be pretty hesitant about auto-firewalling anyone given the number of sites hosted on this server.
Posted by: Pixy Misa at Wednesday, March 28 2007 07:49 PM (PiXy!)
Hide Comments
| Add Comment
Monday, March 26
How Much For Just The Planet?
Bugger.
I knew that Niven-style Ringworlds required materials with impossible tensile strengths (beyond anything known by quite a few orders of magnitude), but I wasn't sure about Banks-style orbitals. Orbitals, as well as being much smaller, have the attraction that they are in orbit about their sun and don't exhibit the dynamic instability of ringworlds.
So how big can we build an orbital? The answer is, not very.
Micro-scale samples of carbon nanotubes have apparently been tested with a tensile strength of 62GPa. That means they can withstand a tensile force of 62GN/m
2* before breaking.
The density of said carbon nanotubes is 1.34g/cm
3. Assuming we want an acceleration of 1G - let's round it up to 10m/s
2 for simplicity - this gives us a maximum circumference of 62GPa / 13.4kN/m
3 = 4600km.**
The theoretical maximum tensile strength for carbon nanotubes is 300GPa, which would give us 23,000km. But I'd want to have a safety factor of 5, which would eat that gain, or reduce our circumference to about 900km for current materials.
That last figure also means that you couldn't rely on gravity to hold your atmosphere in; for any reasonable atomosphere retention you'd need walls rising pretty much the radius of the structure. So it would have to be a sealed environment.
An orbital that size*** with a population density of, say, The Netherlands, would support 18 million people. Its mass is kind of arbitrary, but if we make it 100m thick, we get about 6 trillion tons. The asteroid
10 Hygiea†would provide enough materials to build 10,000 of these, giving us room for 180 billion people.****
Just in case you were looking for an investment property...
* Hey, I have super/subscript in the comments editor, but not in the post editor. I really need to fix that!** I think I got the units right there. I'm pretty sure I at least got the scalar quantities right.
*** Let's give it a width of 50km for the sake of getting a number here.
**** If we had theoretical-limit materials and scaled everything up linearly - including the thickness - we'd get 80 habitats capable of holding 450 million people each.
†A test of my new Wikipedia linky tags, [wp] and [wikipedia]. It's a generalised bit of code that will allow me to extend it to making easy links to any site. It doesn't have the smarts I added for the [youtube] tag, though; that one can take the video ID itself, but given a full Youtube URL it will extract the ID before putting it into the template.
Posted by: Pixy Misa at
11:14 PM
| Comments (16)
| Add Comment
| Trackbacks (Suck)
Post contains 435 words, total size 3 kb.
Posted by: TallDave at Tuesday, March 27 2007 04:17 PM (odS+4)
2
So, the circumference would be about 540 miles, or almost a round trip from Chicago to Minneapolis.
Seems like it'd be a rather... how to put this?... well, dull place to live. Particularly with 18 million people living in it. Maybe I've not got a good handle on the overall scope of the thing, but it seems like it may just be a tad crowded.
Posted by: Wonderduck at Tuesday, March 27 2007 08:20 PM (135bi)
3
And I imagine, for the sake of simplicity, the topography is similar to that of... oh... Nebraska. Anything else could throw things out of balance.
Posted by: Will at Tuesday, March 27 2007 10:06 PM (SOx9v)
4
Seems like it'd be a rather... how to put this?... well, dull place to live.Until you get a meteor puncture, anyway.
Maybe I've not got a good handle on the overall scope of the thing, but it seems like it may just be a tad crowded.Weeeell... It would have the size, population, and approximate topography of the Netherlands.
Posted by: Pixy Misa at Tuesday, March 27 2007 10:39 PM (PiXy!)
5
It would have the size, population, and approximate topography of the Netherlands.
And this is supposed to be something in it's favor, is it?
Why not just 'glue' thirty or forty of these things together, make a tube out of them (and use the materials of another 20 to provide structural strength to the outside)? Though it'd still be a boring place, more or less, it'd be much bigger.
Posted by: Wonderduck at Tuesday, March 27 2007 11:41 PM (VdgKc)
6
"Seems like it'd be a rather... how to put this?... well,
dull place to live. "
Who says there can only be one?
HELP Pixy! You know how your RSS feed opens a page with comments that = New Comment Thingy? Ace's don't, and it makes it painful to read, knowing I can't put a snarky comment on it. And the jawas are even worse, since it lets you comment and then just deletes it.
Please fix their RSS feeds for them. If you do, I promise to send you 1 dollar. Yes, I'm
that serious.
Posted by: Kevin at Wednesday, March 28 2007 12:12 AM (/ndDU)
7
And this is supposed to be something in it's favor, is it?Weeeell...
Why not just 'glue' thirty or forty of these things together, make a
tube out of them (and use the materials of another 20 to provide
structural strength to the outside)? Though it'd still be a boring
place, more or less, it'd be much bigger.The idea of an orbital is that being a narrow ring, you can get plenty of sunlight in, albeit at an oblique angle. Now I just need to calculate how fast the thing would need to rotate... Hmm. This scribble says it would rotate every 7.5 minutes. That makes for a pretty short day...
Posted by: Pixy Misa at Wednesday, March 28 2007 02:02 AM (PiXy!)
8
HELP Pixy! You know how your RSS feed opens a page with comments that
= New Comment Thingy? Ace's don't, and it makes it painful to read,
knowing I can't put a snarky comment on it. And the jawas are even
worse, since it lets you comment and then just deletes it.I'm beta-testing the new mu.nu blogging system here. Ace and the Jawas will be moving to it soon, and then all those problems will be fixed.
(To be replaced by a whole new set of problems...)
Posted by: Pixy Misa at Wednesday, March 28 2007 02:03 AM (PiXy!)
9
2.5 cheers for Pixy!
.5 cheer saved for when it actually happens.
Posted by: Kevin at Wednesday, March 28 2007 10:26 AM (/ndDU)
10
Though it'd still be a boring place, more or less, it'd be much bigger.
That depends. Would it also have the Netherlands' laws?
Anyways, topography need not be dull, at least according to this:
The inside of the hoop can be formed to any type of planetary environment, from desert to ocean to jungle to glacier. The hoop is usually divided into individual 'plates', similar to continents, though there is usually no directly visible indication for the transition from one plate to another. Some plates mimic natural environments very closely, other are wild exaggerations possible only by advanced matter forming and intricate (but usually hidden) machinery - such as a gigantic river circumventing the whole orbital, which in some reaches travels on immense, kilometre-high bridge- or mountain-range-like constructions, and in other regions might act as an immense 'waterslide' for a floating event stadium.
http://en.wikipedia.org/wiki/Orbital_(The_Culture)
Posted by: TallDave at Thursday, March 29 2007 07:42 PM (odS+4)
11
Yeah, but Banks' orbitals are made of unobtainium.
Wiki article wrote:The Culture's orbitals are approximately ten million kilometres in
circumference and have widths varying between one thousand and six
thousand kilometres
That's a whole lot bigger than what we could theoretically build today.
Posted by: Pixy Misa at Thursday, March 29 2007 08:22 PM (PiXy!)
12
"...approximately ten million kilometres in circumference and have widths varying between one thousand and six thousand kilometres..."
Now THAT'S what I'm talkin' about!
Posted by: Wonderduck at Friday, March 30 2007 01:07 AM (135bi)
13
Well, sure. Just slight impossible, is all.
Posted by: Pixy Misa at Friday, March 30 2007 01:25 AM (+7YmF)
14
I calculated how big the orbital would have to be to have a surface gravity of 1g and a rotation period of 24 hours. Unless I've screwed up (quite possible), the answer is on the order of 30 million kilometres in circumference. That could be big enough for tidal forces to be significant.
The Ringworld rotated slower than that; it had the shadow squares to generate day an night.
Posted by: Pixy Misa at Friday, March 30 2007 01:34 AM (+7YmF)
15
Well sure, but as you said, thickness is arbitray. You could do a lot in 900 km^2.
Posted by: TallDave at Wednesday, April 04 2007 02:49 AM (odS+4)
16
Oops thats the circumference. Why did I think it was a square lol?
So 900 long, 50 wide, about 45000 sq km (can you treat the surface of a ringworld as a rectangle?). Hmm, not that big, but you could still have an awesome water park or two.
Posted by: TallDave at Wednesday, April 04 2007 02:54 AM (odS+4)
Hide Comments
| Add Comment
Friday, March 16
Hot Skynet-on-Skynet Action!
When we wake up one day to find that our computers have taken over the world, it won't be a top-secret military project behind it.
It will be a spam filter.
Posted by: Pixy Misa at
09:37 PM
| Comments (6)
| Add Comment
| Trackbacks (Suck)
Post contains 34 words, total size 1 kb.
1
Hmm, make sense. Eradicating the human race would fix the spam issue...
Posted by: Nick at Saturday, March 17 2007 01:02 PM (Q0Emr)
2
Speaking of which, has SpamFilter MeeNu taken over? Or are we being deluged by the depressive spammer again? Nobody's been able to comment at the Pond for over a day now...
Posted by: Wonderduck at Monday, March 19 2007 11:29 AM (aIFaX)
3
Wonderduck - I was wondering the same thing. Although it looks like comments at Munuviana are working, nobody else's are... well maybe jawa's but I haven't had time to get over there and see. :-)
Posted by: Teresa at Monday, March 19 2007 01:59 PM (gsbs5)
4
Yeah, sorry about that. Comments should be working again now.
Ace and the Jawas are on New Comments Thingy, so they don't get affected when Fluffy has an accident.
Beta test of Minx starts today, so soon Fluffy will be able to enter a well-earned retirement.
Posted by: Pixy Misa at Monday, March 19 2007 09:46 PM (PiXy!)
5
Evilnastyspamfilter is back? Or has Fluffy gone and soiled itself again?
Posted by: Wonderduck at Wednesday, March 21 2007 10:01 AM (de+ej)
6
I've thought for a while that spam filters might be the most pressing need for AI, because you need something that can tell the difference between a message someone actually wants and an advertisement, a very sophisticated judgement.
Posted by: TallDave at Tuesday, March 27 2007 04:32 PM (odS+4)
Hide Comments
| Add Comment
Friday, March 09
Reboot, Reinstall... CentOS?
I've been having some strange issues with Shampoo, one of the two main servers for mee.nu. I decided that I'd better get Softlayer (our hosting company) to do a reinstall, because these issues now include not being able to start Apache.
So I opened a support ticket, saying that the server was screwed up, and could they do a memory test and if that doesn't show anything, reinstall Linux.
And they responded saying that this will cost $3, and I said, well, fine, and they said, hey, your server is screwed up, you'll need to reinstall Linux.
And I said,
huh?And they said, there's a button in our management site. Choose the server, and you can repartition your drives and reinstall the operating system.
And I said,
cooool.
And did just that.
Update: It still doesn't work, but hey, I got to reinstall the operating system from the other side of the planet.
Update: The network ports weren't autonegotiating properly, which was causing collisions and errors, which was causing problems with automatic RPM downloads, which was screwing up the install process. Fixed.
Posted by: Pixy Misa at
11:52 AM
| Comments (1)
| Add Comment
| Trackbacks (Suck)
Post contains 186 words, total size 1 kb.
1
GM Roper mentioned your comment system so I had to be nosey and check it out! I remember Pixy Misa - I wanted to blog .nu a couple of years ago and got lost in the system some how! Looks good here!
Posted by: chrys at Saturday, March 10 2007 05:25 AM (nazPp)
Hide Comments
| Add Comment
Saturday, March 03
No Joy In Screenville
I just acquired a compact, quiet, elegantly-styled home-theatre PC.
Yesterday it was a notebook.
Posted by: Pixy Misa at
08:48 AM
| Comments (5)
| Add Comment
| Trackbacks (Suck)
Post contains 22 words, total size 1 kb.
1
Ah, evolution.
Or was that intelligent design?
Posted by: TallDave at Saturday, March 03 2007 05:08 PM (oyQH2)
Posted by: Also Pixy at Sunday, March 04 2007 03:24 PM (3R0bS)
Posted by: Also Pixy at Sunday, March 04 2007 03:25 PM (3R0bS)
Posted by: Also Pixy at Monday, March 05 2007 02:33 AM (3R0bS)
5
Booyah! Timezones are go!
Posted by: Also Pixy at Monday, March 05 2007 02:33 AM (3R0bS)
Hide Comments
| Add Comment
83kb generated in CPU 0.0816, elapsed 0.2052 seconds.
57 queries taking 0.1869 seconds, 399 records returned.
Powered by Minx 1.1.6c-pink.