Thursday, December 20

Geek

I Thought It Was Running A Little Slow...

top - 21:18:47 up 281 days, 5:39, 3 users, load average: 6750.53, 6750.16, 6749.71
Tasks: 34006 total, 1 running, 34005 sleeping, 0 stopped, 0 zombie
Cpu(s): 2.2%us, 6.2%sy, 0.0%ni, 91.0%id, 0.6%wa, 0.0%hi, 0.0%si, 0.0%st
Mem: 7806736k total, 7774284k used, 32452k free, 44788k buffers
Swap: 8192992k total, 3150312k used, 5042680k free, 315284k cached
Only 6750.53, eh? That's not that high. Compared to this, anyway:
top - 21:22:31 up 281 days, 5:42, 3 users, load average: 13190.13, 9409.43, 7743.67
Tasks: 24641 total, 486 running, 19233 sleeping, 0 stopped, 4922 zombie
Cpu(s): 5.7%us, 23.5%sy, 0.0%ni, 67.5%id, 3.2%wa, 0.0%hi, 0.0%si, 0.1%st
Mem: 7806736k total, 7167372k used, 639364k free, 3784k buffers
Swap: 8192992k total, 1565400k used, 6627592k free, 77216k cached
Can Pixy regain control of the dev server, or is it time to cut losses and hit the reset switch?  Yeah, that's kind of what I think too.

Posted by: Pixy Misa at 08:20 PM | Comments (5) | Add Comment | Trackbacks (Suck)
Post contains 148 words, total size 1 kb.

1 How in hell did you end up with 34,006 tasks running? It's no wonder the machine is on its knees.

Posted by: Steven Den Beste at Friday, December 21 2012 02:09 AM (+rSRq)

2 oops?

Posted by: Hypozeuxis at Friday, December 21 2012 04:05 AM (XjJZF)

3 A regular cron job was getting stuck and leaving three processes behind every time.  The impressive thing is that even with 34,006 processes, the server was perfectly responsive for most tasks - it wasn't until I ran a manual backup and was only getting 5MB/s that I realised there was a problem.

Then I killed the wayward processes, and that triggered an attempt to send me about 11,000 notification emails simultaneously.  And that is when things really went downhill...

Posted by: Pixy Misa at Friday, December 21 2012 08:50 AM (PiXy!)

4 Ah... is this why mu.nu and mee.nu blogs were down for a while?  Or was that just me?

Posted by: Wonderduck at Friday, December 21 2012 04:46 PM (cymHZ)

5 This was just the dev server.  I'm not sure what happened to blog access, but it wasn't just you, and it doesn't seem to have been the server.  Probably a network issue somewhere; it was fixed before I could track it down and the server didn't record any problems.

Posted by: Pixy Misa at Friday, December 21 2012 05:13 PM (PiXy!)

Hide Comments | Add Comment

Comments are disabled. Post is locked.
47kb generated in CPU 0.0131, elapsed 0.1315 seconds.
56 queries taking 0.1246 seconds, 342 records returned.
Powered by Minx 1.1.6c-pink.