The Death and Resurrection of Pam

I ran the up2date utility on one of the Fedora servers at work on Friday to catch up with all the latest patches. For some reason - possibly because of a timeout while downloading the new kernel - up2date crashed. Trying to run it again just brought more crashes.

Since it was Friday afternoon, I left it at that. After all, the box was working fine, it just wasn't fully patched.

Fast forward to Monday morning and an irate phone call. It seems that files that normally arrive via FTP aren't, and people who can normally log in using FTP can't.

So I log in using SSH (which works fine) and sure enough, FTP is rejecting my login. Also everyone else's logins. Also a new login I set up just to test it.

Strange.

I restart the FTP server. No difference.

I check the config file and fiddle a little. No difference.

I download the latest source and compile and install a new copy of the FTP server. No difference.

I install a completely different FTP server. No difference.

Now hang on just a minute there. The two FTP servers share no parameters, no config files, no directories. And they reject all logins, but they do ask for a password. So it's an authentication issue.

Linux offers something called PAM: pluggable authentication modules. Basically it's a centralised mechanism for checking usernames and passwords and other such stuff. And both of the FTP servers rely on PAM to check logins.

So, I float a hypothesis: I know up2date was trying to update the kernel. I know that it didn't finish doing this. I know from the log files that the server rebooted over the weekend - possibly due to a power glitch, since it's not on a UPS. (Yeah, yeah.) So maybe up2date had gotten far enough in to mess up PAM but not far enough in to actually make it all work again. So if I download and install the kernel manually, it will say:

error: Failed dependencies:
        /bin/sh is needed by kernel-2.4.22-1.2140.nptl

Bum!

What it really means is that you have to update to the very latest version of bash before updating the kernel. Yes, that seems strange to me too, but what do I know?

So I update bash, and I update the kernel, and I reboot, and it works! It works it works it works!

And a good thing too, because I have no idea how I would have proceeded if that hadn't fixed the problem.

Posted by: Pixy Misa at 02:02 AM | Comments (2) | Add Comment | Trackbacks (Suck)
Post contains 429 words, total size 3 kb.