Tom's FreeBSD blog

At work we’re a predominantly Windows shop but I have a machine which had CentOS 4 loaded on it which I used for various *nix related experimentation.  I’m our AD admin so I’ve used it a bit for kerberos and LDAP interoperability testing.  AD is pretty much krb5 and LDAP on steroids and it’s not hard to get a *nix machine to authenticate against and pull user info from it.  I set this box up a few years ago and recently decided to do a fresh install and move it to CentOS 5.  This fresh install however had some kerberos issues due to some things I forgot I did years back.

When I was first toying with kerberos on this box and authenticating against AD I used some older instructions which avoided Samba.

http://grolmsnet.de/kerbtut/

This worked just fine but step 6 caused me headaches after I installed CentOS 5 and decided to use Samba to join my machine to AD and use it to manage the kerberos keytab on it.  The instructions call for creating a dummy account and using ktpass.exe on a Windows box to generate a keytab file and a corresponding HTTP SPN.  I created an AD user called http_corptech which had the SPN HTTP/[email protected] attached to it.  Once I had the keytab in place and mod_auth_kerb setup this worked fairly well.

When I installed CentOS 5 on the machine I decided to use Samba for managing domain membership since it had advanced over the years and now had the ability to manage the kerberos keytab.  After /etc/krb5.conf was setup the following two commads took care of my domain membership:

# net ads join -U tmclaughlin
# net ads keytab create

A few days ago I went to setup mod_auth_kerb again.  I used the same config I’ve always used and currently have working on other machines and used the following command to create my HTTP SPN

# net ads keytab add HTTP

However, I received 401 errors when trying to access protected content.  I decided to crank Apache’s LogLevel up to “debug” and watched error_log.  The following was what I saw:

[Tue Jul 13 23:54:27 2010] [debug] src/mod_auth_kerb.c(1432): [client 172.30.19.45] kerb_authenticate_user entered with user (NULL) and auth_type Kerberos
[Tue Jul 13 23:54:27 2010] [debug] src/mod_auth_kerb.c(1432): [client 172.30.19.45] kerb_authenticate_user entered with user (NULL) and auth_type Kerberos
[Tue Jul 13 23:54:27 2010] [debug] src/mod_auth_kerb.c(1147): [client 172.30.19.45] Acquiring creds for [email protected]
[Tue Jul 13 23:54:27 2010] [debug] src/mod_auth_kerb.c(1266): [client 172.30.19.45] Verifying client data using KRB5 GSS-API
[Tue Jul 13 23:54:27 2010] [debug] src/mod_auth_kerb.c(1282): [client 172.30.19.45] Verification returned code 589824
[Tue Jul 13 23:54:27 2010] [debug] src/mod_auth_kerb.c(1309): [client 172.30.19.45] Warning: received token seems to be NTLM, which isn't supported by the Kerberos module. Check your IE configuration.
[Tue Jul 13 23:54:27 2010] [error] [client 172.30.19.45] gss_accept_sec_context() failed: Invalid token was supplied (No error)

I saw for some reason with tcpdump that both Firefox and IE were attempting NTLM negotiation instead of GSSAPI when attempting to access this content.  I spun my wheels with this error for a bit.  Eventually I enabled KrbMethodK5Passwd for mod_auth_kerb which I had disabled previously to make sure tickets were working correctly.  If my ticket was being rejected there was already something wrong and dealing with a password prompt was just a waste of time.  When I did this I found the following in error_log.

[Tue Jul 13 23:55:21 2010] [debug] src/mod_auth_kerb.c(1432): [client 172.30.19.45] kerb_authenticate_user entered with user (NULL) and auth_type Kerberos
[Tue Jul 13 23:55:21 2010] [debug] src/mod_auth_kerb.c(915): [client 172.30.19.45] Using HTTP/[email protected] as server principal for password verification
[Tue Jul 13 23:55:21 2010] [debug] src/mod_auth_kerb.c(655): [client 172.30.19.45] Trying to get TGT for user [email protected]
[Tue Jul 13 23:55:21 2010] [debug] src/mod_auth_kerb.c(569): [client 172.30.19.45] Trying to verify authenticity of KDC using principal HTTP/[email protected]
[Tue Jul 13 23:55:21 2010] [debug] src/mod_auth_kerb.c(584): [client 172.30.19.45] krb5_get_credentials() failed when verifying KDC
[Tue Jul 13 23:55:21 2010] [error] [client 172.30.19.45] failed to verify krb5 credentials: Server not found in Kerberos database
[Tue Jul 13 23:55:21 2010] [debug] src/mod_auth_kerb.c(994): [client 172.30.19.45] kerb_authenticate_user_krb5pwd ret=401 user=(NULL) authtype=(NULL)

I could see now that there was a problem with the kerberos setup somewhere.  I had already created a new keytab file and even removed and rejoined the machine to AD.  Finally it struck me.  The “Server not found in Kerberos database” error message was not entirely accurate.  It’s not that the SPN wasn’t found in AD…  It was that multiple objects in AD had that SPN.  Both the computer account I created with Samba and the old dummy user I had created had a servicePrincipalName attribute with the SPN HTTP/corptech.example.com.  Once I deleted the dummy account from AD and it propagated kerberos authentication worked just fine.  Anyone who has used the older instructions I used before and now plans on using Samba to manage the kerberos keytab will probably find this info helpful.  The kerberos error about the server not being found will probably lead you off in the wrong direction.

p.s. I had a similar issue with ksu before on the machine when I first installed CentOS 5.  I had created a host_corptech user so I could get tickets originally and was receiving the same error.