This is an old revision of the document!
Centralize SSH Key Management
Lest assume we have 3 colleges (CollegeA, CollegeB, CollegeC) and we write a grant proposal and each institution will do something unique science wise. Grant gets funded and specialized hardware or software gets deployed at each college (for maybe brain scan analyses, deep learning, and engineering).
The grant mentioned that all members of participating colleges can request access at any college. How would one do that without a mess developing? Nobody, not even admins, should have access to any user level passwords. Accounts should be able to be revoked. CPU usage should be accountable. In short, some sort of “federated SSH access”. Command line access so we can rule out InCommon, it appears not to be ready for this. One option might be GSI-OpenSSH but it looks very complicated. Consult
- http://toolkit.globus.org/toolkit/about.html for GSI-OpenSSH
I asked the OpenHPC community and got some great suggestions back. One involved using Kerberos without tickets and tying into Active Directory. I quickly realized not all AD installations are publicly exposed. I'm providing Alan Sill's comments below for future reference. Another suggestion was to centrally manage SSH keys for password less login functionality, by Derek Simmel.
SSH public key authentication
[need a way to collect key files, push them out, and manage passwd/shadow/group unique UID/GID so they do clash, like weshmeij, lafjsimms]
In the /etc/ssh/sshd_config file, specify the path to user SSH public key files using a description like:
AuthorizedKeysFile /etc/ssh/authorized_keys/%u
For this example, the /etc, /etc/ssh, and /etc/ssh/authorized_keys are directories owned and only writable by root.
# ls -ld /etc /etc/ssh /etc/ssh/authorized_keys drwxr-xr-x. 135 root root 12288 Mar 23 16:59 /etc drwxr-xr-x. 3 root root 4096 Nov 9 14:02 /etc/ssh drwxr-xr-x 2 root root 4096 Nov 9 14:01 /etc/ssh/authorized_keys -rwxr-xr-x 2 root root 1024 Mar 29 11:01 /etc/ssh/authorized_keys/weshmeij
The sshd service interprets %u as the authenticated user's username on the system. So in this approach, every user has a file named after their username in the /etc/ssh/authorized_keys directory, root-owned.
Use a secure, automated method to distribute the public key files to each login host's /etc/ssh/authorized_keys directory - having them on a shared mounted filesystem introduces other potential security and availability issues, and is therefore not recommended.
Kerberos & AD
1. install pam by run commands "yum install pam", "yum install pam_krb5", and "yum install pam_ssh".
The last package is needed on a log-in node, it's not required on compute nodes.
2. Modify /etc/krb5.conf as below
(Note for historical reasons we use both upper case and lower case for our kerberos domains.
Both resolve to the same thing. You may not need this - try it and see.)
[logging]
default = FILE:/var/log/krb5libs.log
kdc = FILE:/var/log/krb5kdc.log
admin_server = FILE:/var/log/kadmind.log
[libdefaults]
default_realm = YOUR-UNIV-KERB-DOMAIN (e.g., MYUNIVERSITY.EDU)
dns_lookup_realm = true
dns_lookup_kdc = true
ticket_lifetime = 24h
renew_lifetime = 7d
forwardable = true
[realms]
YOUR-UNIV-KERB-DOMAIN (e.g., MYUNIVERSITY.EDU) = {
kdc = your-univ-domain (e.g., my university.edu):88
admin_server = your-univ-domain (e.g., my university.edu):749
}
your-univ-kerb-domain (e.g., ttu.edu) = {
kdc = your-univ-domain (e.g., my university.edu):88
admin_server = your-univ-domain (e.g., my university.edu):749
}
[kdc]
profile = /var/kerberos/krb5kdc/kdc.conf
[appdefaults]
pam = {
debug = false
ticket_lifetime = 36000
renew_lifetime = 36000
forwardable = true
krb4_convert = false
}
3. Replace /etc/ntp.conf using your donain’s preferences; example below,
server your-univ-ntp-server (e.g., ntp.myuniversiry.edu) iburst
server 127.127.1.1 iburst
fudge 127.127.1.1 stratum 10
driftfile /var/lib/ntp/drift
4. Replace /etc/ntp/step-tickers
your-univ-ntp-server (e.g., ntp.myuniversiry.edu)
5. Setup banner announcement in /etc/banner (optional, depending on yoru organizational policies)
6. Replace /etc/ssh/ssh_config
Host *
CheckHostIP no
ForwardX11 yes
ForwardAgent yes
StrictHostKeyChecking no
UsePrivilegedPort no
Protocol 2
7. Modify /etc/ssh/sshd_config
Protocol 2
KerberosAuthentication no
KerberosOrLocalPasswd yes
KerberosTicketCleanup yes
Banner /etc/banner
8. Modify /etc/pam.d/system-auth-ac and /etc/pam.d/password-auth-ac as follows,
making any changes needed to fit this into your setup.
auth required pam_env.so
auth sufficient pam_unix.so nullok try_first_pass
auth sufficient pam_krb5.so use_first_pass
auth requisite pam_succeed_if.so uid >= 500 quiet
auth required pam_deny.so
account required pam_unix.so
account sufficient pam_localuser.so
account sufficient pam_succeed_if.so uid < 500 quiet
account [default=bad success=ok user_unknown=ignore] pam_krb5.so
account required pam_permit.so
password requisite pam_cracklib.so try_first_pass retry=3
password sufficient pam_unix.so md5 shadow nullok try_first_pass use_authtok
password required pam_deny.so
session optional pam_keyinit.so revoke
session required pam_limits.so
session [success=1 default=ignore] pam_succeed_if.so service in crond quiet use_uid
session required pam_unix.so
session optional pam_krb5.so
or as the following:
# User changes will be destroyed the next time authconfig is run.
auth required pam_env.so
auth sufficient pam_unix.so nullok try_first_pass
auth [default=ignore success=3] pam_succeed_if.so user ingroup ttuhsc debug
auth requisite pam_succeed_if.so uid >= 500 quiet
auth sufficient pam_krb5.so use_first_pass realm=YOUR-UNIV-KERB-DOMAIN (e.g., MYUNIVERSITY.EDU)
auth [default=ignore success=1] pam_succeed_if.so user notingroup ttuhsc debug
auth sufficient pam_krb5.so use_first_pass realm=TTUHSC.EDU
auth required pam_deny.so
account required pam_unix.so
account sufficient pam_localuser.so
account sufficient pam_succeed_if.so uid < 500 quiet
account [default=bad success=ok user_unknown=ignore] pam_krb5.so
account required pam_permit.so
password requisite pam_cracklib.so try_first_pass retry=3
password sufficient pam_unix.so sha512 shadow nullok try_first_pass use_authtok
password sufficient pam_krb5.so use_authtok
password required pam_deny.so
session optional pam_keyinit.so revoke
session required pam_limits.so
session [success=1 default=ignore] pam_succeed_if.so service in crond quiet use_uid
session required pam_unix.so
session optional pam_krb5.so
9. Reload and restart sshd and ntpd
service sshd reload
service ntpd restart
ntpdate your-univ-ntp-server (e.g., ntp.myuniversiry.edu)
(optional, if the difference of the system time to the ntp server time is too long,
this command can synchronize the system time to the ntp server)
10. If all the above steps are done but it still doesn't work, try running some
or all of the following
authconfig --test //If the output shows "pam_krb5 is disabled",
run "authconfig --enablekrb5 --update" to enable it. If the output shows
"pam_krb5 is enabled" and "krb5 kdc via dns is disabled",
modify /etc/krb5.conf to make sure it has "dns_lookup_realm = true",
and run "service sshd restart" to load the settings then check again.