If you currently store sensitive credentials in plaintext to automate scripting or integration to other systems, you should consider an extra layer of security by storing them encrypted using GPG.
There is no fullproof way to hide sensitive information for a service that also needs to decrypt them as part of normal operations (think DVD players). This applies to all vault type security systems and not just GPG, if the system is compromised at the consuming level then the attacker will have access to intercept sensitive information.
But if you focus on defense in depth and layered security – then keeping sensitive information encrypted, in distinct files, with proper permissions and auditing is part of an overall security strategy.
If you are beyond local management of secrets and are looking for a full fledged solution to secret management, start your research with something like HashiCorp Vault.
Overview
The problem we will address in this article is the need to supply a plaintext password for use in automation. This might be a password to a database, or credentials to an enterprise integration service.
While the best practice is to avoid plaintext passwords (instead using a token, env variable, private key, etc.), the truth is that is not always an option. And on top of that, entering a manual passphrase or inserting a USB stick that holds the private key is not usually an option when managing at scale and needing services, hosts, containers, and workflows to rebuild, restart, and be invoked without manual intervention.
This article will provide one more layer of defense by introducing GPG encryption as an option to replace the plaintext information currently stored on a host. The private key will be secured using file permissions.
I will provide several examples in this article:
- Example 1: How a group with full sudo access can decrypt a secret when the private key is accessible only by root
- Example 2: How a group with limited sudo access can invoke a shell script that utilizes the secret without exposing it to the user
- Example 3: How a member of a group can decrypt a secret without sudo when the private key is readable by the group
Prerequisites
Packages
sudo apt-get update sudo apt-get install python-gnupg rng-tools -y sudo rngd -r /dev/urandom
Groups/Users
We are going to create 2 groups: ‘deployers’ and ‘developers’.
- deployers
- alice – password ‘alicepass’
- bob – password ‘bobpass’
- developers
- billy – password ‘billypass’
‘deployers’ will be those who have production operational needs, and will be the only ones who should can view sensitive credentials as well as possessing full sudo privileges.
‘developers’ only have select capabilities in this production scenario and will not have access to view secrets. They have sudo privileges only to run a single specific script, “/tmp/developersDecrypt.sh”.
There will be two user in the ‘deployers’ role, Alice and Bob. They will both have privileges to view and decrypt the sensitive information. They work with a developer named Billy who should not have any access to the credentials.
A script that installs these packages, then creates the users, groups, and sudo access is located in my github repository as “gpgsetup.sh”. The full Vagrant file is on github here.
Create encrypted secret
With a user that has full sudo access (either alice or bob), generate a private-public keypair in a folder visible only to root, and then write an encrypted secret to a location available to all as shown below.
sudo mkdir /etc/deployerkeys sudo chmod 600 /etc/deployerkeys cat >genkeyinput <<EOF %echo Generating a basic OpenPGP key Key-Type: RSA Key-Length: 2048 Name-Real: deployers Expire-Date: 0 %no-protection %commit %echo done EOF # generate key pair sudo gpg --batch --gen-key --homedir /etc/deployerkeys genkeyinput # start this command with a space, not saved to bash history echo -n "MyP4ss!" | sudo gpg --armor --batch --trust-model always --encrypt --homedir /etc/deployerkeys -r deployers > /tmp/test.crypt
Now we have our private key accessible only to root in the “/etc/deployerkeys” folder, and and encrypted secret located at “/tmp/test.crypt” which is publicly available.
A script that runs these commands is located in my github repository as “createsecret.sh”.
Example 1: User with full sudo
The “/etc/deployerkeys” directory is owned by root and has read/write permissions only for root. But for a group entrusted with full sudo such as ‘deployers’ (granted in ‘/etc/sudoers.d/deployers’), this directory is accessible and can be used to decrypt a file.
Here is how Alice or Bob would decrypt the secret.
# show ability to read keys sudo gpg --homedir /etc/deployerkeys --list-keys # do actual decryption sudo gpg --homedir /etc/deployerkeys -qd /tmp/test.crypt
This command could be used in any script run by a member of the ‘deployers’ group.
Example 2: User with limited sudo
As opposed to our last example, Billy is a member of the ‘developers’ group. With only limited sudo, access to the private key to decrypt the file is not possible.
If Billy tries to run the same decrypt command as Alice or Bob from example 1, he will be told that his user is not allowed to run gpg.
$ sudo gpg --homedir /etc/deployerkeys -qd /tmp/test.crypt [sudo] password for billy: Sorry, user billy is not allowed to execute '/usr/bin/gpg --homedir /etc/deployerkeys -qd /tmp/test.crypt' as root on gpg1604.
However, Billy does have targeted sudo privileges to execute a single script, “/tmp/developersDecrypt.sh” (granted in ‘/etc/sudoers.d/developers’). I have a script in my github repository named “fordevelopers.sh” that will create this bash script in /tmp and set its ownership properly.
If Billy attempts to run the script without sudo, the script will warn that errors should be expected and there will then be multiples lines of permissions errors related to files in “/etc/deployerkeys”.
$ /tmp/developersDecrypt.sh /tmp/developersDecrypt.sh invoked as user 'billy' which is member of groups: developers script being run as user id 20003 EXPECT ERROR!!! this script must be run as root/sudo ....
But if he instead runs the script with sudo access:
$ sudo /tmp/developersDecrypt.sh [sudo] password for billy: /tmp/developersDecrypt.sh invoked as user 'billy' which is member of groups: root script being run as user id 0 gpg: checking the trustdb gpg: 3 marginal(s) needed, 1 complete(s) needed, PGP trust model gpg: depth: 0 valid: 1 signed: 0 trust: 0-, 0q, 0n, 0m, 0f, 1u /etc/deployerkeys/pubring.gpg ----------------------------- pub 2048R/7EFD9DA1 2018-10-30 uid deployers NOTE: This decrypted secret would normally be silently passed to another process, and not ever shown or available to the user. But for purposes of example, here it is: MyP4ss!
Then, as the last line shows, the decrypted password is available to the script, even though it was not available to billy or his group. As noted in the output, in a real-life scenario, this decrypted value would not be shown or exposed to Billy. It would be used silently.
In this way, Billy can take on limited operational tasks (checking a DB connection, connecting to an API, etc.) without needing production credentials.
Example 3: Private key readable by group
In the first two examples, we had a directory owned by root protecting the private key. In this last example, I introduce the idea that the private key could kept in a folder readable by a group – and in that way a group could share amongst themselves a decryption key.
I understand that some may balk at the idea that a private key would be shared among a group, and would instead want to see the public/private key exported, and then each member of the group would individually import the public/private key to their own personal keyring. If you believe going this route is required, then please go ahead. But if you are going that far, then you should also have each individual user passphrase protect their key, and now you have given up silent and unattended operations.
Rather, I feel that a tightly controlled and audited user group can be considered a single origin. In this modern era of computing, hosts must be managed by function and not managed individually (cattle not pets). And those who manage these systems must do so by role, and not as individuals. When I consider how these keys need to be managed: onboarding of new users to a group, user role revocation, etc., I personally see individual management of keyrings and private keys as more fragile, inaccurate, and insecure. That is why I feel that a keypair readable by a group and centrally managed and updated is an acceptable practice.
Remember, that the whole point of this article is to get away from keeping plaintext passwords in the clear. If you are looking for a centralized, full-fledge secret solution, those are available.
Building on our examples from above, we have the private key accessible only as root, which means the only way a normal user can gain access is via sudo. But if we change the ownership to be readable by the group then members of the ‘deployers’ group will be able to decrypt without using sudo.
Execute the commands below as either root or a member of the deployers group which has full sudo privileges.
# private keys also owned by group sudo chown -R root:deployers /etc/deployerkeys # for directory itself: root=rw, deployers=rx sudo chmod 650 /etc/deployerkeys # for files in directory: root=rw, deployers=r sudo chmod 640 /etc/deployerkeys/*
Now login as Alice or Bob and decrypt the secret without using sudo. The unsafe ownership message is only a warning and can be ignored.
# do decryption with no need for sudo gpg --homedir /etc/deployerkeys --no-random-seed-file -qd /tmp/test.crypt
In this way, any member of the ‘deployers’ group can use decryption, and elevated system privileges are not required.
REFERENCES
https://www.gnupg.org/gph/en/manual.html (GnuPG man)
https://www.gnupg.org/documentation/manuals/gnupg/GPG-Configuration.html (gpg configuration files list)
https://zacharyvoase.com/2009/08/20/openpgp/ (intro and details for OpenPGP)
https://superuser.com/questions/655246/are-gnupg-1-and-gnupg-2-compatible-with-each-other (describes differences, GnuPG 1, GnuPG 2, GnuPG 2.1)
http://thegeekyway.com/hands-on-guide-on-gpg-keys/ (gpg2 usage)
https://superuser.com/questions/189355/is-it-ok-to-share-private-key-file-between-multiple-computers-services (private key per origin)
http://irtfweb.ifa.hawaii.edu/~lockhart/gpg/ (example scenarios for wanting to share private key)
https://lists.gnupg.org/pipermail/gnupg-users/2005-October/027242.html (settings: keyring, primary-keyring)
https://github.com/MassimoLauria/global-configuration/blob/master/gpg.conf (example of primary-keyring use in gpg.conf)
https://stackoverflow.com/questions/3318853/hide-encrypt-password-in-bash-file-to-stop-accidentally-seeing-it (encrypt with key on usb, no default keyring, read into env var)
https://serverfault.com/questions/666142/is-there-a-safe-way-to-store-passwords-used-for-ssh-by-a-script (nothing full proof. focus on defense in depth, encrypted passwords in separate file)
https://lists.gnupg.org/pipermail/gnupg-users/2004-November/023735.html (sharing secret keyring with coworker)
https://www.gnupg.org/documentation/manuals/gnupg/Unattended-GPG-key-generation.html (unattended key generation)
https://www.techonthenet.com/linux/sysadmin/ubuntu/create_user_14_04.php (creating user)
https://www.vaultproject.io/docs/vs/index.html (HashiCorp Vault versus other secret management solutions)
https://www.hashicorp.com/blog/using-hashicorps-vault-with-chef (different ways to pull values into use)
https://serverfault.com/questions/413397/how-to-set-environment-variable-in-systemd-service (methods for systemd to use EnvironmentFile for secrets or ExecStart for bash execution)
https://www.reddit.com/r/archlinux/comments/6fi61s/how_to_use_a_systemd_service_with_secret_variables/ (look at systemd override files)
https://gist.github.com/nickjacob/9909574 (ExecStartPre to load env var used in ExecStart, PermissionStartOnly=true if only ExecStart must run as ‘User’)
NOTES
Set a personal default keyring and secret keyring if you do not want to specify –homedir
gpg --list-keys (creates .gnupg/gpg.conf) echo "keyring /etc/deployerkeys/pubring.gpg" >> .gnupg/gpg.conf echo "secret-keyring /etc/deployerkeys/secring.gpg" >> .gnupg/gpg.conf
Commands that create gpg conf files
gpg --list-keys (gpg.conf, pubring.gpg, trustdb.gpg) gpg --list-secret-keys (keyring created ~/.gnupg/secring.gpg')
Check for error when decrypting
gpg --homedir /etc/gpgkeys -qd /tmp/test.crypt echo $?