To solve this problem, I wrote git-crypt[1], which uses git's smudge/clean filters to transparently encrypt/decrypt files when you check them in/out. So it's a lot like this solution except you don't need the manual makefile steps. As an added bonus, git diff/blame still work on the encrypted file.<p>[1] <a href="https://github.com/AGWA/git-crypt" rel="nofollow">https://github.com/AGWA/git-crypt</a> and <a href="http://www.agwa.name/projects/git-crypt/" rel="nofollow">http://www.agwa.name/projects/git-crypt/</a>
First: I completely agree. Keeping plaintext secrets in source control is a bad idea. Encrypting them is a good idea. If you have plaintext secrets, encrypt them <i>now</i> using this makefile or git-crypt. Then rotate them.<p>That said, this solution has a couple of issues:<p>1. It encrypts the entire file instead of individual secrets in the settings file. Encrypted files can't take advantage of many version control features. A small change in plaintext creates a huge diff in ciphertext. Git blame doesn't work anymore. Git diff gets a lot more spammy, since you'll see a diff for the entire settings file if there's the slightest change in it.<p>2. It uses symmetric key encryption. If a developer knows the password to encrypt a secret setting, they can decrypt all the other secret settings. This is true until someone rotates the passphrase and re-encrypts the file.<p>To fix both of these problems, I recommend using Keyczar (<a href="http://code.google.com/p/keyczar/" rel="nofollow">http://code.google.com/p/keyczar/</a>). If you write the right wrappers, it allows you to encrypt individual settings with a public key. Decrypting them requires a private key that exists only on production servers.<p>At a past job (Cloudkick), sensitive things in our settings.py looked like this:<p><pre><code> from cloudkick.crypto_wrappers import kz_decrypt
...
BORING_THING = "whatever"
SECRET_THING = kz_decrypt("kz::xxxx....", "/path/to/private/key")
</code></pre>
kz_decrypt did exactly what you'd think: given an encrypted string and a private key, return the decrypted string. The private key was only on production servers, so the risk of leaking a secret was minimal. The public key was in source control, so anyone could encrypt a secret. For debugging or testing, one could also replace the call to kz_decrypt with a plaintext string. I wish the code had been released. It was only 100 lines or so.<p>This set-up would require a some extra work for settings files that don't allow code execution. Still, once you've set it up, it's pretty close to the most secure and convenient way to store secrets.
I don't think storing passwords in any format in source control is good. Like someone else said, it's mixing app logic with deployment.<p>We use a combination of Google's open source Keyczar [1] and a relatively new Python keyring [2] library which uses the Keyczar crypter to read/write keys to a keyring storage backend, where the backend interface can be implemented with local crypted files or a cloud service.<p>[1] <a href="http://www.keyczar.org/" rel="nofollow">http://www.keyczar.org/</a>
[2] <a href="http://pypi.python.org/pypi/keyring" rel="nofollow">http://pypi.python.org/pypi/keyring</a><p>We built this Python wrapper called appauth around of the concept of a Keyring service by application domain.<p>e.g. pseudo code:<p><pre><code> import appauth
auth_service = appauth.AuthService('my-web-app')
db_creds_cfg = auth_service.get('primary-db')
</code></pre>
Inside of db_creds_cfg, it can be a free-form dictionary that provides whatever details is needed to get into a resource:<p><pre><code> db_creds_cfg['db_host']
db_creds_cfg['db_port']
db_creds_cfg['username']
db_creds_cfg['password']
</code></pre>
I put in some honest effort to find an open source solution to this, but failed to find anything with a simple install process AND programming interface. Is there any interest from HN if we choose to open source this?<p>Furthermore, we use Google Authenticator on our servers to require two-factor auth: <a href="http://code.google.com/p/google-authenticator/" rel="nofollow">http://code.google.com/p/google-authenticator/</a>, on top of disabling password auth in favor of signing in with ssh keys. All log files are then either set to permission 600 just to be super paranoid.
Actually, having no passwords and using a platform which supports integrated authentication (like Windows) is probably the best approach with respect to handling this. The authentication requirements are handled at an infrastructure level, meaning no credentials are kept in source control or on your production web servers.<p>In fact, none of our web servers carry ANY credentials at all. Our IIS processes run as a specific user and are granted access to resources (message queues, databases etc) as required.<p>I'm not sure stuff like this is entirely possible on Linux (I haven't tried to be honest), but I assume you can do the equivalent with OpenLDAP / pam_ldap and SELinux.
In a previous job, we had a lot of components with configuration files containing credentials for databases, etc. What we had done instead, was to put placeholder tokens (imagine %DB_ROLE1_PASSWORD%) in the configuration files, and then puppet (chef later) would be used to deploy the packages and replace the tokens. In this way, no developer ever knew of the production passwords, and only the system admins had access to the source control for the puppet scripts. There really shouldn't be any access to production credentials by developers if you have separate roles for developer and admin. (Some companies are too small, I know =)
This seems pretty reasonable. The obvious risk is that if the encryption keys leak, all of your credentials may be retroactively compromised...<p>Still, much better than keeping the passwords in VC unprotected, of course.<p>My preference still is using environment variables so that the secure bits can be fully decoupled from your code, however..
I like this idea. I would add one tip though, use a group password safe, rather than contacting person X. As a sysadmin we have passwords all over the place (root, network, wifi, desktop, remote sites, etc, etc). There are five people on our team, and we use Password Safe (windows) [1] and/or KeePassX (linux/mac) [2] to manage lots of passwords. You do not have to contact person X, for a password, if the are away for some reason, just check the safe.<p>[1] <a href="http://passwordsafe.sourceforge.net/" rel="nofollow">http://passwordsafe.sourceforge.net/</a><p>[2] <a href="http://www.keepassx.org/" rel="nofollow">http://www.keepassx.org/</a>
I recommend the scrypt command line utility [1] instead of openssl. Openssl use md5 as a key derivation function [2], and cost of recovering a reasonable length, randomly generated password is surprisingly low [3]. The costs in the presentation are from 2009, and I can only imagine how they've dropped thanks to a few years of bitcoin-driven gpu/hardware developments. If you trust your code host, e.g. github or bitbucket, this isn't a concern, but neither are plaintext passwords in version control. If you're using a very long, randomly generated password, you're safe as well.<p>The disadvantage is that you'll need to compile scrypt from source.<p>[1] <a href="http://www.tarsnap.com/scrypt.html" rel="nofollow">http://www.tarsnap.com/scrypt.html</a><p>[2] slide 20: <a href="http://www.tarsnap.com/scrypt/scrypt-slides.pdf" rel="nofollow">http://www.tarsnap.com/scrypt/scrypt-slides.pdf</a><p>[3] slide 19: <a href="http://www.tarsnap.com/scrypt/scrypt-slides.pdf" rel="nofollow">http://www.tarsnap.com/scrypt/scrypt-slides.pdf</a>
I'm sorry but storing sensitive information (e.g. passwords) in SCM is a terrible idea even if they are encrypted.<p>Why are you mixing deployment with development? They should be two different things IMHO.
Since we manage our servers with Puppet, we use hiera-gpg to securely store sensitive information in encrypted form in git. Puppet then safely deploys these files to our servers and our application deployment process (Capistrano) symlinks/copies these config files in to the application as part of the deployment process. The sensitive config files themselves are excluded from our application's git repository and developers keep local copies of these files (containing local dev. credentials only) for development purposes.<p>More info on hiera-gpg here: <a href="http://www.craigdunn.org/2011/10/secret-variables-in-puppet-with-hiera-and-gpg/" rel="nofollow">http://www.craigdunn.org/2011/10/secret-variables-in-puppet-...</a>
I find only keeping boilerplate configs in repos helps decouple application code from any one installation. Software can sometimes be used in more than one environment, it's not always a one-to-one relationship.
you're still exposing yourself by putting your settings and credentials albeit encrypted, out there. i dont like this approach at all, i'd prefer either environment variable or a more ruby way of doing things like using a rake command to convert an erb to yaml file. make sure you then encrypt or at least obfuscate credentials in the config file (base64 or encryption), though hackable if you can read ruby, but at least you're adding another layer of indirection.
> console.error("Did you forget to run `make decrypt_conf`?");<p>> console.error("You need to run `make decrypt_conf` to update it.");<p>Couldn't you just make the decrypt_conf target depend on the encrypted configuration file and make the standard build command depend on the decrypted file? This way it would get enforced with every 'make'.<p>In case you don't use Makefiles for building at all (because you only use script languages for example) I don't get why you use a Makefile instead of just two shell scripts.
I prefer using environment variables, and then enforcing this in the Makefile: <a href="https://gist.github.com/ChimeraCoder/4728823" rel="nofollow">https://gist.github.com/ChimeraCoder/4728823</a><p>If this is the first target (or a prerequisite of the first target), then running 'make' will ensure that those variables are set to non-empty strings.
<a href="https://github.com/jimktrains/polygonus" rel="nofollow">https://github.com/jimktrains/polygonus</a> I wrote this to allow us to encrypt and search passwords. The encrypted file may be kept under version control, though I don't know what types of attacks that could aid.
and i would also like to comment that you should be very careful not to commit code in source control that hardcodes credentials because there's a history that could be exploited.
the fixing of the problem is easy , use freaking envirronment variables ! there is not one language/framework whatever that doesnt support them , so for exemple in node :<p>> var db_password = process.env.DB_PASS ;<p>You dont have to keep any password sensitive file whatever inside an opensource project. that is what envirronment variables are fucking made for !