Ansible playbooks are useful. They help a lot with automating repetitive tasks or making changes on a lot of machines quickly. Keeping the playbooks in a Git repository can help with sharing and version control. However, what happens when we need to use credentials in the playbook? We certainly cannot keep them in plain text in a Git repository, right? That’s where Ansible vault comes in. It’s very similar to vars file, but it’s encrypted. Whatever you cannot keep in plain text, feel free to store in the vault where it’s locked behind a password. Everyone using the playbooks will need to know the password, but it’s still better than keeping sensitive data in plain text.
Creating a vault is very easy. Once you have a file you want to encrypt, for example foo.yml, creating a vault is as simple as executing ansible-vault encrypt foo.yml. You will be asked for a password and, after confirming it, the file will be encrypted. That’s it! Decrypting a file is just as easy. Executing ansible-vault decrypt foo.yml will prompt you for the password and, if the password is correct, decrypt the vault.
You can pretty much do anything with those two commands, but there are a couple more quality of life commands that can improve your workflow. Using ansible-vault view foo.yml will allow you to read the contents of the vault without worrying about encrypting it again. Similar to that, there’s ansible-vault edit foo.yml which allows you to edit a file and handles decrypting and encrypting using the same password for you. The last command for this section is ansible-vault rekey foo.yml. It allows you to decrypt the vault and encrypt it again using a different password. All of these, as well as the first two, allow listing multiple vaults as arguments which can make automation very simple.
Encrypting variables and files
It’s possible that you only have a couple values that should not be stored in plain text. Having a whole new file for them might seem like a waste. Luckily, it’s possible to encrypt some variables in vars file and keep the situation relatively clean. The drawback is having to encrypt every value separately using the same password. Also, when it comes to password rotation, it is going to be more difficult than simply using ansible-vault rekey on a vault file. By the way, keep in mind that this feature is not supported everywhere. Using it in vars or inventory file should work, but if you’re dynamically including some files, it’s possible that the !vault tag will not be recognized.
Ansible vault also offers the functionality of encrypting arbitrary files and using them in your playbooks. The files can be freely used in ansible tasks as if they were not encrypted at all. As long as the password is provided, Ansible will decrypt files on the fly and use them instead. Ansible also allows encryption of its own files, such as tasks and handlers. So if you want to keep some parts of your playbook secret or prevent someone from editing the tasks. Of course, that means the files won’t be as easy to read and it can make debugging a pain.
Encrypted application configuration
Using Ansible vault to store secrets required for applications is certainly possible. If you’re already using Ansible for deploying your application, it should be relatively easy to add another task that copies configuration as well. If you’re deploying the application on a server, it can be as simple as adding an encrypted configuration file to your playbook and copying it to the server before starting the application. In case of deploying to Kubernetes or OpenShift, it can require slightly more effort, but it’s certainly not an issue. You can simply encrypt a Kubernetes secret manifest and apply it to the appropriate namespace when deploying the application. You can even keep most of the configuration in plain text and apply secrets from the vault using Jinja2 templates. However, keep in mind that using Ansible vault has its drawbacks.
As the files are encrypted, it’s really easy to make a simple mistake, such as a typo, and have it go through code review unnoticed. It would require decrypting the vault to look through its contents, after all. If you only want to see what’s changed, you would have to manually decrypt the old and the new version and then check the differences. A lot more work than just running git diff. The worst part is that in case of a vault used only in production or similar environment that might not be deployed often, the error most likely won’t be noticed until a deployment fails. That is usually the point at which all hell breaks loose.
It’s also possible for vault to degrade to per-environment variable storage, especially with distributed vaults maintained by different teams. Difficulties with finding differences between versions of a vault prove to be an issue here as well, since any errors done while synchronizing the vault contents will only become apparent once the application is deployed. It’s possible to reduce the number of mistakes by keeping all the variables listed in vars file and just extracting the values from the vault using Jinja2 syntax, but typos are still possible.
There’s also an issue with saving boolean values in the vault. It’s possible to write boolean values without quotes in the vault and just use them as booleans anywhere in the playbook. You could also keep them quoted and parse to boolean when required. And if the playbook expects the value to be without quotes and vault ends up with a quoted value, running it will cause an error. Fun!
Also, be careful when generating passwords. Backslash is a special character in YAML specification and having it in a string can cause subtle bugs. Imagine having a newline in your password when you actually wanted a \n instead. Also, having a single backslash without an escapable character or sequence after it will cause a syntax error that might leave you stumped for a while. This error can be avoided by not using double quotes to represent strings, but that can bring another set of problems because Ansible has a few special characters that can cause errors. The best way to solve this is to use single quotes since they don’t support escaping and avoid issues with Ansible special characters.
Using multiple vaults
It’s possible to use multiple vaults with the same playbook. That way, it’s possible to have a vault per environment, containing sensitive information locked behind a different password. The way of getting the information is equal, just the vault id and password are different.
Ansible vault is a powerful tool. It provides encryption mechanisms for your playbooks in order to keep information secret and prevent unauthorized changes, all while still allowing you to keep it under version control. It is flexible enough to be useful in different use cases, but it is by no means a universal solution. It is possible to run into some issues while using it, as well as abuse some of its features. However, when used properly and with care, it can prove to be a very valuable addition to your Ansible playbooks.