Getting started with Vault
What’s a Network Protocol?
Vault is a popular tool for securely accessing sensitive data (secrets). It is often used in modern application architectures to tightly control access to keys and credentials. Vault can be used to authenticate users (machines or humans) to make sure they’re authorized to access a particular file.
Most applications need access to sensitive data in order to work properly and storing credentials in a configuration file is a considerably risky solution, even when the data is encrypted. Data security is a major concern nowadays, and organizations need a way to avoid unauthorized access or exfiltration to information such as database credentials, SSL certificates, API keys for external services or credentials for service-oriented architecture communication.
Vault makes it possible to control and manage access to secrets and transmit them safely within an organization or an environment by providing a unified interface to any secret. At the same time Vault provides tight access control and a detailed audit log to keep track of who accessed what information.
Vault key features
- Secure Secret Storage:
- Arbitrary key/value secrets storage
- Secrets are encrypted prior to writing them to persistent storage
- Accessing the raw storage is not enough to access your secrets
- Vault can write to disk, Consul, and more.
- Dynamic Secrets:
- Secrets can be generated on-demand for some systems (e.g. AWS S3, SQL databases, etc.)
- Example:
- An application needs to access an S3 bucket and asks Vault for credentials
- An AWS keypair with valid permissions is generated on demand
- Vault revokes dynamic secrets automatically after the lease time is up
- Data Encryption:
- Data encryption and decryption without storing it
- Encryption parameters can be defined by teams
- Eliminates the need for developers to design their own encryption methods in order to store secret data in a location such as a SQL database
- Leasing and Renewal:
- All secrets have a lease
- At the end of the lease, Vault automatically revokes that secret
- Clients can renew leases via built-in renew APIs
- Revocation:
- Vault has built-in support for secret revocation
- It is possible to revoke single secrets as well as trees of secrets (e.g. all secrets read by a specific user)
- Revocation assists in key rolling as well as locking down systems in the case of an intrusion
Stateless Protocols
The client sends a request to the server and the server responds back according to the current state. It uses less resources since the receiver must not retain or save session information from previous requests.
Examples:
- HTTP (Hypertext Transfer Protocol)
- UDP (User Datagram Protocol)
- DNS (Domain Name System)
Vault Components
Vault is made up of a persistence backend where all the secrets are stored, an API server that handles client requests and performs operations on secrets.
It also has secret engines and authentication methods, which are pluggable components that allow you to integrate with external systems.
All these components manage and protect the secrets in dynamic infrastructure (e.g. database credentials, passwords, API keys).
Benefits
Delegating all secret handling to Vault means not having to store secrets, instead you can notify Vault when you need to use them or discard them.
It allows you to use short-lived secrets, which shortens the time window where a secret can be stolen or used.
How it works
- All data is encrypted with an encryption key before it’s written to the store
- The first key is encrypted by another key (master key) used only at startup
- The master key does not need to be stored in the server, which means that not even Vault can access its saved data after startup
- The Vault instance adopts “sealed” state
- Vault can accept API requests once unsealed
- API requests need authentication
Authentication
Authentication is the process by which user or machine supplied information is verified against an internal or external system. Vault supports multiple auth methods including GitHub, LDAP, AppRole, and more. Each auth method has a specific use case.
Authentication is needed to access secrets in Vault. The client has to authenticate itself against an auth method by the use of either passwords or dynamic values to generate temporary tokens that allow the client to access a particular path.
A token is generated upon authentication and it can be compared to a session ID on a website, they are strings sent on every API request using a special HTTP header.
Tokens by default will form a parent-child relationship. A child token can’t have a higher level of privileges than its parent token, therefore. When a token is invalidated, all its descendants are also invalidated. It is also possible to create a child token with restrictive policies.
Use Cases
Before understanding use cases, it’s useful to know what Vault is. Below are some concrete use cases for Vault, but the possible use cases are much broader than what we cover.
General Secret Storage
Storage of sensitive environment variables, database credentials, API keys, etc.
This method is more secure than storing credentials in plaintext in files, configuration management, a database, etc. It does not only protect the plaintext version of these secrets but also records access in the Vault audit log.
Employee Credential Storage
Vaul is a good method for storing credentials that employees share to access (web-) services.
You can keep track of what secrets an employee accessed and when an employee leaves, it is easier to roll keys and understand which keys have and haven’t been rolled.
API Key Generation for Scripts
With the “dynamic secrets” feature, an AWS access key can be generated for the duration of a script, then revoked. The keypair will not exist before or after the script runs, and the creation of the keys is logged and can be traced.
Data Encryption
Vault can be used to encrypt/decrypt data that is stored elsewhere. This allows applications to encrypt their data while still storing it in the primary data store.
This eliminates the need for developers to know how to properly encrypt data. The responsibility of encryption is on Vault and the security team managing it, and developers just encrypt/decrypt data as needed.
Conclusion
Vault is a powerful and widely used solution for securing sensitive data and managing secrets in modern application architectures. It addresses the critical need for maintaining large number of confidential information with its access controls for credentials, API keys, database passwords, SSL certificates, and more.
By using Vault, organizations can secure their data and mitigate the risks associated with storing credentials in local configuration files or other insecure locations.