The Three Musketeers - Jenkins, Terraform, Vault

Introduction to DevOps

DevOps is evolving perpetually and rapidly in the IT industry due to its feature of producing high-quality products at a quick pace. Automation is the fundamental principle of DevOps, and it utilizes the cloud paradigm to provide agility, flexibility, and automation in the software development life cycle. The challenge that is associated with building these automated pipelines is the dynamic nature of the cloud and as well as the need for controlling the access to these resources. We have to decide which user has access to our cloud resources like RDS instance or google function, to do so cloud providers provide us tools that offer us lots of norms that we can turn to decide very granular permission on what action can be executed by the user or a particular application making a call to our cloud account.

DevOps Solutions and Services for application modernization and Cloud-native Enterprise applications for continuous Delivery Explore our Services, DevOps Consulting Services and Solutions Company

Identity and Access Management(IAM):

Identity and Access Management is a web service that helps us securely access cloud resources, and This tool helps us in providing authentication and authorization to a user or app to access cloud resources. It uses the access key and secret key to make a call to cloud providers, and cloud providers will permit us according to these keys. These keys are stored on the application side, and it’s essential to secure these keys because if these secrets are compromised, then our cloud accounts can get hijacked that can be used for bitcoin mining or in worst case scenario it can lead to data breaches. To secure all of these keys and manage cloud policies all from within our CI/CD pipeline we use Three DevOps tools (Jenkins, Terraform and Vault) and these tools are also known as the three musketeers.

The 3 Musketeers: Jenkins, Terraform, Vault

Jenkins

Jenkins is an open-source automation tool, and it has excellent integration and plugins for AWS, Azure, Terraform, etc. It also can build a sophisticated pipeline and also supports pipeline as code. Also, Read Implementing Pipeline as Code with Jenkins and Apache Groovy

Terraform

Terraform is an open-source infrastructure provisioning tool, and it also provides Infrastructure as code, it has plugins for all major cloud providers and also supports remote state management. One of the methods to secure tokens of the cloud provider is by using Jenkins and Terraform; we use Jenkins credential store for that, Jenkins credential is a plugin that provides us with an internal credential store, we can use it to store the secret key of the cloud and Terraform will pick it up and deploy it on the cloud. Still, it isn’t the best method to secure the keys of the cloud because:

It cannot revoke the key.
We need to manage different plugins for every cloud provider and application type.
It can not generate tokens based upon user/machine identity.
For application, storing secrets is not a good practice because there is a risk of these secrets getting exposed to the git repository.

Due to these drawback of Jenkin Credential, Vault came into the picture: Also, Read Infrastructure as Code (IaC) using Terraform Overview

Vault

Vault is an open-source secret store use to access the secrets securely, AWS supports this secret engine, Azure, GCP along with k8s and it also provides authentication methods like Github, Okta, LDAP, etc.

Why Vault

It a centralized secrets store that comes with encryption
It also provides granular policies designed to control the permission to every key stored within the vault
It gives the ability to attach the lease period and also we can renew the lease period.
It is cloud-agnostic, and it also supports dynamic secret.
It has integration with various opensource and licensed databases like influx, MongoDB, Hana as well as Kubernetes.

Also, Read Enterprise-Grade Secret Management using Vault for Kubernetes

User-Identity based authentication

In this method, the pipeline is built in such a manner that it tracks the user identity. Also, it provides end to end tracking of the user or developer, and the uses identity metrics for the authentication.

It uses LDAP, Github, Okta, etc., for authentication and returns a token.
Policies can be designed for organizations/teams/users.
It provides audit logs of credential usage based on user identity.

AppRole based authentication

It is used for the machine to machine identification, and this method is used to identify a particular application that is making a call to the cloud environment.

It uses the application and needs role_id and secret_id to return the token.
In this method, policies can be designed based on application type.
It also provides audit logs of credential usage based on user identity.

Conclusion

These tools help in generating tokens based on the identity of a user or a machine. By using these tools, we can securely store the cloud credentials along with application secrets. The three Musketeers access keys get destroyed after the deployment to prevent it from misuse. They also help in Auditing of usage credential is possible. Additionally, these tools can work on any cloud provider via pipeline. Download the use-case Triggering Jobs using Jenkins and Powershell