Protect sensitive input variables

13min
|
Terraform

Often you need to configure your infrastructure using sensitive or secret information such as usernames, passwords, API tokens, or Personally Identifiable Information (PII). When you do so, you need to ensure that you do not accidentally expose this data in CLI output, log output, or source control. Terraform provides several features to help avoid accidentally exposing sensitive data.

In this tutorial, you will use Terraform to deploy a web application on AWS, including a VPC, load balancer, EC2 instances, and a database. You will replace the database's hard-coded credentials with variables configured with the sensitive flag. Terraform will then redact these values in the output of Terraform commands or log messages. Next, you will set values for these variables using environment variables and with a .tfvars file. Finally, you will identify the sensitive values in state, and learn about ways to protect your state file.

Prerequisites

You can complete this tutorial using the same workflow with either Terraform Community Edition or HCP Terraform. HCP Terraform is a platform that you can use to manage and execute your Terraform projects. It includes features like remote state and execution, structured plan output, workspace resource summaries, and more.

Select the HCP Terraform tab to complete this tutorial using HCP Terraform.

This tutorial assumes that you are familiar with the Terraform workflow. If you are new to Terraform, complete the Get Started tutorials first.

In order to complete this tutorial, you will need the following:

Terraform v1.2+ installed locally.
An AWS account with local credentials configured for use with Terraform.
The git CLI.

This tutorial assumes that you are familiar with the Terraform and HCP Terraform workflows. If you are new to Terraform, complete the Get Started tutorials first. If you are new to HCP Terraform, complete the HCP Terraform Get Started tutorials first.

In order to complete this tutorial, you will need the following:

Terraform v1.2+ installed locally.
An AWS account.
A HCP Terraform account with HCP Terraform locally authenticated.
A HCP Terraform variable set configured with your AWS credentials.
The git CLI.

Note

Some of the infrastructure in this tutorial may not qualify for the AWS free tier. Destroy the infrastructure at the end of the tutorial to avoid unnecessary charges. We are not responsible for any charges that you incur.

Create infrastructure

Clone the Learn Terraform Sensitive Variables GitHub repository for this tutorial.

$ git clone https://github.com/hashicorp-education/learn-terraform-sensitive-variables

Change to the repository directory.

$ cd learn-terraform-sensitive-variables

This configuration defines a web application, including a VPC, load balancer, EC2 instances, and a database.

Initialize this configuration.

$ terraform initInitializing the backend...##...Terraform has been successfully initialized!You may now begin working with Terraform. Try running "terraform plan" to seeany changes that are required for your infrastructure. All Terraform commandsshould now work.If you ever set or change modules or backend configuration for Terraform,rerun this command to reinitialize your working directory. If you forget, othercommands will detect it and remind you to do so if necessary.

Open your terraform.tf file and uncomment the cloud block. Replace the organization name with your own HCP Terraform organization.

terraform {  cloud {    organization = "organization-name"    workspaces {      name = "learn-terraform-sensitive-variables"    }  }  required_providers {    aws = {      source  = "hashicorp/aws"      version = "~> 4.4.0"    }  }  required_version = ">= 1.2"}

Initialize your configuration. Terraform will automatically create the learn-terraform-sensitive-variables workspace in your HCP Terraform organization.

$ terraform initInitializing HCP Terraform...Initializing provider plugins...- Reusing previous version of hashicorp/aws from the dependency lock file- Installing hashicorp/aws v4.4.0...- Installed hashicorp/aws v4.4.0 (signed by HashiCorp)HCP Terraform has been successfully initialized!You may now begin working with HCP Terraform. Try running "terraform plan" tosee any changes that are required for your infrastructure.If you ever set or change modules or Terraform Settings, run "terraform init"again to reinitialize your working directory.

Note

This tutorial assumes that you are using a tutorial-specific HCP Terraform organization with a global variable set of your AWS credentials. Review the Create a Credential Variable Set for detailed guidance. If you are using a scoped variable set, assign it to your new workspace now.

Now apply the configuration to create the example infrastructure.

$ terraform apply Running apply in HCP Terraform. Output will stream here. Pressing Ctrl-Cwill cancel the remote apply if it's still pending. If the apply started itwill stop streaming the logs, but will not stop the apply running remotely. Preparing the remote apply... To view this run in a browser, visit:https://app.terraform.io/app/hashicorp/learn-terraform-sensitive-variables/runs/run-3nZZ9owarvBTaMBs Waiting for the plan to start... Terraform v1.2.3on linux_amd64Initializing plugins and modules...module.ec2_instances.data.aws_ami.amazon_linux: Reading...data.aws_availability_zones.available: Reading...data.aws_availability_zones.available: Read complete after 0s [id=us-east-1]module.ec2_instances.data.aws_ami.amazon_linux: Read complete after 2s [id=ami-065efef2c739d613b] Terraform used the selected providers to generate the following executionplan. Resource actions are indicated with the following symbols:  + create ## ... Plan: 36 to add, 0 to change, 0 to destroy.  Do you want to perform these actions in workspace "learn-terraform-sensitive-variables"?  Terraform will perform the actions described above.  Only 'yes' will be accepted to approve.   Enter a value: yes ## ... Apply complete! Resources: 36 added, 0 changed, 0 destroyed.

Respond to the confirmation prompt with a yes.

Tip

This tutorial shows the output for Terraform commands run with Terraform Community Edition. If you are following the HCP Terraform workflow, the output may differ slightly but the results will be the same.

If you use HCP Terraform to provision your resources, your workspace now displays the list of all of the resources it manages.

Terraform workspace resource overview

Refactor database credentials

Open main.tf in your text editor. Near the bottom of the file, find the aws_db_instance.database block that defines your database. The database username and password are hard-coded. Refactor this configuration to remove these values.

First, declare input variables for the database administrator username and password in variables.tf.

variable "db_username" {  description = "Database administrator username"  type        = string  sensitive   = true} variable "db_password" {  description = "Database administrator password"  type        = string  sensitive   = true}

Notice that you've declared the variables as sensitive. Now update main.tf to reference these variables.

resource "aws_db_instance" "database" {  allocated_storage = 5  engine            = "mysql"  instance_class    = "db.t3.micro"-  username          = "admin"-  password          = "notasecurepassword"+  username          = var.db_username+  password          = var.db_password   db_subnet_group_name = aws_db_subnet_group.private.name   skip_final_snapshot = true}

If you were to run terraform apply now, Terraform would prompt you for values for these new variables since you haven't assigned defaults to them. However, entering values manually is time consuming and error prone. Next, you will use two different methods to set the sensitive variable values, and learn about security considerations of each method.

Set values with a `.tfvars` file

Terraform supports setting variable values with variable definition (.tfvars) files. You can use multiple variable definition files, and many practitioners use a separate file to set sensitive or secret values.

Create a new file called secret.tfvars to assign values to the new variables.

db_username = "admin"db_password = "insecurepassword"

Apply these changes using the -var-file parameter. Respond to the confirmation prompt with yes.

$ terraform apply -var-file="secret.tfvars"random_string.lb_id: Refreshing state... [id=2Mw]module.vpc.aws_vpc.this[0]: Refreshing state... [id=vpc-05f973211a47fb6f4]   ## ...   # aws_db_instance.database will be updated in-place  ~ resource "aws_db_instance" "database" {      + domain                                = ""      + domain_iam_role_name                  = ""        id                                    = "terraform-20210113192204255400000004"      + kms_key_id                            = ""      + monitoring_role_arn                   = ""      + name                                  = ""      # Warning: this attribute value will be marked as sensitive and will      # not display in UI output after applying this change      ~ password                              = (sensitive value)      + performance_insights_kms_key_id       = ""      + replicate_source_db                   = ""        tags                                  = {}      + timezone                              = ""      # Warning: this attribute value will be marked as sensitive and will      # not display in UI output after applying this change      ~ username                              = (sensitive)        # (41 unchanged attributes hidden)    } Plan: 0 to add, 1 to change, 0 to destroy.

Because you flagged the new variables as sensitive, Terraform redacts their values from its output when you run a plan, apply, or destroy command. Notice that the password is marked sensitive value, while the username is marked sensitive. The AWS provider considers the password argument for any database instance as sensitive, whether or not you declare the variable as sensitive, and will redact it as a sensitive value. You should still declare this variable as sensitive to make sure it's redacted if you reference it in other locations than the specific password argument.

Setting values with a .tfvars file allows you to separate sensitive values from the rest of your variable values, and makes it clear to people working with your configuration which values are sensitive. However, it requires that you maintain and share the secret.tfvars file with only the appropriate people. You must also be careful not to check .tfvars files with sensitive values into version control. For this reason, GitHub's recommended .gitignore file for Terraform configuration is configured to ignore files matching the pattern *.tfvars.

Set values with variables

Set the database administrator username and password using environment variables for Terraform Community Edition or Terraform variables for HCP Terraform.

When Terraform runs, it looks in your environment for variables that match the pattern TF_VAR_<VARIABLE_NAME>, and assigns those values to the corresponding Terraform variables in your configuration.

$ export TF_VAR_db_username=admin TF_VAR_db_password=adifferentpassword

HCP Terraform provides secure variable management by encrypting all variable values and allowing you mark them as sensitive during creation.

Marking a variable as sensitive makes it write-only and prevents all users from viewing its value in the HCP Terraform UI or reading it through the Variables API endpoint. Users with permission to read and write variables can set new values for sensitive variables, but you must delete and recreate the variable to edit its other attributes.

Navigate to your learn-terraform-sensitive-variables workspace's Variables page in HCP Terraform.

Create the below input variables. Mark them as 'sensitive' by clicking the Sensitive checkbox.

key	value
`db_username`	admin
`db_password`	adifferentpassword

Save the Terraform variables.

Now, run terraform apply, and Terraform will assign these values to your new variables.

$ terraform apply Terraform used the selected providers to generate the following executionplan. Resource actions are indicated with the following symbols:  ~ update in-place Terraform will perform the following actions:   # aws_db_instance.database will be updated in-place  ~ resource "aws_db_instance" "database" {        id                                    = "terraform-20220725200214442200000004"        name                                  = ""      ~ password                              = (sensitive value)        tags                                  = {}        # (57 unchanged attributes hidden)    } Plan: 0 to add, 1 to change, 0 to destroy.  Do you want to perform these actions in workspace "learn-terraform-sensitive-variables"?  Terraform will perform the actions described above.  Only 'yes' will be accepted to approve.   Enter a value: yes ##... Apply complete! Resources: 0 added, 1 changed, 0 destroyed.

Respond to the confirmation prompt with yes.

Note

When using environment variables to set sensitive values, keep in mind that those values will be in your environment and command-line history.

Reference sensitive variables

When you use sensitive variables in your Terraform configuration, you can use them as you would any other variable. Terraform will redact these values in command output and log files, and raise an error when it detects that they will be exposed in other ways.

Add the following output values to outputs.tf.

output "db_connect_string" {  description = "MySQL database connection string"  value       = "Server=${aws_db_instance.database.address}; Database=ExampleDB; Uid=${var.db_username}; Pwd=${var.db_password}"}

Now apply this change. Terraform will raise an error, since the output is derived from sensitive variables.

$ terraform applyrandom_string.lb_id: Refreshing state... [id=2Mw]module.vpc.aws_vpc.this[0]: Refreshing state... [id=vpc-05f973211a47fb6f4]module.vpc.aws_internet_gateway.this[0]: Refreshing state... [id=igw-0678406e38f54eb60] ## ... Error: Output refers to sensitive values   on outputs.tf line 3:   3: output "db_connect_string" { Expressions used in outputs can only refer to sensitive values if thesensitive attribute is true.

Flag the database connection string output as sensitive, causing Terraform to hide it.

output "db_connect_string" {  description = "MySQL database connection string"  value       = "Server=${aws_db_instance.database.address}; Database=ExampleDB; Uid=${var.db_username}; Pwd=${var.db_password}"+  sensitive   = true}

Apply this change to see that Terraform will now redact the database connection string output. Respond to the confirmation prompt with yes to apply these changes.

$ terraform applyrandom_string.lb_id: Refreshing state... [id=2Mw]module.vpc.aws_vpc.this[0]: Refreshing state... [id=vpc-05f973211a47fb6f4] ## ... Plan: 0 to add, 0 to change, 0 to destroy. Changes to Outputs:  + db_connect_string = (sensitive value) Do you want to perform these actions?  Terraform will perform the actions described above.  Only 'yes' will be accepted to approve.   Enter a value: yes Apply complete! Resources: 0 added, 0 changed, 0 destroyed. Outputs: db_connect_string = <sensitive>

Sensitive values in state

When you run Terraform commands with a local state file, Terraform stores the state as plain text, including variable values, even if you have flagged them as sensitive. Terraform needs to store these values in your state so that it can tell if you have changed them since the last time you applied your configuration.

$ grep "password" terraform.tfstate      "value": "Server=terraform-20210113192204255400000004.ct4cer62f3td.us-east-1.rds.amazonaws.com; Database=ExampleDB; Uid=admin; Pwd=adifferentpassword",            "password": "adifferentpassword",## ...

Tip

If you are using an operating system without the grep command, open the terraform.tfstate file in your text editor and search for password to see the relevant lines.

Marking variables as sensitive is not sufficient to secure them. You must also keep them secure while passing them into Terraform configuration, and protect them in your state file. HCP Terraform and Terraform Enterprise manage and share sensitive values, and encrypt all variable values before storing them. HashiCorp Vault secures, stores, and tightly controls access to tokens, passwords, and other sensitive values.

Clean up your infrastructure

Before moving on, destroy the infrastructure you created in this tutorial.

$ terraform destroy

Be sure to respond to the confirmation prompt with yes.

If you used HCP Terraform for this tutorial, after destroying your resources, delete the learn-terraform-sensitive-variables workspace from your HCP Terraform organization.

Next steps

In this tutorial you used Terraform variables to set sensitive information about your infrastructure. In addition to setting the sensitive flag for sensitive inputs, you must also ensure that sensitive values are not accidentally exposed when you pass them into your Terraform configuration. Also, you must keep your Terraform state file secure to avoid accidentally exposing sensitive data.

Now that you know how to use sensitive input variables, check out the following resources for more information.

Inject secrets into Terraform using the Vault provider by following another tutorial.
Automatically encrypt state at rest by storing it in HCP Terraform. Get Started with Terraform Cloud.
Read the documentation about how to manage sensitive data in state.

Collection Overview

0.14 Release

Provider versions

This tutorial also appears in:

37 tutorials

Terraform Associate (003) Tutorials
Progress through these tutorials to prepare for the Terraform Associate (003) certification exam.
- Terraform
19 tutorials

Write Terraform Configuration
Learn Terraform configuration language by example. Write configurations to manage multiple pieces of infrastructure and iterate over structured data. Deploy and manage related infrastructure by referring to resources in other configurations.
- Terraform

Prerequisites

Create infrastructure

Refactor database credentials

Set values with a .tfvars file

Set values with variables

Reference sensitive variables

Sensitive values in state

Clean up your infrastructure

Next steps

This tutorial also appears in:

Set values with a `.tfvars` file