Testing and CI for Terraform
Catch Terraform mistakes before they reach production: fmt and validate, the native test framework with mocks, saved plans, policy checks on plan JSON, and a GitHub Actions pipeline.
By The Dark DebuggerUpdated 27 min read
This tutorial builds the safety net around a Terraform repository: formatting and
validation checks, unit and integration tests with Terraform's built-in test
framework (including mocked providers), saved plans that guarantee you apply what was
reviewed, policy checks on the plan's JSON, and a GitHub Actions pipeline that plans
on every pull request and applies on merge, plus a scheduled drift check. It builds on
the web-service module and environment layout from
Managing Multiple Environments and
assumes you're comfortable with modules.
Why bother testing infrastructure code
Here's a bug that a plan review won't catch. The web-service module merges a
standard set of tags with extra_tags from the caller. Somebody passes
extra_tags = { Environment = "dev" } in production by copy-paste. The plan is
perfectly valid, the reviewer skims past a tag, and for the next quarter the cost
report files production's spend under dev. Nothing failed; the code did exactly what
it said.
Tests encode what the code should do, so a change like that fails loudly. Around them, a pipeline adds the other checks humans are bad at: consistent formatting, syntax, policy ("no destroys without sign-off"), and the guarantee that the plan you approved is the plan that runs. Each layer is cheap. Together they turn "I think this is safe" into evidence.
The project
The repository from the environments tutorial, reorganized the way a CI pipeline expects, with tests next to each module:
.
├── envs
│ └── prod
│ ├── backend.tfbackend
│ ├── main.tf
│ └── terraform.tfvars
├── modules
│ ├── log-bucket
│ │ ├── main.tf
│ │ ├── tests
│ │ │ └── log_bucket.tftest.hcl
│ │ └── versions.tf
│ └── web-service
│ ├── main.tf
│ ├── outputs.tf
│ ├── tests
│ │ ├── render.tftest.hcl
│ │ └── unit.tftest.hcl
│ ├── variables.tf
│ └── versions.tf
└── scripts
├── ci-local.sh
└── policy-check.shweb-service renders a deployment file with local providers, so it can be tested and
applied anywhere. log-bucket is a small AWS module, which we'll test without an AWS
account using a mocked provider.
Layer 1: terraform fmt
terraform fmt rewrites files into the canonical style. In CI you don't want it to
rewrite anything; you want it to fail if anything would be rewritten. That's
-check. With -recursive it covers every subdirectory, and -diff shows what's
wrong. Here's envs/prod/main.tf after a hasty edit:
terraform fmt -check -recursive
echo "exit $?"envs/prod/main.tf
exit 3terraform fmt -check -recursive -diffenvs/prod/main.tf
--- old/envs/prod/main.tf
+++ new/envs/prod/main.tf
@@ -32,8 +32,8 @@
project = "acme"
environment = var.environment
service = "checkout"
- replicas = var.replicas
- log_level = var.log_level
+ replicas = var.replicas
+ log_level = var.log_level
extra_tags = var.extra_tags
output_dir = "${path.root}/rendered"
}A non-zero exit code (3 here) is what fails the CI step. Locally, terraform fmt -recursive fixes everything in place. Formatting sounds trivial, but it's what keeps
diffs in code review down to real changes instead of realigned equals signs.
Layer 2: terraform validate
validate checks that the configuration is internally consistent: valid syntax,
known arguments, correct types, references that exist. It needs terraform init
first (for provider schemas), but no credentials and no state. A typo in a module
argument:
terraform validateError: Unsupported argument
on main.tf line 35, in module "checkout":
35: replica = var.replicas
An argument named "replica" is not expected here. Did you mean "replicas"?After fixing it:
Success! The configuration is valid.validate can't tell you whether values make sense; replicas = -3 is a valid
number. That's what variable validation rules and tests are for. For modules, which
have no backend of their own, initialize with terraform init -backend=false.
Layer 3: the native test framework
Since Terraform 1.6, terraform test runs .tftest.hcl files from a tests/
directory. Each file contains run blocks. A run block executes a plan or an apply
of the module under test, then checks assert conditions against the result.
| Element | Where | What it does |
|---|---|---|
variables { } | File level | Default input values for every run in the file |
run "name" { } | File level | One test step, executed in order |
command = plan | Run block | Plan only: fast, creates nothing |
command = apply | Run block | The default. Really applies; everything is destroyed when the file finishes |
variables { } | Run block | Overrides for this run only |
assert { condition, error_message } | Run block | Fails the run if the condition is false |
expect_failures = [var.x] | Run block | Passes only if that variable's validation (or that check) fails |
mock_provider "aws" { } | File level | Replaces a provider with fake responses |
override_resource { target, values } | File or run | Fixes the values of one resource under a mock |
Unit tests with command = plan
modules/web-service/tests/unit.tftest.hcl tests the naming convention, the tag
rules and the environment validation, all at plan time:
# Plan-only tests: fast, no files written.
variables {
project = "acme"
environment = "prod"
service = "checkout"
replicas = 4
output_dir = "./test-output"
}
run "name_follows_convention" {
command = plan
assert {
condition = output.name == "acme-prod-checkout"
error_message = "Name should be project-environment-service, got ${output.name}."
}
}
run "standard_tags_present" {
command = plan
assert {
condition = alltrue([for k in ["Project", "Environment", "Service", "ManagedBy"] : contains(keys(output.tags), k)])
error_message = "Missing a standard tag: ${jsonencode(output.tags)}"
}
}
run "extra_tags_cannot_override_standard_tags" {
command = plan
variables {
extra_tags = {
Environment = "dev"
CostCenter = "cc-1042"
}
}
assert {
condition = output.tags["Environment"] == "prod"
error_message = "extra_tags overrode the Environment tag: got ${output.tags["Environment"]}."
}
assert {
condition = output.tags["CostCenter"] == "cc-1042"
error_message = "extra_tags should still add new tags."
}
}
run "rejects_unknown_environment" {
command = plan
variables {
environment = "qa"
}
expect_failures = [var.environment]
}rejects_unknown_environment is a negative test: it passes only because the
module's validation rule rejects qa. If someone loosens that rule, the test fails.
An integration test with apply
render.tftest.hcl has no command, so it applies: the file really gets written,
the assertions read it back from disk, and Terraform destroys it when the file's runs
are done.
# Apply test: really writes the file, checks it, and destroys it afterwards.
variables {
project = "acme"
environment = "staging"
service = "checkout"
replicas = 2
output_dir = "./test-output"
}
run "renders_deployment_file" {
assert {
condition = fileexists(output.deployment_file)
error_message = "Deployment file was not written to ${output.deployment_file}."
}
assert {
condition = jsondecode(file(output.deployment_file)).replicas == 2
error_message = "Rendered replicas do not match the input."
}
assert {
condition = jsondecode(file(output.deployment_file)).health_check.path == "/healthz"
error_message = "Health check path should default to /healthz."
}
}With a cloud provider, an apply test creates real resources in a real account, which costs money and time but proves things a plan can't: that the API accepts your configuration, and that computed attributes look the way you expect.
Running the tests, and the bug they found
terraform init -backend=false
terraform testtests/render.tftest.hcl... in progress
run "renders_deployment_file"... pass
tests/render.tftest.hcl... tearing down
tests/render.tftest.hcl... pass
tests/unit.tftest.hcl... in progress
run "name_follows_convention"... pass
run "standard_tags_present"... pass
run "extra_tags_cannot_override_standard_tags"... fail
Error: Test assertion failed
on tests/unit.tftest.hcl line 39, in run "extra_tags_cannot_override_standard_tags":
39: condition = output.tags["Environment"] == "prod"
├────────────────
│ Diff:
│ --- actual
│ +++ expected
│ - "dev"
│ + "prod"
extra_tags overrode the Environment tag: got dev.
run "rejects_unknown_environment"... pass
tests/unit.tftest.hcl... tearing down
tests/unit.tftest.hcl... fail
Failure! 4 passed, 1 failed.That's the cost-report bug from the start of the tutorial, and it was in the module
as I wrote it for the environments tutorial. merge() lets later arguments win, and
var.extra_tags came last. The fix is to swap the order, so callers can add tags but
never replace the standard ones:
locals {
# acme-prod-checkout: project, environment, service, always in that order.
name = "${var.project}-${var.environment}-${var.service}"
# Standard tags go last so extra_tags can add keys but never override them.
tags = merge(
var.extra_tags,
{
Project = var.project
Environment = var.environment
Service = var.service
ManagedBy = "terraform"
},
)
}terraform testtests/render.tftest.hcl... in progress
run "renders_deployment_file"... pass
tests/render.tftest.hcl... tearing down
tests/render.tftest.hcl... pass
tests/unit.tftest.hcl... in progress
run "name_follows_convention"... pass
run "standard_tags_present"... pass
run "extra_tags_cannot_override_standard_tags"... pass
run "rejects_unknown_environment"... pass
tests/unit.tftest.hcl... tearing down
tests/unit.tftest.hcl... pass
Success! 5 passed, 0 failed.The apply test leaves an empty test-output/ directory behind (the local provider
deletes files, not directories), so add it to .gitignore.
Two flags you'll use in CI: -filter runs a single file, and -junit-xml writes a
report that CI systems can display:
terraform test -filter=tests/unit.tftest.hcl -junit-xml=test-report.xml
head -12 test-report.xml[…]
Success! 4 passed, 0 failed.
<?xml version="1.0" encoding="UTF-8"?><testsuites>
<testsuite name="tests/unit.tftest.hcl" tests="4" skipped="0" failures="0" errors="0">
<testcase name="name_follows_convention" classname="tests/unit.tftest.hcl" time="0.116348735" timestamp="2026-11-10T06:20:55Z"></testcase>
<testcase name="standard_tags_present" classname="tests/unit.tftest.hcl" time="0.073060522" timestamp="2026-11-10T06:20:55Z"></testcase>
<testcase name="extra_tags_cannot_override_standard_tags" classname="tests/unit.tftest.hcl" time="0.078077886" timestamp="2026-11-10T06:20:55Z"></testcase>
<testcase name="rejects_unknown_environment" classname="tests/unit.tftest.hcl" time="0.061592846" timestamp="2026-11-10T06:20:55Z"></testcase>
</testsuite>
</testsuites>-verbose prints the plan or state for each run, which is the first thing to reach
for when an assertion fails and you don't understand why.
Mocking providers: mock_provider and override_resource
Testing an AWS module with apply means an AWS account, credentials in CI, and money.
mock_provider replaces the provider with one that answers every call with made-up
values and never touches the network. Here's modules/log-bucket/main.tf:
variable "name" {
description = "Bucket name."
type = string
}
variable "retention_days" {
description = "Days to keep log objects before they expire."
type = number
default = 90
validation {
condition = var.retention_days >= 1 && var.retention_days <= 3650
error_message = "retention_days must be between 1 and 3650."
}
}
resource "aws_s3_bucket" "this" {
bucket = var.name
}
resource "aws_s3_bucket_public_access_block" "this" {
bucket = aws_s3_bucket.this.id
block_public_acls = true
block_public_policy = true
ignore_public_acls = true
restrict_public_buckets = true
}
resource "aws_s3_bucket_lifecycle_configuration" "this" {
bucket = aws_s3_bucket.this.id
rule {
id = "expire-logs"
status = "Enabled"
filter {}
expiration {
days = var.retention_days
}
}
}
output "id" {
description = "Bucket name."
value = aws_s3_bucket.this.id
}
output "arn" {
description = "Bucket ARN."
value = aws_s3_bucket.this.arn
}Its versions.tf requires hashicorp/aws >= 6.0. And tests/log_bucket.tftest.hcl:
mock_provider "aws" {}
variables {
name = "acme-prod-alb-logs"
}
run "blocks_all_public_access" {
command = plan
assert {
condition = alltrue([
aws_s3_bucket_public_access_block.this.block_public_acls,
aws_s3_bucket_public_access_block.this.block_public_policy,
aws_s3_bucket_public_access_block.this.ignore_public_acls,
aws_s3_bucket_public_access_block.this.restrict_public_buckets,
])
error_message = "All four public access blocks must be enabled."
}
}
run "expires_logs_after_retention" {
command = plan
variables {
retention_days = 30
}
assert {
condition = aws_s3_bucket_lifecycle_configuration.this.rule[0].expiration[0].days == 30
error_message = "Lifecycle expiration should follow retention_days."
}
}
run "exposes_bucket_arn" {
override_resource {
target = aws_s3_bucket.this
values = {
arn = "arn:aws:s3:::acme-prod-alb-logs"
}
}
assert {
condition = output.arn == "arn:aws:s3:::acme-prod-alb-logs"
error_message = "The arn output should come from the bucket."
}
}terraform init -backend=false
terraform testtests/log_bucket.tftest.hcl... in progress
run "blocks_all_public_access"... pass
run "expires_logs_after_retention"... pass
run "exposes_bucket_arn"... pass
tests/log_bucket.tftest.hcl... tearing down
tests/log_bucket.tftest.hcl... pass
Success! 3 passed, 0 failed.No credentials, no network calls to AWS, a couple of seconds. The override_resource
block matters more than it looks. Without it, the mock invents the ARN, and the same
test fails:
run "exposes_bucket_arn"... fail
Error: Test assertion failed
on tests/log_bucket.tftest.hcl line 36, in run "exposes_bucket_arn":
36: condition = output.arn == "arn:aws:s3:::acme-prod-alb-logs"
├────────────────
│ Diff:
│ --- actual
│ +++ expected
│ - "01hk0mnn"
│ + "arn:aws:s3:::acme-prod-alb-logs"
The arn output should come from the bucket.That's the rule for mocks: values you set in configuration come through unchanged,
and every computed value is a random string unless you override it. Related tools:
override_data does the same for data sources, override_module replaces a whole
child module's outputs, and a mock_data/mock_resource block inside
mock_provider sets defaults for every instance of a type (the
modules tutorial
needed one for policy JSON).
What mocks can't tell you: whether AWS accepts the configuration. A mocked test passes for a bucket name with uppercase letters that the real S3 API would reject. I treat mocked tests as fast unit tests of the module's logic, and run apply tests against a real sandbox account less often (nightly, or before a release).
Saved plans: apply exactly what was reviewed
terraform apply without a plan file re-plans first. If anything changed between the
plan someone reviewed and the apply (a merge, another apply, drift), you apply
something nobody saw. A saved plan closes that gap. In envs/prod, bump the replica
count from 4 to 6 in terraform.tfvars and save the plan:
terraform plan -out=tfplan[…]
Terraform will perform the following actions:
# module.checkout.local_file.deployment must be replaced
-/+ resource "local_file" "deployment" {
~ content = jsonencode(
~ {
name = "acme-prod-checkout"
~ replicas = 4 -> 6
tags = {
CostCenter = "cc-1042"
Environment = "prod"
ManagedBy = "terraform"
OnCall = "payments-oncall"
Project = "acme"
Service = "checkout"
}
# (2 unchanged attributes hidden)
} # forces replacement
)
[…]
}
Plan: 1 to add, 0 to change, 1 to destroy.
─────────────────────────────────────────────────────────────────────────────
Saved the plan to: tfplan
To perform exactly these actions, run the following command to apply:
terraform apply "tfplan"terraform show tfplan prints the same plan for humans, and -json gives a machine-
readable version, which is what policy checks and PR comments are built from:
terraform show -json tfplan | jq '[.resource_changes[] | {address, actions: .change.actions}]'[
{
"address": "module.checkout.local_file.deployment",
"actions": [
"delete",
"create"
]
}
]A replacement shows up as ["delete", "create"] (or ["create", "delete"] with
create_before_destroy). What's inside the plan file is worth knowing, because it's
a zip archive:
unzip -l tfplanArchive: tfplan
Length Date Time Name
--------- ---------- ----- ----
2380 2026-11-10 01:21 tfplan
2549 2026-11-10 01:21 tfstate
2768 2026-11-10 01:21 tfstate-prev
802 2026-11-10 01:21 tfconfig/m-/main.tf
855 2026-11-10 01:21 tfconfig/m-checkout/main.tf
357 2026-11-10 01:21 tfconfig/m-checkout/outputs.tf
1077 2026-11-10 01:21 tfconfig/m-checkout/variables.tf
135 2026-11-10 01:21 tfconfig/m-checkout/versions.tf
155 2026-11-10 01:21 tfconfig/modules.json
1267 2026-11-10 01:21 .terraform.lock.hcl
--------- -------
12345 10 filesIt carries a copy of the state and the configuration. Treat a plan file as a secret, exactly like state: short retention, restricted access.
Applying a plan file doesn't ask for confirmation, because the review already happened:
terraform apply tfplanmodule.checkout.local_file.deployment: Destroying... [id=b959e95aabfec7ce506b005fac47c9db2251f9d7]
module.checkout.local_file.deployment: Destruction complete after 0s
module.checkout.local_file.deployment: Creating...
module.checkout.local_file.deployment: Creation complete after 0s [id=b36f1160b45b50264e42d349fcdf7ec37b03e265]
Apply complete! Resources: 1 added, 0 changed, 1 destroyed.
[…]And the safety property: once state has moved on, an old plan refuses to run. Applying the same file a second time:
terraform apply tfplanError: Saved plan is stale
The given plan file can no longer be applied because the state was changed by
another operation after the plan was created.That's what makes "plan on merge, approve, apply the saved plan" safe: if anything happened in between, the apply fails instead of doing something unreviewed.
Policy checks on the plan JSON
Some rules shouldn't depend on a reviewer noticing. A few lines of jq over the plan
JSON enforce them in every pipeline run. scripts/policy-check.sh:
#!/usr/bin/env bash
# policy-check.sh PLAN_JSON
# Fails (exit 1) when a plan breaks a rule we never want merged unreviewed.
# Produce the input with: terraform show -json tfplan > plan.json
set -euo pipefail
plan_json=$1
failed=0
# Rule 1: nothing is deleted or replaced unless ALLOW_DESTROY=1 is set.
deletes=$(jq -r '.resource_changes[]
| select(.change.actions | index("delete"))
| "\(.address) (\(.change.actions | join("+")))"' "$plan_json")
if [[ -n "$deletes" && "${ALLOW_DESTROY:-0}" != "1" ]]; then
echo "POLICY FAIL: the plan deletes or replaces resources:"
sed 's/^/ - /' <<< "$deletes"
failed=1
fi
# Rule 2: every AWS resource being created or updated with tags carries
# the tags finance and on-call depend on.
untagged=$(jq -r '.resource_changes[]
| select(.change.actions | index("create") or index("update"))
| select(.change.after.tags_all? != null)
| select((.change.after.tags_all | has("Environment") and has("CostCenter")) | not)
| .address' "$plan_json")
if [[ -n "$untagged" ]]; then
echo "POLICY FAIL: missing Environment or CostCenter tag:"
sed 's/^/ - /' <<< "$untagged"
failed=1
fi
if [[ $failed -eq 0 ]]; then
echo "policy-check: $(jq '.resource_changes | length' "$plan_json") resource change(s) checked, all rules passed"
fi
exit $failedAgainst the replica-bump plan from the previous section:
terraform show -json tfplan > plan.json
../../scripts/policy-check.sh plan.jsonPOLICY FAIL: the plan deletes or replaces resources:
- module.checkout.local_file.deployment (delete+create)Replacing a rendered file is fine, so a human makes the call and sets the override:
ALLOW_DESTROY=1 ../../scripts/policy-check.sh plan.jsonpolicy-check: 1 resource change(s) checked, all rules passedRule 2 needs AWS resources. I ran it against a plan of the default_tags bucket from
the environments tutorial (planned with dummy credentials), whose tags include
Environment but no CostCenter:
POLICY FAIL: missing Environment or CostCenter tag:
- aws_s3_bucket.assetsChecking tags_all rather than tags matters: it includes the provider's
default_tags, which is what AWS actually receives.
A script like this covers a surprising amount. When the rule list grows, dedicated policy engines read the same plan JSON: Open Policy Agent (with Conftest) is the common open-source choice, and HCP Terraform has built-in policy sets. I haven't used either for this tutorial.
A complete GitHub Actions pipeline
Here's the workflow I'd start a team on, saved as .github/workflows/terraform.yml.
Pull requests run checks, tests and a prod plan with the policy check. A merge to
main plans again, saves the plan, waits for approval through a GitHub environment,
and applies exactly that plan.
name: terraform
on:
pull_request:
paths: ["infra/**"]
push:
branches: [main]
paths: ["infra/**"]
permissions:
contents: read
id-token: write # lets the job assume an AWS role through OIDC
concurrency:
group: terraform-prod
cancel-in-progress: false # never cancel a running apply
env:
TF_IN_AUTOMATION: "true"
TF_INPUT: "false"
jobs:
checks:
runs-on: ubuntu-24.04
defaults:
run:
working-directory: infra
steps:
- uses: actions/checkout@v4
- uses: hashicorp/setup-terraform@v3
with:
terraform_version: 1.17.1
terraform_wrapper: false
- name: Format
run: terraform fmt -check -recursive -diff .
- name: Module tests
run: |
for dir in modules/*/; do
echo "::group::$dir"
terraform -chdir="$dir" init -backend=false
terraform -chdir="$dir" test -junit-xml=test-report.xml
echo "::endgroup::"
done
plan:
needs: checks
runs-on: ubuntu-24.04
defaults:
run:
working-directory: infra/envs/prod
steps:
- uses: actions/checkout@v4
- uses: hashicorp/setup-terraform@v3
with:
terraform_version: 1.17.1
terraform_wrapper: false
- uses: aws-actions/configure-aws-credentials@v4
with:
role-to-assume: arn:aws:iam::111122223333:role/terraform-plan
aws-region: eu-west-1
- name: Init
run: terraform init -backend-config=backend.tfbackend
- name: Validate
run: terraform validate
- name: Plan
run: terraform plan -lock-timeout=5m -out=tfplan
- name: Policy check
run: |
terraform show -json tfplan > plan.json
../../scripts/policy-check.sh plan.json
- name: Keep the reviewed plan
if: github.event_name == 'push'
uses: actions/upload-artifact@v4
with:
name: prod-tfplan
path: infra/envs/prod/tfplan
retention-days: 3
apply:
if: github.event_name == 'push' && github.ref == 'refs/heads/main'
needs: plan
runs-on: ubuntu-24.04
environment: production # required reviewers approve before this job starts
defaults:
run:
working-directory: infra/envs/prod
steps:
- uses: actions/checkout@v4
- uses: hashicorp/setup-terraform@v3
with:
terraform_version: 1.17.1
terraform_wrapper: false
- uses: aws-actions/configure-aws-credentials@v4
with:
role-to-assume: arn:aws:iam::111122223333:role/terraform-apply
aws-region: eu-west-1
- uses: actions/download-artifact@v4
with:
name: prod-tfplan
path: infra/envs/prod
- name: Init
run: terraform init -backend-config=backend.tfbackend
- name: Apply the reviewed plan
run: terraform apply -lock-timeout=5m tfplanThe decisions in it, and why:
terraform_wrapper: false. The setup action can wrap theterraformbinary to capture its output for later steps. Turning that off keepsterraform show -jsonoutput exactly as Terraform printed it, which the policy script depends on.- OIDC, not stored keys.
id-token: writeplusconfigure-aws-credentialsgives each job short-lived credentials for an IAM role. The plan role can be read-only; only the apply role can change things. - A GitHub environment for approval.
environment: productionwith required reviewers pauses the apply job until someone approves it, after reading the plan output from the plan job. - The saved plan travels between jobs as an artifact with short retention, because it contains state. If someone else applied in between, the apply fails with "Saved plan is stale", and you re-run the workflow.
concurrencyserializes runs for prod and never cancels one mid-apply. Together with-lock-timeout=5m, merges in quick succession queue up instead of failing.TF_IN_AUTOMATIONtrims the "run terraform apply next" hints from output, andTF_INPUT=falsemakes any prompt fail immediately instead of hanging the job.
Running the same checks locally
I can't run GitHub Actions in my lab, and the AWS steps need an account. What I did
run is everything the workflow does with Terraform, in the same order, as a script
developers can also run before pushing. scripts/ci-local.sh:
#!/usr/bin/env bash
# ci-local.sh: run the same checks as the pipeline, in the same order.
set -euo pipefail
cd "$(dirname "$0")/.."
echo "== terraform fmt"
terraform fmt -check -recursive -diff .
echo "== module tests"
for dir in modules/*/; do
echo "-- $dir"
terraform -chdir="$dir" init -backend=false -input=false > /dev/null
terraform -chdir="$dir" test
done
echo "== prod: validate, plan, policy"
cd envs/prod
terraform init -input=false -backend-config=backend.tfbackend > /dev/null
terraform validate
terraform plan -input=false -lock-timeout=5m -out=tfplan > /dev/null
terraform show -json tfplan > plan.json
../../scripts/policy-check.sh plan.json./scripts/ci-local.sh== terraform fmt
== module tests
-- modules/log-bucket/
tests/log_bucket.tftest.hcl... in progress
run "blocks_all_public_access"... pass
run "expires_logs_after_retention"... pass
run "exposes_bucket_arn"... pass
tests/log_bucket.tftest.hcl... tearing down
tests/log_bucket.tftest.hcl... pass
Success! 3 passed, 0 failed.
-- modules/web-service/
tests/render.tftest.hcl... in progress
run "renders_deployment_file"... pass
tests/render.tftest.hcl... tearing down
tests/render.tftest.hcl... pass
tests/unit.tftest.hcl... in progress
run "name_follows_convention"... pass
run "standard_tags_present"... pass
run "extra_tags_cannot_override_standard_tags"... pass
run "rejects_unknown_environment"... pass
tests/unit.tftest.hcl... tearing down
tests/unit.tftest.hcl... pass
Success! 5 passed, 0 failed.
== prod: validate, plan, policy
Success! The configuration is valid.
policy-check: 1 resource change(s) checked, all rules passedWith nothing to change, the plan still lists the deployment as a no-op entry in
resource_changes, which is why the policy script counts one change.
A scheduled drift-detection job
The state tutorial
showed terraform plan -refresh-only -detailed-exitcode. In CI it becomes a nightly
job per environment that opens an issue when reality and state disagree.
.github/workflows/drift.yml:
name: terraform-drift
on:
schedule:
- cron: "17 6 * * 1-5" # 06:17 UTC on weekdays
workflow_dispatch: {}
permissions:
contents: read
id-token: write
issues: write
jobs:
drift:
runs-on: ubuntu-24.04
strategy:
fail-fast: false
matrix:
env: [dev, staging, prod]
defaults:
run:
working-directory: infra/envs/${{ matrix.env }}
steps:
- uses: actions/checkout@v4
- uses: hashicorp/setup-terraform@v3
with:
terraform_version: 1.17.1
terraform_wrapper: false
- uses: aws-actions/configure-aws-credentials@v4
with:
role-to-assume: arn:aws:iam::111122223333:role/terraform-plan
aws-region: eu-west-1
- name: Init
run: terraform init -input=false -backend-config=backend.tfbackend
- name: Check for drift
id: drift
run: |
set +e
terraform plan -refresh-only -detailed-exitcode -lock-timeout=5m -no-color > drift.txt
code=$?
set -e
echo "exitcode=$code" >> "$GITHUB_OUTPUT"
if [ "$code" -eq 1 ]; then cat drift.txt; exit 1; fi
- name: Open an issue
if: steps.drift.outputs.exitcode == '2'
env:
GH_TOKEN: ${{ github.token }}
run: |
gh issue create \
--title "Drift detected in ${{ matrix.env }}" \
--body-file drift.txt \
--label driftThe set +e matters: GitHub's default bash shell stops at the first non-zero exit,
and exit code 2 (drift found) isn't an error here. Exit code 1 is, so that still
fails the job. I ran the core of the "Check for drift" step locally in envs/prod
after editing the rendered deployment file by hand:
set +e
terraform plan -refresh-only -detailed-exitcode -lock-timeout=5m -no-color > drift.txt
code=$?
set -e
echo "drift exit code: $code"
grep -E '^ # ' drift.txtdrift exit code: 2
# module.checkout.local_file.deployment has been deletedAfter a normal terraform apply put the file back, the same command returned 0.
Where you'll use this
- Every shared module gets tests in its
tests/directory: plan-mode unit tests for naming, tags and validation rules, mocked tests for cloud resources, and an apply test against a sandbox account before a release tag. - Every root module (environment) gets the plan pipeline: fmt, validate, plan
with
-out, policy check, and apply of the saved plan after approval. The plan output goes into the pull request so reviewers read the diff, not just the code. - Every environment gets a drift job. Drift is how you find the console hotfix from last Thursday before it's overwritten by the next apply.
- Developers run
ci-local.shbefore pushing, so the pipeline mostly confirms what they already know.
The layering is the point. fmt and validate cost nothing, tests catch logic bugs
in modules, the policy check catches dangerous plans, and the saved-plan flow makes
sure the reviewed plan is the applied plan.
Common mistakes
Asserting on a value that's only known after apply
run "file_has_an_id" {
command = plan
assert {
condition = local_file.deployment.id != ""
error_message = "The deployment file should have an ID."
}
} run "file_has_an_id"... fail
Error: Unknown condition value
on tests/unknown.tftest.hcl line 13, in run "file_has_an_id":
13: condition = local_file.deployment.id != ""
├────────────────
│ local_file.deployment.id is a string
Condition expression could not be evaluated at this time. This means you have
executed a `run` block with `command = plan` and one of the values your
condition depended on is not known until after the plan has been applied.
Either remove this value from your condition, or execute an `apply` command
from this `run` block. Alternatively, if there is an override for this value,
you can make it available during the plan phase by setting `override_during =
plan` in the `override_` block.IDs, ARNs and hashes are computed by the provider, so they're unknown in a plan.
Either switch the run to apply, or assert on inputs and derived values that exist
at plan time. The message also points at override_during = plan for mocked values.
Applying a saved plan after something else changed the state
That's the "Saved plan is stale" error from the saved-plans section. It's not a bug
to work around; it means the plan you reviewed no longer describes what would
happen. Re-run the plan, review it again, and apply the new file. Don't "fix" it by
switching the pipeline to a plain terraform apply -auto-approve.
Trusting mocked values
A mocked provider returned "01hk0mnn" as a bucket ARN. Any assertion about the
format of a computed value passes or fails at random unless you override it. Assert
on what your module controls, and use override_resource when a test depends on a
computed value.
Summary
terraform fmt -check -recursivefails (exit 3) when files aren't canonically formatted;-diffshows why.terraform validatecatches structural errors and needsinit(use-backend=falsefor modules).terraform testrunsrunblocks fromtests/*.tftest.hcl.command = planis fast and creates nothing; the defaultapplycreates real resources and destroys them afterwards.variablesblocks set inputs per file or per run,assertchecks conditions, andexpect_failuresturns validation rules into negative tests.mock_providerruns cloud modules offline; computed values are random unless set withoverride_resource(oroverride_data,override_module).terraform plan -out=tfplanplusterraform apply tfplanapplies exactly what was reviewed. A plan file contains state, so treat it as a secret. A stale plan refuses to apply.terraform show -json tfplanfeeds policy checks: replacements appear asdeletepluscreateinchange.actions, and tags astags_all.- In CI: plan on pull requests, plan and save on merge, approve, apply the saved plan.
Schedule
plan -refresh-only -detailed-exitcodeto catch drift.
Exercises
All exercises use the infra repository from this tutorial.
Set prod's replicas to 3, save a plan, and write a jq command that prints a
one-line summary of plan.json grouped by action, such as 1 delete+create.
Show Solution
terraform plan -out=tfplan > /dev/null
terraform show -json tfplan > plan.json
jq -r '[.resource_changes[].change.actions | join("+")] | group_by(.) | map("\(length) \(.[0])") | join(", ")' plan.json1 delete+createJoining the actions array turns a replacement into one readable token
(delete+create) instead of counting it as both a delete and a create, which is how
reviewers think about it. On a bigger plan the output looks like
3 create, 1 delete+create, 12 no-op.
Add validation to the web-service module's replicas variable: a whole number from
1 to 20. Add two negative tests to unit.tftest.hcl, one for 0 and one for 2.5,
and run only that file.
Hint
floor(x) == x is true only for whole numbers.
Show Solution
variable "replicas" {
description = "Number of instances to run."
type = number
validation {
condition = var.replicas >= 1 && var.replicas <= 20 && floor(var.replicas) == var.replicas
error_message = "replicas must be a whole number from 1 to 20."
}
}run "rejects_zero_replicas" {
command = plan
variables {
replicas = 0
}
expect_failures = [var.replicas]
}
run "rejects_fractional_replicas" {
command = plan
variables {
replicas = 2.5
}
expect_failures = [var.replicas]
}terraform test -filter=tests/unit.tftest.hcltests/unit.tftest.hcl... in progress
run "name_follows_convention"... pass
run "standard_tags_present"... pass
run "extra_tags_cannot_override_standard_tags"... pass
run "rejects_unknown_environment"... pass
run "rejects_zero_replicas"... pass
run "rejects_fractional_replicas"... pass
tests/unit.tftest.hcl... tearing down
tests/unit.tftest.hcl... pass
Success! 6 passed, 0 failed.The existing runs use replicas = 4 from the file-level variables block, so they
also prove the rule doesn't reject valid values.
Write modules/log-bucket/tests/retention.tftest.hcl with a mocked AWS provider and
two runs: one that checks the default retention is 90 days, and one that checks
retention_days = 0 is rejected.
Show Solution
mock_provider "aws" {}
variables {
name = "acme-prod-alb-logs"
}
run "defaults_to_90_days" {
command = plan
assert {
condition = aws_s3_bucket_lifecycle_configuration.this.rule[0].expiration[0].days == 90
error_message = "Default retention should be 90 days."
}
}
run "rejects_zero_days" {
command = plan
variables {
retention_days = 0
}
expect_failures = [var.retention_days]
}terraform test -filter=tests/retention.tftest.hcltests/retention.tftest.hcl... in progress
run "defaults_to_90_days"... pass
run "rejects_zero_days"... pass
tests/retention.tftest.hcl... tearing down
tests/retention.tftest.hcl... pass
Success! 2 passed, 0 failed.Testing defaults looks redundant until someone changes one "temporarily". A test turns that into a conscious, reviewed decision.
Write scripts/apply-reviewed.sh PLANFILE that runs the policy check on a saved plan,
refuses to apply if it fails, applies the plan otherwise, and prints a clear message
(instead of Terraform's raw error) when the plan is stale. Demonstrate all three
outcomes with the replica-change plan from exercise 1.
Show Solution
#!/usr/bin/env bash
# apply-reviewed.sh PLANFILE: policy-check a saved plan, then apply exactly it.
set -uo pipefail
plan_file=$1
script_dir=$(cd "$(dirname "$0")" && pwd)
terraform show -json "$plan_file" > "$plan_file.json" || exit 1
if ! "$script_dir/policy-check.sh" "$plan_file.json"; then
echo "apply-reviewed: refusing to apply $plan_file"
exit 1
fi
if ! terraform apply -input=false -no-color "$plan_file" 2> apply-errors.txt; then
if grep -q "Saved plan is stale" apply-errors.txt; then
echo "apply-reviewed: $plan_file is stale (state changed since it was planned). Re-run plan and review again."
else
cat apply-errors.txt
fi
rm -f apply-errors.txt
exit 1
fi
rm -f apply-errors.txt../../scripts/apply-reviewed.sh tfplan; echo "exit $?"
ALLOW_DESTROY=1 ../../scripts/apply-reviewed.sh tfplan | tail -3; echo "exit ${PIPESTATUS[0]}"
ALLOW_DESTROY=1 ../../scripts/apply-reviewed.sh tfplan; echo "exit $?"POLICY FAIL: the plan deletes or replaces resources:
- module.checkout.local_file.deployment (delete+create)
apply-reviewed: refusing to apply tfplan
exit 1
"Project" = "acme"
"Service" = "checkout"
}
exit 0
policy-check: 1 resource change(s) checked, all rules passed
apply-reviewed: tfplan is stale (state changed since it was planned). Re-run plan and review again.
exit 1The script deliberately avoids set -e, because it needs to inspect failures rather
than die on them. Capturing stderr to a file lets it recognize the stale-plan case
while still showing any other error in full. PIPESTATUS[0] reports the script's
exit code rather than tail's.
Tested with: Terraform v1.17.1 on Ubuntu 24.04 (WSL2, linux_amd64), with
hashicorp/local v2.9.1 and hashicorp/aws v6.72.0, jq 1.7, November 2026. Every
terraform fmt, validate, test, plan, show and apply command shown was run
as shown, with -no-color added; the AWS module was tested only against
mock_provider, and the tag policy ran on a plan made with dummy credentials. The two
GitHub Actions workflows were not executed (that needs GitHub and an AWS account);
I checked that both files parse as YAML and ran their Terraform steps locally through
ci-local.sh and the drift commands shown.
Written by The Dark Debugger
Engineer documenting the things that took me too long to learn, so they take you less time. Every Terraform tutorial here is tested end to end before publishing.