Terraform State: Backends, Locking, and Refactoring
What Terraform state holds, where to store it, how locking protects it, and how to rename, import, hand over and drift-check resources without destroying anything.
By The Dark DebuggerUpdated 36 min read
State is the file that connects your configuration to the real objects Terraform
created. This tutorial opens it up, moves it to a shared backend, makes two applies
fight over the lock, and then works through every tool for changing state safely:
moved, removed and import blocks, the terraform state subcommands, drift
detection and recovery from mistakes. You should know the init/plan/apply loop
from the Terraform Introduction and be
comfortable with for_each from
count, for_each, and Dynamic Blocks,
because instance addresses like local_file.service_env["api"] show up everywhere
here. Next in the path is Writing and Using Modules.
Why state deserves your attention
Picture a small team that has been running Terraform from one engineer's laptop. The
terraform.tfstate file lives in that engineer's project directory and nowhere else.
A colleague clones the repository, runs terraform plan, and Terraform proposes to
create every resource again, because as far as that copy of Terraform knows, nothing
exists yet. If the colleague applies, the best case is a wall of "already exists"
errors. The worst case is a duplicate copy of the infrastructure.
Other ways state bites:
- Two people apply at the same moment and one overwrites the other's record of what was created.
- Someone renames a resource in a refactor, and the plan quietly says
1 to add, 1 to destroyfor a production database. - A database password marked
sensitive = trueturns out to be sitting in plain text in a file that was committed to git.
Every one of those has a specific fix, and all of them come down to understanding what state is. We'll build a small project for a fictional payments service and break it in each of those ways.
What's inside terraform.tfstate
The demo project needs no cloud account. It uses random_password for a database
password and local_file to render per-service configuration files. Treat the files
as stand-ins for anything with an ID and attributes: a bucket, a VM, a DNS record.
terraform {
required_version = ">= 1.10"
required_providers {
local = { source = "hashicorp/local", version = "~> 2.5" }
random = { source = "hashicorp/random", version = "~> 3.6" }
}
}
resource "random_password" "db" {
length = 24
special = false
}
resource "local_file" "api_env" {
filename = "${path.module}/rendered/api.env"
file_permission = "0640"
content = <<-EOT
SERVICE_NAME=api
LISTEN_PORT=8080
DB_HOST=payments-db.internal.example.com
EOT
}
resource "local_file" "worker_env" {
filename = "${path.module}/rendered/worker.env"
file_permission = "0640"
content = <<-EOT
SERVICE_NAME=worker
LISTEN_PORT=9090
DB_HOST=payments-db.internal.example.com
EOT
}
resource "local_file" "logrotate" {
filename = "${path.module}/rendered/payments.logrotate"
content = <<-EOT
/var/log/payments/*.log {
daily
rotate 14
compress
}
EOT
}
output "db_password" {
value = random_password.db.result
sensitive = true
}Save that as main.tf in a payments directory, then initialize and apply:
terraform init
terraform apply -auto-approve[…]
local_file.worker_env: Creating...
random_password.db: Creating...
local_file.logrotate: Creating...
local_file.logrotate: Creation complete after 0s [id=bcc5e061c5415b54de815cfca780ea50a56bd403]
local_file.api_env: Creating...
local_file.worker_env: Creation complete after 0s [id=4571e130063757eebb7da6845e0cef961d2d666e]
local_file.api_env: Creation complete after 0s [id=73306c9e6b881361742635347390d80f61dad51d]
random_password.db: Creation complete after 0s [id=none]
Apply complete! Resources: 4 added, 0 changed, 0 destroyed.
Outputs:
db_password = <sensitive>ls -ltotal 16
-rw-r--r-- 1 sid sid 1044 Nov 9 21:40 main.tf
drwxr-xr-x 2 sid sid 4096 Nov 9 21:40 rendered
-rw-r--r-- 1 sid sid 6017 Nov 9 21:40 terraform.tfstateterraform.tfstate is plain JSON, so jq is the right tool for a first look. The
top level carries bookkeeping fields:
jq '{version, terraform_version, serial, lineage}' terraform.tfstate{
"version": 4,
"terraform_version": "1.17.1",
"serial": 5,
"lineage": "2bdedcc9-4eb0-0ee6-45d8-6cd4b94a9849"
}versionis the state file format. Version 4 has been current for years.serialincreases every time the state changes. Terraform uses it to refuse to write an older copy over a newer one, which we'll trip over later.lineageis a random ID assigned when the state was first created. Two state files with different lineages are unrelated, and Terraform won't let you push one over the other.
The resources array is the important part. Each entry maps a resource address in
your configuration to the real object's ID and its last known attributes:
jq -r '.resources[] | "\(.type).\(.name) id=\(.instances[0].attributes.id)"' terraform.tfstatelocal_file.api_env id=73306c9e6b881361742635347390d80f61dad51d
local_file.logrotate id=bcc5e061c5415b54de815cfca780ea50a56bd403
local_file.worker_env id=4571e130063757eebb7da6845e0cef961d2d666e
random_password.db id=nonejq '.resources[] | select(.name == "api_env") | .instances[0].attributes | {filename, file_permission, content_sha256}' terraform.tfstate{
"filename": "./rendered/api.env",
"file_permission": "0640",
"content_sha256": "169baff98f3607716561c3ca959d33a3d8d0ef10073feeb5e7ed5a67e55410fc"
}That mapping is why state exists. Your configuration says "there should be a
local_file called api_env"; only the state knows it corresponds to the object with
ID 73306c9e…. On every plan Terraform reads the state, asks the provider for the
current attributes of each recorded object (the "Refreshing state" lines you see in
plans), and diffs the result against your configuration. Lose the state and Terraform
loses the mapping. It still sees your configuration, but it has no idea those objects
exist.
Sensitive values are stored in plain text
sensitive = true on the output hides the password from normal CLI output:
terraform outputdb_password = <sensitive>It does not hide it from the state file:
jq -r '.resources[] | select(.type == "random_password") | .instances[0].attributes.result' terraform.tfstateBtpdyMl1cPoC7EdZtDfHpFJqTwo documented commands print it on purpose, which matters if either one runs in a CI log:
terraform output -json
terraform output -raw db_password{
"db_password": {
"sensitive": true,
"type": "string",
"value": "BtpdyMl1cPoC7EdZtDfHpFJq"
}
}
BtpdyMl1cPoC7EdZtDfHpFJqThe sensitive flag is a display filter, nothing more. Every attribute a provider
returns, including database passwords, private keys and access tokens, is written to
state. That gives you three rules:
- Treat state as a secret. Never commit it to git, and restrict who can read the backend (bucket policies, IAM, HCP Terraform team permissions).
- Use a backend that encrypts at rest. The S3 backend with
encrypt = true, HCP Terraform, Azure Storage and GCS all do. - Where the provider supports it, keep secrets out of state entirely with ephemeral resources (Terraform 1.10+) and write-only arguments (1.11+).
Here's rule 3 for an RDS database. The ephemeral random_password exists only while
Terraform runs, and the AWS provider's password_wo argument sends the value to AWS
without recording it in state or in plan files:
terraform {
required_version = ">= 1.11"
required_providers {
aws = { source = "hashicorp/aws", version = "~> 6.0" }
random = { source = "hashicorp/random", version = "~> 3.7" }
}
}
provider "aws" {
region = "eu-west-1"
}
# Ephemeral: generated during the run, never written to state or plan files.
ephemeral "random_password" "db" {
length = 24
special = false
}
resource "aws_db_instance" "payments" {
identifier = "payments-prod"
engine = "postgres"
instance_class = "db.t4g.micro"
allocated_storage = 20
username = "payments_admin"
skip_final_snapshot = true
# Write-only argument: sent to AWS, never stored in state.
password_wo = ephemeral.random_password.db.result
password_wo_version = 1
}terraform init
terraform validate[…]
Success! The configuration is valid.I validated that configuration but didn't apply it, because creating the database
needs a real AWS account. The trade-off with write-only arguments is that Terraform
can't see whether the password changed, so you rotate it by incrementing
password_wo_version. Applications then read the password from a secrets manager,
not from Terraform outputs.
Reading state with the CLI
You rarely need jq for day-to-day work. These commands read state through
Terraform, so they also work with remote backends where there is no local file:
terraform state listlocal_file.api_env
local_file.logrotate
local_file.worker_env
random_password.dbterraform state show local_file.api_env# local_file.api_env:
resource "local_file" "api_env" {
content = <<-EOT
SERVICE_NAME=api
LISTEN_PORT=8080
DB_HOST=payments-db.internal.example.com
EOT
content_base64sha256 = "Fpuv+Y82B3FlYcPKlZ0zo9jQ7xAHP+615+1aZ+VUEPw="
[…]
directory_permission = "0777"
file_permission = "0640"
filename = "./rendered/api.env"
id = "73306c9e6b881361742635347390d80f61dad51d"
}| Command | What it does | Writes state? |
|---|---|---|
terraform state list | Every resource address in state | No |
terraform state show ADDR | Attributes of one instance | No |
terraform show | Whole state (or a saved plan) for humans; -json for scripts | No |
terraform output [-json] | Root module outputs | No |
terraform state pull | Raw state JSON from whatever backend is configured | No |
terraform state mv SRC DST | Change an object's address | Yes |
terraform state rm ADDR | Forget an object without destroying it | Yes |
terraform state push FILE | Overwrite the backend's state with a local file | Yes |
terraform force-unlock ID | Release a stuck lock | Lock only |
The last four are the dangerous ones. Later sections show the declarative replacements for most of them.
Backends: where state lives
A backend decides where state is stored and how it's locked. Without a backend
block you get the local backend writing terraform.tfstate next to your code. That's
fine for learning and for single-person projects, and wrong for anything a team
touches.
Moving state with init -migrate-state
Changing the backend takes two steps: edit the backend block, then let
terraform init copy the state across. To see the mechanics without a cloud account,
point the local backend at a shared directory. Add backend.tf:
terraform {
backend "local" {
path = "../state/payments.tfstate"
}
}Every other command now refuses to run until you initialize:
terraform planError: Backend initialization required, please run "terraform init"
Reason: Initial configuration of the requested backend "local"
The "backend" is the interface that Terraform uses to store state,
perform operations, etc. If this message is showing up, it means that the
Terraform configuration you're using is using a custom configuration for
the Terraform backend.
Changes to backend configurations require reinitialization. This allows
Terraform to set up the new configuration, copy existing state, etc. Please
run
"terraform init" with either the "-reconfigure" or "-migrate-state" flags to
use the current configuration.
[…]-migrate-state copies the existing state to the new location. -reconfigure would
start the new backend empty and leave the old state behind. You almost always want
-migrate-state, and Terraform asks before it copies (I answered yes):
terraform init -migrate-stateInitializing the backend...
Do you want to copy existing state to the new backend?
Pre-existing state was found while migrating the previous "local" backend to the
newly configured "local" backend. No existing state was found in the newly
configured "local" backend. Do you want to copy this state to the new "local"
backend? Enter "yes" to copy and "no" to start with an empty state.
Enter a value:
Successfully configured the backend "local"! Terraform will automatically
use this backend unless the backend configuration changes.
[…]
Terraform has been successfully initialized!ls -l . ../state.:
total 20
-rw-r--r-- 1 sid sid 77 Nov 9 21:40 backend.tf
-rw-r--r-- 1 sid sid 1044 Nov 9 21:40 main.tf
drwxr-xr-x 2 sid sid 4096 Nov 9 21:40 rendered
-rw-r--r-- 1 sid sid 0 Nov 9 21:40 terraform.tfstate
-rw-r--r-- 1 sid sid 6017 Nov 9 21:40 terraform.tfstate.backup
../state:
total 8
-rw-r--r-- 1 sid sid 6017 Nov 9 21:40 payments.tfstateNotice what was left behind: a zero-byte terraform.tfstate and a
terraform.tfstate.backup holding the old copy. Terraform no longer reads either.
Once you've confirmed terraform state list works against the new location, delete
them so nobody mistakes them for the real thing.
The same command moves state to S3, HCP Terraform, Azure Storage or GCS. Only the
backend block differs.
The S3 backend
For AWS teams the standard choice is an S3 bucket. This is the configuration I'd use for a new project today:
terraform {
backend "s3" {
bucket = "acme-terraform-state-prod"
key = "payments/terraform.tfstate"
region = "eu-west-1"
encrypt = true
use_lockfile = true
}
}keyis the object path inside the bucket. One bucket holds many projects; give each project, and each environment, its own key.use_lockfile = trueturns on S3-native locking. Terraform creates a lock object next to the state (payments/terraform.tfstate.tflock) with a conditional write, so a second writer can't create it at the same time. It arrived in Terraform 1.10 and replaces the older DynamoDB lock table.encrypt = trueturns on server-side encryption for the state object.
I can't run an S3 backend without an AWS account, so I checked this configuration the
only way available offline. terraform init parses and validates the backend
arguments before it contacts AWS, and with dummy credentials it got as far as the
credential check:
terraform initInitializing the backend...
Error: validating provider credentials: retrieving caller identity from STS: operation error STS: GetCallerIdentity, https response error StatusCode: 403, RequestID: 6f12db1f-b68d-4971-b5e8-64d574c66b5d, api error InvalidClientTokenId: The security token included in the request is invalid.That failure is the expected one. A mistake in the block fails earlier, before any network call. Here's an older configuration that still has a DynamoDB table, with a typo in it:
Error: Unsupported argument
on backend.tf line 8, in terraform:
8: dynamodb_tabel = "terraform-locks"
An argument named "dynamodb_tabel" is not expected here. Did you mean
"dynamodb_table"?With the typo fixed, Terraform 1.17 accepts it but tells you to move on:
Warning: Deprecated Parameter
on backend.tf line 8, in terraform:
8: dynamodb_table = "terraform-locks"
The parameter "dynamodb_table" is deprecated. Use parameter "use_lockfile"
instead.The bucket itself has to exist before any project can use it. The Where you'll use this section has a complete configuration for it.
HCP Terraform
With HCP Terraform (formerly Terraform Cloud), you replace the backend block with a
cloud block and run terraform login once. State, locking, run history and access
control are then handled for you:
terraform {
cloud {
organization = "acme"
workspaces {
name = "payments-prod"
}
}
}I haven't run this one; it needs an HCP Terraform account. The trade-off against S3 is the usual one: you pay for a managed service and get a UI, run history, policy checks and team permissions without building them yourself.
State locking
A backend that supports locking takes a lock before it writes state, and every command
that might write state (plan, apply, destroy, import, the state write
subcommands) asks for it. To watch it work we need an apply that lasts long enough to
collide with. time_sleep from the hashicorp/time provider waits during creation,
which makes a good stand-in for a slow database migration. This is a separate deploy
project next to payments:
terraform {
required_providers {
time = { source = "hashicorp/time", version = "~> 0.12" }
}
}
variable "release" {
type = string
default = "2026.11.1"
}
# Stands in for a slow change (a database migration, a load balancer
# update) so the apply holds the state lock for about 20 seconds.
resource "time_sleep" "migration" {
create_duration = "20s"
triggers = {
release = var.release
}
}Start an apply in the background. While it runs, the local backend keeps a description of the lock next to the state file:
terraform init
terraform apply -auto-approve > applyA.log 2>&1 &
sleep 4
cat .terraform.tfstate.lock.info | jq .{
"ID": "d41d3506-00e0-a47a-9724-8e1c240d35d3",
"Operation": "OperationTypeApply",
"Info": "",
"Who": "sid@devbox",
"Version": "1.17.1",
"Created": "2026-11-10T02:40:45.437370499Z",
"Path": "terraform.tfstate"
}Now a colleague tries to ship the next release from another terminal:
terraform apply -auto-approve -var release=2026.11.2Error: Error acquiring the state lock
Error message: resource temporarily unavailable
Lock Info:
ID: d41d3506-00e0-a47a-9724-8e1c240d35d3
Path: terraform.tfstate
Operation: OperationTypeApply
Who: sid@devbox
Version: 1.17.1
Created: 2026-11-10 02:40:45.437370499 +0000 UTC
Info:
Terraform acquires a state lock to protect the state from being written
by multiple users at the same time. Please resolve the issue above and try
again. For most commands, you can disable locking with the "-lock=false"
flag, but this is not recommended.That error is the lock doing its job. Who and Created tell you who to ask. With a
remote backend you see the same message when a CI job is in the middle of an apply.
And please ignore the -lock=false suggestion; it exists for broken backends, not for
impatient colleagues.
Waiting instead of failing: -lock-timeout
By default Terraform gives up at once. -lock-timeout makes it retry until the lock
is free:
time terraform apply -auto-approve -lock-timeout=60s -var release=2026.11.2Acquiring state lock. This may take a few moments...
time_sleep.migration: Refreshing state... [id=2026-11-10T02:41:04Z]
[…]
Plan: 1 to add, 0 to change, 1 to destroy.
time_sleep.migration: Destroying... [id=2026-11-10T02:41:04Z]
time_sleep.migration: Destruction complete after 0s
time_sleep.migration: Creating...
time_sleep.migration: Still creating... [00m09s elapsed]
time_sleep.migration: Still creating... [00m19s elapsed]
time_sleep.migration: Creation complete after 19s [id=2026-11-10T02:41:37Z]
Apply complete! Resources: 1 added, 0 changed, 1 destroyed.
real 0m48.434s
user 0m0.723s
sys 0m0.411sLook at the Refreshing state line: it shows the ID the first apply had just created,
because the second run only read the state after the lock was released. That's the
guarantee you want. The 48 seconds are roughly 20 spent waiting for the first apply
plus 20 for its own time_sleep.
I recommend -lock-timeout=5m in CI pipelines, where two merges in quick succession
are normal and a lock failure just means somebody has to click "re-run".
force-unlock
If a process dies while holding a lock on a remote backend (a CI runner killed
mid-apply, a laptop that lost its connection), the lock stays behind and every later
run fails with the error above. terraform force-unlock LOCK_ID releases it. Before
you run it, confirm the holder is really gone, because unlocking under a live apply is
exactly the corruption locking exists to prevent.
The local backend uses an operating system file lock, which the kernel releases when the process exits, so there is never anything to force:
terraform force-unlock -force 1234Failed to unlock state: LocalState not lockedRefactoring without destroying: moved blocks
Back in the payments project. The two local_file resources for the api and worker
services are copy-paste, so let's collapse them into one resource with for_each. In
main.tf, replace api_env and worker_env with:
variable "services" {
description = "Service name => listen port"
type = map(number)
default = {
api = 8080
worker = 9090
}
}
resource "local_file" "service_env" {
for_each = var.services
filename = "${path.module}/rendered/${each.key}.env"
file_permission = "0640"
content = <<-EOT
SERVICE_NAME=${each.key}
LISTEN_PORT=${each.value}
DB_HOST=payments-db.internal.example.com
EOT
}The files it describes are byte-for-byte identical to the existing ones. Terraform doesn't know that, because the addresses changed:
terraform plan | grep -E '^ # |^Plan:' # local_file.api_env will be destroyed
# (because local_file.api_env is not in configuration)
# local_file.service_env["api"] will be created
# local_file.service_env["worker"] will be created
# local_file.worker_env will be destroyed
# (because local_file.worker_env is not in configuration)
Plan: 2 to add, 0 to change, 2 to destroy.Destroy and recreate. With local files that's harmless; with an RDS instance or a
bucket full of data it's an outage. A moved block tells Terraform that the old
address and the new address are the same object. I keep them in a refactor.tf file
with a date, so reviewers can see why they exist:
# refactor.tf: November 2026, collapse the per-service files into one for_each.
moved {
from = local_file.api_env
to = local_file.service_env["api"]
}
moved {
from = local_file.worker_env
to = local_file.service_env["worker"]
}terraform planrandom_password.db: Refreshing state... [id=none]
local_file.service_env["worker"]: Refreshing state... [id=4571e130063757eebb7da6845e0cef961d2d666e]
local_file.logrotate: Refreshing state... [id=bcc5e061c5415b54de815cfca780ea50a56bd403]
local_file.service_env["api"]: Refreshing state... [id=73306c9e6b881361742635347390d80f61dad51d]
Terraform will perform the following actions:
# local_file.api_env has moved to local_file.service_env["api"]
resource "local_file" "service_env" {
id = "73306c9e6b881361742635347390d80f61dad51d"
# (10 unchanged attributes hidden)
}
# local_file.worker_env has moved to local_file.service_env["worker"]
resource "local_file" "service_env" {
id = "4571e130063757eebb7da6845e0cef961d2d666e"
# (10 unchanged attributes hidden)
}
Plan: 0 to add, 0 to change, 0 to destroy.
[…]Nothing to add or destroy, and two "has moved to" notes. Apply it, and the state now uses the new addresses:
terraform apply -auto-approve
terraform state list[…]
Apply complete! Resources: 0 added, 0 changed, 0 destroyed.
local_file.logrotate
local_file.service_env["api"]
local_file.service_env["worker"]
random_password.dbWhy a block in code rather than a command? Because a moved block goes through code
review and runs everywhere the configuration runs. If the same configuration manages
dev, staging and prod with separate states, each one picks up the move on its next
apply. A terraform state mv has to be run by hand against each state, and nobody
reviews it.
Leave moved blocks in place until every state that uses the configuration has been
applied. In a module other teams consume, keep them for good: you don't know when a
consumer will upgrade.
terraform state mv, when you need it
terraform state mv still has a place: one-off fixes to a single state, and moves a
moved block can't express. Always try it with -dry-run first:
terraform state mv -dry-run 'local_file.service_env["worker"]' 'local_file.service_env["jobs"]'Would move "local_file.service_env[\"worker\"]" to "local_file.service_env[\"jobs\"]"Notice the single quotes around each address. They matter, and Common mistakes shows what happens without them.
Handing a resource over: removed blocks
Say the platform team now manages logrotate configuration with Ansible, and the
payments project should stop owning local_file.logrotate without deleting the file.
Delete the logrotate resource block from main.tf and add a removed block:
# The platform team's Ansible role owns logrotate from now on.
removed {
from = local_file.logrotate
lifecycle {
destroy = false
}
}terraform plan[…]
Terraform will perform the following actions:
# local_file.logrotate will no longer be managed by Terraform, but will not be destroyed
# (destroy = false is set in the configuration)
. resource "local_file" "logrotate" {
id = "bcc5e061c5415b54de815cfca780ea50a56bd403"
# (10 unchanged attributes hidden)
}
Plan: 0 to add, 0 to change, 0 to destroy.
Warning: Some objects will no longer be managed by Terraform
If you apply this plan, Terraform will discard its tracking information for
the following objects, but it will not delete them:
- local_file.logrotate
After applying this plan, Terraform will no longer manage these objects. You
will need to import them into Terraform to manage them again.
[…]terraform apply -auto-approve
terraform state list
ls rendered[…]
local_file.service_env["api"]
local_file.service_env["worker"]
random_password.db
api.env
payments.logrotate
worker.envThe state no longer mentions logrotate, and the file is still on disk. Without the
removed block, deleting the resource from the configuration would have planned a
destroy. The older way is terraform state rm local_file.logrotate, which does the
same thing to one state with no review. As with moved, prefer the block.
With destroy = true (the default), a removed block destroys the object and
records why in code, which is a clearer history than just deleting the resource block.
Importing existing resources
The opposite situation: something exists that Terraform didn't create, and you want Terraform to manage it from now on. In AWS this is the bucket someone made in the console last year:
import {
to = aws_s3_bucket.audit_logs
id = "acme-audit-logs-prod"
}Each resource type documents its import ID format at the bottom of its registry page
(for S3 buckets it's the bucket name). An import block is reviewed like any other
code, and terraform plan shows the import before anything happens.
To run an import without an AWS account, here's a case with the same mechanics. The
payments team's read-only reporting user has a password someone set by hand, and we
want Terraform to own it without rotating it. random_password supports import, with
the existing password as the import ID. Add import.tf:
variable "existing_reporting_password" {
description = "The reporting user's current password, set by hand in 2024"
type = string
sensitive = true
}
import {
to = random_password.reporting_db
id = var.existing_reporting_password
}There's no resource "random_password" "reporting_db" block yet. Instead of writing
it by hand, let Terraform draft it from the imported object with
-generate-config-out:
export TF_VAR_existing_reporting_password='Rep0rtingReadOnly2024'
terraform plan -generate-config-out=reporting_db.tflocal_file.service_env["api"]: Refreshing state... [id=73306c9e6b881361742635347390d80f61dad51d]
local_file.service_env["worker"]: Refreshing state... [id=4571e130063757eebb7da6845e0cef961d2d666e]
random_password.reporting_db: Preparing import... [id=Rep0rtingReadOnly2024]
random_password.db: Refreshing state... [id=none]
random_password.reporting_db: Refreshing state... [id=none]
Terraform will perform the following actions:
# random_password.reporting_db will be imported
# (config will be generated)
resource "random_password" "reporting_db" {
bcrypt_hash = (sensitive value)
id = "none"
length = 21
lower = true
min_lower = 0
min_numeric = 0
min_special = 0
min_upper = 0
number = true
numeric = true
result = (sensitive value)
special = true
upper = true
}
Plan: 1 to import, 0 to add, 0 to change, 0 to destroy.
Warning: Config generation is experimental
Generating configuration during import is currently experimental, and the
generated configuration format may change in future versions.
─────────────────────────────────────────────────────────────────────────────
Terraform has generated configuration and written it to reporting_db.tf.
Please review the configuration and edit it as necessary before adding it to
version control.
[…]cat reporting_db.tf# __generated__ by Terraform
# Please review these resources and move them into your main configuration files.
# __generated__ by Terraform from "Rep0rtingReadOnly2024"
resource "random_password" "reporting_db" {
keepers = null
length = 21
lower = true
min_lower = 0
min_numeric = 0
min_special = 0
min_upper = 0
numeric = true
override_special = null
special = true
upper = true
}Look closely at two lines, because this surprised me. The variable is marked
sensitive, yet the import ID appears in plain text both in the
Preparing import... [id=Rep0rtingReadOnly2024] progress line and in the comment at
the top of the file. Import IDs are treated as identifiers, not secrets. For a bucket
name that's fine; for this case, delete that comment before committing, and don't run
such an import in a CI job whose logs other people can read.
Review the rest of the file like a pull request from a stranger. Terraform wrote every
argument it could fill in, including null placeholders you can delete. length = 21
comes from the password itself, and the other flags are the defaults the provider
assigns on import; they match what's now in state, which is what keeps the next plan
clean. Apply it and check:
terraform apply -auto-approve
terraform planrandom_password.reporting_db: Importing... [id=Rep0rtingReadOnly2024]
random_password.reporting_db: Import complete [id=Rep0rtingReadOnly2024]
Apply complete! Resources: 1 imported, 0 added, 0 changed, 0 destroyed.
[…]
No changes. Your infrastructure matches the configuration.
Terraform has compared your real infrastructure against your configuration
and found no differences, so no changes are needed.No changes after an import is the goal: the configuration describes the object
exactly as it is. If the first plan after an import wants to update or replace
something, fix the configuration before you apply. Once the import is done you can
delete the import block; leaving it in place is harmless.
The older terraform import ADDRESS ID command does the same thing imperatively. It
still works, but it doesn't show a plan first and can't draft the configuration for
you.
Detecting drift with -refresh-only
Drift is when reality changes behind Terraform's back: someone hotfixes a config file on a server, or edits a security group in the console. To simulate it, change the api service's port in the rendered file by hand:
sed -i 's/LISTEN_PORT=8080/LISTEN_PORT=8081/' rendered/api.env
terraform plan -refresh-only[…]
Note: Objects have changed outside of Terraform
Terraform detected the following changes made outside of Terraform since the
last "terraform apply" which may have affected this plan:
# local_file.service_env["api"] has been deleted
- resource "local_file" "service_env" {
- content = <<-EOT
SERVICE_NAME=api
LISTEN_PORT=8080
DB_HOST=payments-db.internal.example.com
EOT -> null
[…]
- filename = "./rendered/api.env" -> null
- id = "73306c9e6b881361742635347390d80f61dad51d" -> null
}
This is a refresh-only plan, so Terraform will not take any actions to undo
these. If you were expecting these changes then you can apply this plan to
record the updated values in the Terraform state without changing any remote
objects.
[…]A refresh-only plan answers one question: how does reality differ from the state? It
never proposes changes to infrastructure. The local provider identifies a file by
the checksum of its content, so an edited file counts as gone and the drift shows up
as "has been deleted". A cloud provider shows the changed attribute instead, such as
a modified ingress rule on a security group.
For automation, add -detailed-exitcode: exit code 0 means no drift, 2 means drift,
and 1 means an error.
terraform plan -refresh-only -detailed-exitcode > /dev/null; echo "exit code: $?"exit code: 2Once you know about the drift you have two choices:
- Reality is right.
terraform apply -refresh-onlyrecords the new values in state, and then you update the configuration to match. - The configuration is right. A normal apply puts things back:
terraform apply -auto-approve
grep LISTEN_PORT rendered/api.envlocal_file.service_env["api"]: Creating...
local_file.service_env["api"]: Creation complete after 0s [id=73306c9e6b881361742635347390d80f61dad51d]
Apply complete! Resources: 1 added, 0 changed, 0 destroyed.
LISTEN_PORT=8080The Testing and CI tutorial turns this into a scheduled drift-detection job.
Recovering from mistakes
State operations are the one place in Terraform where a typo can cost you real objects. The habits that protect you are short.
Back up before surgery. terraform state pull works with any backend, so it's the
universal backup:
terraform state pull > pre-surgery.tfstateDon't count on automatic backups. The local backend keeps the previous state in a
.backup file when an apply changes it. Here's an accidental state rm:
terraform state rm 'local_file.service_env["worker"]'
ls ../state
terraform plan | grep -E '^ # |^Plan:'Removed local_file.service_env["worker"]
Successfully removed 1 resource instance(s).
payments.tfstate
payments.tfstate.backup
# local_file.service_env["worker"] will be created
Plan: 1 to add, 0 to change, 0 to destroy.There is a payments.tfstate.backup, but when I checked its serial it predated the
state rm: with a backend block configured, state rm did not refresh it. (In an
earlier run with no backend block, state rm did write a timestamped
terraform.tfstate.<timestamp>.backup.) The behavior varies by setup, which is
exactly why the state pull habit matters.
The worker file still exists on disk, but Terraform has forgotten it and plans to create it. On a real cloud resource that create would fail with "already exists" or, worse, succeed and make a duplicate.
Restore with state push, and respect the serial check. Pushing the backup we took fails:
terraform state push pre-surgery.tfstateFailed to write state: cannot import state with serial 9 over newer state with serial 10That's the serial from the first section protecting you: the backup is older than
the current state. When you're certain the backup is what you want (you know the only
change since was the accident), -force overrides the check:
terraform state push -force pre-surgery.tfstate
terraform plan | tail -1and found no differences, so no changes are needed.Lineage protects you from pushing the wrong project. Pushing the deploy
project's state into the payments backend fails:
terraform state push ../deploy/terraform.tfstateFailed to write state: cannot import state with lineage "bcafc177-7262-a325-b848-3fa7df184fbc" over unrelated state with lineage "2bdedcc9-4eb0-0ee6-45d8-6cd4b94a9849"With an S3 backend, turn on bucket versioning (the configuration below does). Every
state write then becomes a recoverable object version, and "restore last Tuesday's
state" is an S3 operation instead of a disaster. If state is lost completely and there
is no backup, the way back is an import block for every resource. It's slow, but it
works.
Where you'll use this
Here's the setup I'd build on day one for a team on AWS: a dedicated bucket for state,
created once, with versioning, encryption, public access blocked and old versions
expiring after 90 days. Every other project points its S3 backend at it with its own
key.
# bootstrap/main.tf: the bucket that holds every other project's state.
# Applied once per AWS account, by hand, with local state.
terraform {
required_version = ">= 1.11"
required_providers {
aws = { source = "hashicorp/aws", version = "~> 6.0" }
}
}
provider "aws" {
region = "eu-west-1"
}
resource "aws_s3_bucket" "state" {
bucket = "acme-terraform-state-prod"
lifecycle {
prevent_destroy = true
}
}
resource "aws_s3_bucket_versioning" "state" {
bucket = aws_s3_bucket.state.id
versioning_configuration {
status = "Enabled"
}
}
resource "aws_s3_bucket_server_side_encryption_configuration" "state" {
bucket = aws_s3_bucket.state.id
rule {
apply_server_side_encryption_by_default {
sse_algorithm = "aws:kms"
}
}
}
resource "aws_s3_bucket_public_access_block" "state" {
bucket = aws_s3_bucket.state.id
block_public_acls = true
block_public_policy = true
ignore_public_acls = true
restrict_public_buckets = true
}
resource "aws_s3_bucket_lifecycle_configuration" "state" {
bucket = aws_s3_bucket.state.id
rule {
id = "expire-old-state-versions"
status = "Enabled"
filter {}
noncurrent_version_expiration {
noncurrent_days = 90
}
}
}prevent_destroy makes any plan that would delete the bucket fail, which is the right
default for the one bucket that holds everyone's state.
To plan this without an AWS account, I added an override file. Terraform merges any
file whose name ends in _override.tf into the matching block, so the real
configuration stays clean:
# offline_override.tf: lets `terraform plan` run without an AWS account.
# Never commit this file.
provider "aws" {
skip_credentials_validation = true
skip_requesting_account_id = true
skip_metadata_api_check = true
}export AWS_ACCESS_KEY_ID=AKIAIOSFODNN7EXAMPLE
export AWS_SECRET_ACCESS_KEY=wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY
terraform init
terraform plan[…]
# aws_s3_bucket_versioning.state will be created
+ resource "aws_s3_bucket_versioning" "state" {
+ bucket = (known after apply)
+ id = (known after apply)
+ region = "eu-west-1"
+ versioning_configuration {
+ mfa_delete = (known after apply)
+ status = "Enabled"
}
}
Plan: 5 to add, 0 to change, 0 to destroy.
[…]The plan is real; the apply needs a real account, and I didn't run it. The rest of the day-one checklist uses only things you've now seen:
- Every project gets
backend "s3"withuse_lockfile = trueand a uniquekey. - CI runs
planandapplywith-lock-timeout=5m. - Refactors ship with
movedblocks in the same pull request, hand-overs withremovedblocks, and adoptions withimportblocks. - A nightly job runs
terraform plan -refresh-only -detailed-exitcodeand alerts on exit code 2. - Nobody runs a
terraform statewrite command withoutterraform state pullfirst.
Common mistakes
Unquoted instance addresses in the shell
terraform state show local_file.service_env["api"]Error: Error parsing instance address: local_file.service_env[api]
This command requires that the address references one specific instance.
To view the available instances, use "terraform state list". Please modify
the address to reference a specific instance.Bash removed the double quotes before Terraform saw them, so Terraform received
local_file.service_env[api], which isn't a valid instance address. Wrap any address
that contains brackets in single quotes: 'local_file.service_env["api"]'. Single
quotes work in PowerShell too.
Changing the backend block and forgetting init
The "Backend initialization required" error from the backends section looks
alarming, but it's the safe behavior: Terraform refuses to guess whether you meant to
migrate the state or start fresh. Run terraform init -migrate-state to carry the
state over. Use -reconfigure only when you really do want an empty new backend.
Importing a resource type that doesn't support import
Not every resource type can be imported, and local_file is one that can't:
import {
to = local_file.nginx
id = "nginx-shop.conf"
}terraform plan -generate-config-out=adopted.tflocal_file.nginx: Preparing import... [id=nginx-shop.conf]
Planning failed. Terraform encountered an error while generating this plan.
Error: Resource Import Not Implemented
This resource does not support import. Please contact the provider developer
for additional information.Check the "Import" section at the bottom of a resource type's documentation before you plan an adoption. Without import support, your options are to recreate the object under Terraform or to leave it unmanaged and read it with a data source.
Two laptops, two state files
If two people each have their own terraform.tfstate, neither file is the truth, and
locking can't help because each lock only protects its own file. The symptom is a plan
that wants to create resources you know exist. The fix is the backends section: one
shared backend with locking, and delete the local copies after migrating.
Summary
- State maps each resource address in your configuration to a real object ID and its last known attributes. Without it Terraform can't tell that anything exists.
sensitive = trueonly hides values from CLI output. State stores them in plain text, so keep state in an encrypted, access-controlled backend, and use ephemeral resources and write-only arguments where the provider supports them. Import IDs are printed in plain text even when they come from a sensitive variable.- A backend stores and locks state; change backends with
terraform init -migrate-state. On AWS, use the S3 backend withuse_lockfile = true; the DynamoDB lock table is deprecated. - Locks stop concurrent writes. Use
-lock-timeoutin CI so runs wait instead of failing, andforce-unlockonly when the lock holder is definitely dead. movedblocks rename and restructure without destroying anything, and they get code review.removedblocks stop managing objects (destroy = false) or destroy them on purpose.importblocks adopt existing objects, and-generate-config-outdrafts the configuration.terraform plan -refresh-only -detailed-exitcodedetects drift: exit code 2 means reality and state disagree.- Run
terraform state pullto a file before anyterraform statewrite command. Theserialandlineagechecks stop you from pushing an older or unrelated state unless you force it.
Exercises
All four use the payments project as it stands at the end of the tutorial.
Without opening any file in rendered/, print the listen port configured for the
worker service. Do it twice: once with a terraform state command, and once with jq
on the output of terraform state pull.
Hint
The worker's file is the instance local_file.service_env["worker"]. In the JSON,
instances created by for_each carry an index_key.
Show Solution
terraform state show 'local_file.service_env["worker"]' | grep LISTEN_PORT
terraform state pull | jq -r '.resources[] | select(.name == "service_env") | .instances[] | select(.index_key == "worker") | .attributes.content' | grep LISTEN_PORT LISTEN_PORT=9090
LISTEN_PORT=9090terraform state show prints the whole instance, so grep narrows it down. The jq
version is better in scripts: it selects the instance by index_key, and because
state pull fetches state from wherever the backend keeps it, it works unchanged
against S3.
Rename random_password.db to random_password.payments_db, and update the output
that references it. Prove two things: the plan adds and destroys nothing, and the
password is the same before and after the apply.
Show Solution
Rename the resource and the reference in the output, then add a moved block to
refactor.tf:
moved {
from = random_password.db
to = random_password.payments_db
}terraform output -raw db_password; echo
terraform plan | grep -E 'has moved|Plan:'
terraform apply -auto-approve > /dev/null
terraform output -raw db_password; echoBtpdyMl1cPoC7EdZtDfHpFJq
# random_password.db has moved to random_password.payments_db
Plan: 0 to add, 0 to change, 0 to destroy.
BtpdyMl1cPoC7EdZtDfHpFJqWithout the moved block the plan would destroy the old password and create a new
one. In real life, that's every service using the database losing access at once.
Delete rendered/worker.env by hand. Write a shell function that runs a refresh-only
plan and prints CLEAN, DRIFT or ERROR based on the exit code. Run it, put the
file back with Terraform, and run it again.
Show Solution
rm rendered/worker.env
check_drift() {
terraform plan -refresh-only -detailed-exitcode > /dev/null 2>&1
case $? in
0) echo "CLEAN" ;;
2) echo "DRIFT" ;;
*) echo "ERROR" ;;
esac
}
check_drift
terraform apply -auto-approve > /dev/null
check_driftDRIFT
CLEAN-detailed-exitcode is what makes the check scriptable. Reading $? in the very next
statement matters, because any command in between replaces it. Treating every code
other than 0 and 2 as an error means a broken backend or expired credentials can't
masquerade as "no drift".
The reporting team is getting its own Terraform project. Move
random_password.reporting_db out of the payments state and into a new reporting
project next to it, without changing the password. Both projects must end with
No changes from terraform plan.
Hint
There's no command that moves a resource between two backends. Combine two blocks you already know: one that lets go in the old project, and one that adopts in the new one.
Show Solution
In payments, read the current value, delete import.tf and reporting_db.tf, and
add a removed block (in a file such as handover-reporting.tf):
terraform state pull | jq -r '.resources[] | select(.name == "reporting_db") | .instances[0].attributes.result'Rep0rtingReadOnly2024removed {
from = random_password.reporting_db
lifecycle {
destroy = false
}
}terraform apply -auto-approve | grep -E 'no longer|Apply complete' # random_password.reporting_db will no longer be managed by Terraform, but will not be destroyed
Warning: Some objects will no longer be managed by Terraform
After applying this plan, Terraform will no longer manage these objects. You
Apply complete! Resources: 0 added, 0 changed, 0 destroyed.In a new ../reporting directory, create main.tf:
terraform {
required_providers {
random = { source = "hashicorp/random", version = "~> 3.6" }
}
}
variable "reporting_password" {
type = string
sensitive = true
}
import {
to = random_password.reporting_db
id = var.reporting_password
}terraform init
export TF_VAR_reporting_password='Rep0rtingReadOnly2024'
terraform plan -generate-config-out=reporting_db.tf | grep -E 'imported|Plan:'
terraform apply -auto-approve | grep -E 'Import complete|Apply complete'
terraform plan | grep -E 'No changes|Plan:'
cd ../payments && terraform plan | grep -E 'No changes|Plan:' # random_password.reporting_db will be imported
Plan: 1 to import, 0 to add, 0 to change, 0 to destroy.
random_password.reporting_db: Import complete [id=Rep0rtingReadOnly2024]
Apply complete! Resources: 1 imported, 0 added, 0 changed, 0 destroyed.
No changes. Your infrastructure matches the configuration.
No changes. Your infrastructure matches the configuration.removed with destroy = false followed by import is the reviewable way to split
a state. Do it in that order: if both projects managed the object at the same time,
each would try to correct the other's changes.
Tested with: Terraform v1.17.1 on Ubuntu 24.04 (WSL2, linux_amd64), with
hashicorp/local v2.9.1, hashicorp/random v3.9.1, hashicorp/time v0.14.2 and
hashicorp/aws v6.72.0, November 2026. Everything with the local, random and time
providers was applied for real, including both concurrent applies; commands ran with
-no-color, and the -migrate-state prompt was answered by piping yes into it.
Nothing touched a real AWS account: the write-only RDS example was only validated, the
S3 backend only got as far as the credential check with dummy keys, the state bucket
was planned (not applied) with dummy credentials and the override file shown, and the
HCP Terraform cloud block was not run.
Written by The Dark Debugger
Engineer documenting the things that took me too long to learn, so they take you less time. Every Terraform tutorial here is tested end to end before publishing.