Generate Terraform Backend

The backend section generates Terraform backend configuration files (backend.tf.json) for your components. Define backend settings once in your stacks and Atmos generates the appropriate files for each environment.

How It Works

When you run any atmos terraform command, Atmos:

1. Reads your backend and backend_type configuration from the stack
2. Deep-merges settings from all inherited stack manifests
3. Generates a backend.tf.json file in the component directory
4. Terraform uses this file to configure state storage

This separation means your Terraform modules stay clean—no hardcoded backend configuration in your source code.

Generated Files

Add backend.tf.json to your .gitignore since these files are generated automatically by Atmos and should not be committed to version control:

# Atmos generated files
backend.tf.json

Some automation systems may require the generated file to be committed—in those cases, committing backend.tf.json is acceptable.

Use Cases

• State Management: Configure S3, Azure Blob, GCS, or other backends for remote state storage.
• Environment Isolation: Use different state storage per environment or account.
• State Locking: Configure native locking (use_lockfile) or DynamoDB for legacy setups.
• Remote State Access: Configure remote_state_backend for cross-component state references.

Configuration Scopes

Backend settings can be defined at multiple levels, with more specific scopes overriding broader ones:

Scope	Example File	Effect
Organization	stacks/orgs/acme/_defaults.yaml	All components inherit
Account/Stage	stacks/orgs/acme/plat/prod/_defaults.yaml	Override for prod
Component-type	Under terraform: in any stack	All Terraform components
Component	Under components.terraform.<name>:	Single component

Component-Type Level

Backend settings defined under terraform apply to all Terraform components:

Stack Configuration

stacks/orgs/acme/_defaults.yaml

terraform:
  backend_type: s3
  backend:
    s3:
      bucket: acme-ue1-root-tfstate
      region: us-east-1
      encrypt: true
      use_lockfile: true

Generated File

components/terraform/vpc/backend.tf.json

{
  "terraform": {
    "backend": {
      "s3": {
        "bucket": "acme-ue1-root-tfstate",
        "region": "us-east-1",
        "encrypt": true,
        "use_lockfile": true,
        "key": "ue1/prod/vpc/terraform.tfstate"
      }
    }
  }
}

Component Level

Backend settings within a component override the defaults:

Stack Configuration

stacks/orgs/acme/plat/prod/us-east-1.yaml

components:
  terraform:
    special-component:
      backend_type: s3
      backend:
        s3:
          bucket: acme-ue1-prod-special-tfstate
          key: "special/terraform.tfstate"

Generated File

components/terraform/special-component/backend.tf.json

{
  "terraform": {
    "backend": {
      "s3": {
        "bucket": "acme-ue1-prod-special-tfstate",
        "key": "special/terraform.tfstate"
      }
    }
  }
}

Backend Types

Atmos supports all Terraform backend types. Configure using backend_type and the corresponding configuration under backend.

S3 Backend

The most common backend for AWS environments. We recommend using use_lockfile: true for native S3 state locking (Terraform 1.10+, OpenTofu 1.8+) instead of DynamoDB:

Stack Configuration

terraform:
  backend_type: s3
  backend:
    s3:
      bucket: acme-ue1-root-tfstate
      region: us-east-1
      key: "{{ .environment }}/{{ .stage }}/{{ .component }}/terraform.tfstate"
      encrypt: true
      use_lockfile: true  # Native S3 locking (Terraform 1.10+)

Generated File

components/terraform/vpc/backend.tf.json

{
  "terraform": {
    "backend": {
      "s3": {
        "bucket": "acme-ue1-root-tfstate",
        "region": "us-east-1",
        "key": "ue1/prod/vpc/terraform.tfstate",
        "encrypt": true,
        "use_lockfile": true
      }
    }
  }
}

S3 Backend with Assume Role

For cross-account access, configure the backend to assume a role:

Stack Configuration

terraform:
  backend_type: s3
  backend:
    s3:
      bucket: acme-ue1-root-tfstate
      region: us-east-1
      key: "{{ .environment }}/{{ .stage }}/{{ .component }}/terraform.tfstate"
      encrypt: true
      use_lockfile: true
      assume_role:
        role_arn: "arn:aws:iam::{{ .vars.state_account_id }}:role/TerraformStateAccess"
        session_name: "atmos-{{ .component }}"

Generated File

components/terraform/vpc/backend.tf.json

{
  "terraform": {
    "backend": {
      "s3": {
        "bucket": "acme-ue1-root-tfstate",
        "region": "us-east-1",
        "key": "ue1/prod/vpc/terraform.tfstate",
        "encrypt": true,
        "use_lockfile": true,
        "assume_role": {
          "role_arn": "arn:aws:iam::123456789012:role/TerraformStateAccess",
          "session_name": "atmos-vpc"
        }
      }
    }
  }
}

S3 Backend with SSE-C Encryption

For state files encrypted with SSE-C (Server-Side Encryption with Customer-Provided Keys), provide the customer key so that !terraform.state can decrypt state when reading directly from S3:

terraform:
  backend_type: s3
  backend:
    s3:
      bucket: acme-ue1-root-tfstate
      region: us-east-1
      key: "{{ .environment }}/{{ .stage }}/{{ .component }}/terraform.tfstate"
      encrypt: true
      use_lockfile: true
      sse_customer_key: "base64-encoded-32-byte-key"

The sse_customer_key must be a base64-encoded 256-bit (32-byte) key. It can also be provided via the AWS_SSE_CUSTOMER_KEY environment variable. The backend attribute takes precedence over the environment variable.

note

The sse_customer_key attribute is used by Atmos when reading state directly via !terraform.state. Terraform/OpenTofu handles SSE-C independently through its own backend configuration.

S3 Backend with DynamoDB Locking (Legacy)

For Terraform versions before 1.10, use DynamoDB for state locking:

terraform:
  backend_type: s3
  backend:
    s3:
      bucket: acme-ue1-root-tfstate
      region: us-east-1
      key: "{{ .environment }}/{{ .stage }}/{{ .component }}/terraform.tfstate"
      dynamodb_table: acme-ue1-root-tfstate-lock
      encrypt: true

Azure Blob Backend

For Azure environments:

Stack Configuration

terraform:
  backend_type: azurerm
  backend:
    azurerm:
      resource_group_name: terraform-state-rg
      storage_account_name: tfstateaccount
      container_name: tfstate
      key: "{{ .environment }}/{{ .stage }}/{{ .component }}/terraform.tfstate"

Generated File

components/terraform/vpc/backend.tf.json

{
  "terraform": {
    "backend": {
      "azurerm": {
        "resource_group_name": "terraform-state-rg",
        "storage_account_name": "tfstateaccount",
        "container_name": "tfstate",
        "key": "ue1/prod/vpc/terraform.tfstate"
      }
    }
  }
}

GCS Backend

For Google Cloud environments:

Stack Configuration

terraform:
  backend_type: gcs
  backend:
    gcs:
      bucket: acme-terraform-state
      prefix: "{{ .environment }}/{{ .stage }}/{{ .component }}"

Generated File

components/terraform/vpc/backend.tf.json

{
  "terraform": {
    "backend": {
      "gcs": {
        "bucket": "acme-terraform-state",
        "prefix": "ue1/prod/vpc"
      }
    }
  }
}

Terraform Cloud / Enterprise

For Terraform Cloud or Enterprise:

Stack Configuration

terraform:
  backend_type: remote
  backend:
    remote:
      hostname: app.terraform.io
      organization: acme
      workspaces:
        name: "{{ .namespace }}-{{ .environment }}-{{ .stage }}-{{ .component }}"

Or using the newer cloud backend:

terraform:
  backend_type: cloud
  backend:
    cloud:
      organization: acme
      workspaces:
        name: "{{ .namespace }}-{{ .environment }}-{{ .stage }}-{{ .component }}"

Generated File

components/terraform/vpc/backend.tf.json

{
  "terraform": {
    "backend": {
      "remote": {
        "hostname": "app.terraform.io",
        "organization": "acme",
        "workspaces": {
          "name": "acme-ue1-prod-vpc"
        }
      }
    }
  }
}

Local Backend

For development or single-user scenarios:

Stack Configuration

terraform:
  backend_type: local
  backend:
    local:
      path: "{{ .component }}/terraform.tfstate"

Generated File

components/terraform/vpc/backend.tf.json

{
  "terraform": {
    "backend": {
      "local": {
        "path": "vpc/terraform.tfstate"
      }
    }
  }
}

HTTP Backend

For state stored behind a generic HTTP endpoint, such as GitLab-managed Terraform state:

Stack Configuration

terraform:
  backend_type: http
  backend:
    http:
      address: "https://gitlab.example.com/api/v4/projects/123/terraform/state/vpc"
      lock_address: "https://gitlab.example.com/api/v4/projects/123/terraform/state/vpc/lock"
      unlock_address: "https://gitlab.example.com/api/v4/projects/123/terraform/state/vpc/lock"
      lock_method: POST
      unlock_method: DELETE
      username: gitlab-ci-token
      password: !env CI_JOB_TOKEN

Generated File

components/terraform/vpc/backend.tf.json

{
  "terraform": {
    "backend": {
      "http": {
        "address": "https://gitlab.example.com/api/v4/projects/123/terraform/state/vpc",
        "lock_address": "https://gitlab.example.com/api/v4/projects/123/terraform/state/vpc/lock",
        "unlock_address": "https://gitlab.example.com/api/v4/projects/123/terraform/state/vpc/lock",
        "lock_method": "POST",
        "unlock_method": "DELETE",
        "username": "gitlab-ci-token",
        "password": "glcbt-..."
      }
    }
  }
}

warning

backend.tf.json is generated with credentials expanded in plain text — in the example above, the resolved CI_JOB_TOKEN value. The same plaintext values also persist in .terraform/terraform.tfstate and in any saved plan file (see Planfile Backend Limitations). Never commit these files, log their contents, or upload them as CI artifacts.

Prefer the TF_HTTP_USERNAME and TF_HTTP_PASSWORD environment variables and omit username/password from the backend configuration entirely — Terraform/OpenTofu reads them at runtime without persisting them to disk. backend.tf.json should still be excluded via .gitignore (Atmos regenerates it on demand when auto_generate_backend_file is enabled).

Remote State Backend

The remote_state_backend configuration is separate from backend and controls how other components can read this component's state:

terraform:
  # Where this component stores its state
  backend_type: s3
  backend:
    s3:
      bucket: acme-ue1-root-tfstate
      region: us-east-1

  # How other components read this component's state
  remote_state_backend_type: s3
  remote_state_backend:
    s3:
      bucket: acme-ue1-root-tfstate
      region: us-east-1

This separation allows for scenarios like:

• Using Terraform Cloud for operations but S3 for remote state access
• Different credentials for writing vs. reading state
• Custom remote state configurations

Backend Provisioning

Atmos can automatically provision S3 backend infrastructure before running Terraform commands, solving the bootstrap problem of "how do I create state storage before I can use Terraform?"

Enable Automatic Provisioning

terraform:
  backend_type: s3
  backend:
    s3:
      bucket: acme-ue1-dev-tfstate
      region: us-east-1
      use_lockfile: true

  provision:
    backend:
      enabled: true  # Automatically create backend if it doesn't exist

When provision.backend.enabled is true, Atmos will:

1. Check if the backend exists before running Terraform
2. Create it with secure defaults (versioning, encryption, public access blocked)
3. Proceed with normal Terraform operations

See Backend Provisioning for complete documentation.

Examples

Environment-Specific State Buckets

stacks/orgs/acme/plat/dev/_defaults.yaml

terraform:
  backend_type: s3
  backend:
    s3:
      bucket: acme-ue1-dev-tfstate
      region: us-east-1
      encrypt: true
      use_lockfile: true

stacks/orgs/acme/plat/prod/_defaults.yaml

terraform:
  backend_type: s3
  backend:
    s3:
      bucket: acme-ue1-prod-tfstate
      region: us-east-1
      encrypt: true
      use_lockfile: true

Dynamic State Keys with Templates

stacks/orgs/acme/_defaults.yaml

terraform:
  backend_type: s3
  backend:
    s3:
      bucket: acme-ue1-root-tfstate
      region: "{{ .vars.region }}"
      key: "{{ .environment }}/{{ .stage }}/{{ .component }}/terraform.tfstate"
      encrypt: true
      use_lockfile: true

Component-Specific Backend Override

stacks/orgs/acme/plat/prod/us-east-1.yaml

import:
  - orgs/acme/_defaults

components:
  terraform:
    # Uses default backend
    vpc:
      vars:
        vpc_cidr: "10.0.0.0/16"

    # Uses custom backend for sensitive data
    secrets-manager:
      backend_type: s3
      backend:
        s3:
          bucket: acme-ue1-prod-secrets-tfstate
          region: us-east-1
          key: "secrets/terraform.tfstate"
          encrypt: true
          use_lockfile: true
          kms_key_id: alias/terraform-state-key
      vars:
        # ...

Multi-Cloud Configuration

stacks/orgs/acme/aws/_defaults.yaml

terraform:
  backend_type: s3
  backend:
    s3:
      bucket: acme-ue1-root-tfstate
      region: us-east-1
      use_lockfile: true

stacks/orgs/acme/azure/_defaults.yaml

terraform:
  backend_type: azurerm
  backend:
    azurerm:
      resource_group_name: terraform-state-rg
      storage_account_name: acmetfstate
      container_name: tfstate

Workspace Configuration

For backends that support workspaces, you can configure workspace patterns:

terraform:
  backend_type: remote
  backend:
    remote:
      organization: acme
      workspaces:
        prefix: "acme-"

components:
  terraform:
    vpc:
      terraform_workspace: "{{ .environment }}-{{ .stage }}-vpc"

See Terraform Workspaces for detailed workspace configuration.

Best Practices

1. Use Remote State: Always use a remote backend for team environments to enable collaboration and state locking.

2. Enable Encryption: Always enable encryption for backends that support it (S3, Azure Blob, GCS).

3. Configure State Locking: Use use_lockfile: true for S3 (Terraform 1.10+), blob leases for Azure, or built-in locking for other backends.

4. Organize State Keys: Use a consistent key structure that includes environment, stage, and component information.

5. Separate Sensitive State: Consider using separate state buckets or additional encryption for components managing secrets.

6. Use Templates: Leverage Go templates for dynamic backend configuration based on context variables.

Related

• Terraform Backend Configuration - Component-level backend defaults
• Terraform Backends - Conceptual overview and inheritance
• Backend Provisioning - Automatic backend creation
• Remote State - Reading state from other components
• Terraform Workspaces