# Testing Automation

Test your automation as you develop it, then run the same checks in CI. Check
shared functions, command behavior, and deployed services so changes to your
build and release process have tests of their own.

A `type: test` step runs checks and reports which passed and which failed.
Each check can be a script, an HTTP request, a shell command, or another supported
step. Use test steps in workflows, custom commands, and lifecycle hooks. Atmos
collects the results, shows the output of failed checks, and returns a failing
exit code when a suite fails.

In a script written in the Atmos Automation Language, use a condition and
`fail()` to report an unmet expectation. A script that finishes successfully
passes.

## Create a test workflow

With Atmos installed and on your `PATH`, save this as `atmos.yaml`. The first
example uses only built-in functions and needs no external tools:

```yaml title="atmos.yaml"
base_path: .
workflows:
  base_path: workflows
```

Save this complete workflow as `workflows/checks.yaml`. It produces a structured
result, then checks both that result and a reusable calculation:

```yaml title="workflows/checks.yaml"
workflows:
  check:
    steps:
      - name: calculate
        type: script
        interpreter: starlark
        script: |
          output = {"replicas": 3, "workers": 12}
      - name: checks
        type: test
        title: Capacity checks
        steps:
          - name: result
            type: script
            interpreter: starlark
            env:
              RESULT: '{{ .steps.calculate.value }}'
            script: |
              actual = json.decode(env["RESULT"])
              if actual["workers"] != actual["replicas"] * 4:
                  fail("worker count must equal replicas times four")
          - name: calculation
            type: script
            interpreter: starlark
            script: |
              def capacity(replicas):
                  return replicas * 4
              for replicas, expected in [(1, 4), (2, 8), (3, 12)]:
                  actual = capacity(replicas)
                  if actual != expected:
                      fail("expected {}, got {}".format(expected, actual))
```

Run from the directory containing `atmos.yaml`:

```shell
atmos workflow check -f checks
```

Both checks pass and the command exits with code zero. The `output` value from
the first step is JSON-encoded; the test receives it through its environment and
decodes it into a dictionary. Shared logic can live in a `.star` module loaded
by both your automation and its checks.

Change the produced worker count from `12` to `11` and rerun. The result check
fails with `worker count must equal replicas times four`, the independent
calculation check still passes, and the workflow exits nonzero.

## Test a custom command's behavior

Test the interface your team uses as well as the logic behind it. For the
[custom command guide's](/automation/custom-commands) `atmos capacity`, add this
child under the test step's `steps` list in the same project:

```yaml
- name: rejects-zero
  type: script
  interpreter: starlark
  script: |
    result = exec.run(
        ["atmos", "capacity", "--replicas", "0"],
        check=False,
        output="capture",
    )
    if result.exit_code == 0:
        fail("capacity accepted zero replicas")
    if "replicas must be positive" not in result.stdout + result.stderr:
        fail("capacity failed for an unexpected reason")
```

This runs the real custom command. `check=False` returns an unsuccessful process
result so the script can check both the exit code and the diagnostic. Launch or
cancellation errors still fail the step. `output="capture"` keeps the expected
error out of the live output while making it available to the check.

The same approach tests standalone scripts and other tools. Use fixed inputs
for deterministic checks; subprocess tests still need the tools they invoke.

## Read results and use them in CI

Successful logs stay hidden by default. Failed checks display their buffered
output and errors; set `output: all` on the test step to include successful logs.
Interactive terminals show a live tree, while CI and redirected output use
static results.

The default failure policy lets independent checks finish, then fails the group
if unhandled failures remain. Nest `parallel` or `matrix` groups when checks can
run concurrently or need several input combinations. Keep the default failing
policy for suites intended to gate CI.

Test steps also work inside custom commands and lifecycle hooks. See the
[test step reference](/steps/type/test) for failure policies, supported
children, counts, and hook configuration. The
[language overview](/automation/language) describes output
values and process execution; the [test example](/examples/tests) shows a larger
suite.