CLI Testing

A CLI's API is: args + env + stdin → stdout + stderr + exit code + side effects. Test that contract.

Contract checklist

Tooling

# Bats (bash)
#!/usr/bin/env bats
@test "prints version" { run mycli --version; [ "$status" -eq 0 ]; [[ "$output" =~ ^mycli\ [0-9]+\.[0-9]+ ]]; }
@test "rejects unknown flag" { run mycli --nope; [ "$status" -eq 2 ]; [[ "$output" == *"Usage"* ]]; }

# pytest + subprocess
import subprocess, json
def run(*a, inp=None): return subprocess.run(["mycli",*a], input=inp, capture_output=True, text=True)
def test_json_mode():
    r = run("list","--json"); assert r.returncode == 0 and r.stderr == ""
    assert isinstance(json.loads(r.stdout), list)

Other: cram/prysk (transcript tests), expect/pexpect/node-pty for interactive TTY flows, snapshot-golden-testing for help text and reports, shellcheck + shfmt for shell scripts, Go testscript, Rust assert_cmd + predicates, Node execa.

Test the installed artifact

Package (npm/pip/brew/cargo/binary), install into a clean container, run smoke commands, verify shebang/permissions/completions (bash, zsh, fish), man page, --version matches tag.

Related

unit-testing, integration-testing, snapshot-golden-testing, installation-upgrade-testing, fuzz-testing