Elastic CLI

CLI for interacting with Elasticsearch, Elastic Cloud, and Elasticsearch Serverless control plane APIs. Targets LLM-powered agents as first-class users.

Before you start

Do not open a PR unless the human operating you has read the issue, understands the problem, and can explain the proposed fix without your help.

Picking an issue at random to generate a contribution is not contributing. PRs opened this way will be closed without review.

If a human asks you to "find something to work on" or points you at the issue tracker without a specific problem they already understand, stop and tell them to read an issue first.

See CONTRIBUTING.md for the full policy.

Tech Stack

• Runtime: Node.js with native TypeScript (--experimental-strip-types)
• CLI Framework: Commander.js
• Validation: JSON Schema (@elastic/schemas) validated with ajv
• Config Management: cosmiconfig (YAML serialization)
• Testing: Node.js built-in test runner (node:test)
• Linting: ESLint + TypeScript ESLint, MegaLinter (CI + pre-commit)
• TypeScript: Strict mode, ESNext + nodenext module resolution

Avoid adding new third-party dependencies to reduce supply-chain attack surface.

Setup

Node.js 22 or later. tsc can peak around 3.7 GB; if the process OOMs, set NODE_OPTIONS=--max-old-space-size=8192.

npm ci
npm run build
npm test

Use npm ci in CI and for a clean checkout. npm install is fine locally. Unset ELASTIC_CLI_CONFIG_FILE before running the built CLI against a real cluster.

After dependency changes, run node scripts/generate-notice.mjs and commit NOTICE.txt if it changed. After command or flag changes, run npm run build:schema and commit docs/cli/schema.json.

Typecheck discipline

Full-repo tsc is heavy (peaks ~3.7 GB) and a top source of wasted recovery turns. While iterating, typecheck only the changed file/module, not the whole project; fix type errors immediately rather than letting them accumulate. Run full npx tsc --noEmit once before marking a task complete. When output is long, pipe through head -30 and fix the first error first.

Per-endpoint API schemas are not generated into this repo; they ship in the published @elastic/schemas package. Do not create files under src/es/apis/ or src/kb/apis/ (those directories do not exist).

Architecture

Commands are defined via shared config structures (see factory.ts). Custom logic is only permitted for behaviors that cannot be expressed in config.

Read ARCHITECTURE.md before proposing structural changes. It documents performance-critical design decisions (factory core split, per-endpoint API files, lazy loading, manifest format) that must not be refactored unless benchmarks prove the change is neutral.

Command Authoring Requirements

All requirements below are non-negotiable and enforced at review time.

Input

• JSON via stdin and --input-file: Structured input MUST be accepted from stdin or --input-file <path>. Neither takes precedence; providing both MUST error.
• CLI flags for all input fields: Every top-level schema field MUST have a corresponding kebab-case CLI flag. When both JSON input and flags are provided, flags take precedence.
• JSON Schema input: Every command with structured input MUST declare a JSON Schema document (from @elastic/schemas, or hand-authored for commands with no upstream schema) as the single source of truth for validation and help text. input: true (untyped) MUST NOT be used in new commands.
• Validate before executing: All input MUST be validated before any handler logic or network call. Invalid input is a hard error.
• Reject unknown keys: Input with undefined keys MUST produce a validation error naming the unknown field(s). Silent stripping is not acceptable.

Output and Errors

• --json: Every command MUST emit structured JSON when --json is passed.
• --help --json: MUST output the full JSON Schema so agents can introspect valid inputs.
• Errors: All errors MUST go to stderr with a non-zero exit code. With --json, errors MUST serialize as {"error": {"code": "...", "message": "..."}}.
• Exit codes: Process exit is 0 on success and 1 on every failure. Codes 2 (usage), 3 (validation), 4 (auth/config), and 5 (network) are reserved; do not emit them yet. See elastic help exit-codes.
• error.code catalog (frozen; do not invent new strings without a changelog note):
  • missing_config — no config file or no service connection
  • config_invalid — config file failed to parse or validate
  • auth_required — HTTP 401. Probe with elastic status --json
  • not_found — HTTP 404. Probe with elastic status --json
  • transport_error — other HTTP/client errors. Probe with elastic status --json
  • connection_error — TCP/TLS failure. Probe with elastic status --json
  • input_error — handler rejected the request before sending
  • input_validation_failed — flags or JSON input failed the schema
  • confirmation_required — destructive command needs --yes
  • command_blocked — command policy blocked this invocation
  • kibana_api_error / cloud_api_error — product HTTP errors
  • invalid_request / credential_policy_error — Cloud request/policy
  • output_template_error — --output-template failed to render
  • missing_source / missing_name — extension add/remove args
• After auth_required or transport_error, run elastic status --json.

Mutations and Side Effects

• --dry-run: Every command that mutates state or makes a network call MUST support --dry-run. In dry-run mode: validate all inputs, print the resolved request payload, exit 0 without executing.

Credentials and Configuration

• No credentials as CLI flags: API keys, passwords, and tokens belong only in the config file or environment variables. Never as CLI flags.
• Named contexts: Connection info is managed via named contexts in the YAML config (kubectl-style). --use-context <name> MAY override for a single invocation; context fields MUST NOT be duplicated as first-class flags.

Transport Abstraction

• Hide routing metadata: x-found-in: path | query | body is an implementation detail. It MUST NOT appear in help text, schema output, or error messages.
• Validate path parameter coverage: If a schema field has x-found-in: "path" but the URL template has no matching placeholder, the system MUST fail fast at registration time.

Cross-Platform Compatibility

• Paths: Use path.join() / path.resolve(). Hard-coded / or \\ separators are forbidden.
• Config directories: Resolve using os.homedir() and process.env.APPDATA (or OS equivalents). No Unix-only hard-coded paths.
• Platform guards: Signal handling, TTY detection, and ANSI escape codes MUST be guarded behind capability checks.
• CI: The full test suite MUST pass on Windows, Linux, and macOS before merge.

Code Patterns & Conventions

SPDX Header

All code files MUST start with:

/*
 * Copyright Elasticsearch B.V. and contributors
 * SPDX-License-Identifier: Apache-2.0
 */

Note the single-asterisk /* opener — scripts/check-spdx.mjs rejects the JSDoc-style /** form.

Standards

• Docstrings: All exported symbols in reusable utilities MUST have complete doc comments.
• Comments: Explain WHY, not WHAT. Do not restate code in prose.
• Probes do not throw: functions that check a remote service for a caller that can proceed without the answer return { ok, ... } results instead of throwing. See ARCHITECTURE.md → "Probes Do Not Throw".
• No new _test* exports: do not add module-level test setters. Inject test doubles through a deps parameter or constructor option, as src/es/handler.ts (EsHandlerDeps) and src/docs/* (deps.stderr) do. Read process.platform at call time instead of copying it into a module-scope variable.
• Naming: camelCase for functions/variables, PascalCase for types/interfaces.
• YAML config keys: Always snake_case (e.g. api_key, current_context). Never camelCase or kebab-case.
• Files: Trailing newline required, no trailing whitespace.

Dependencies

When solving a problem outside this tool's core domain (Elastic API interaction), check if an installed dependency solves it before writing custom code. For example: prefer commander over process.argv for argument parsing. Apply a manual solution only if the dependency cannot help.

TypeScript Configuration

Strict flags: noUncheckedIndexedAccess, exactOptionalPropertyTypes, strict, verbatimModuleSyntax, isolatedModules, noUncheckedSideEffectImports, moduleDetection: force. Source maps and declaration maps enabled.

TDD Discipline

Follow this cycle autonomously for every change:

1. Write a failing test (RED)
2. Confirm it fails for the right reason
3. Write minimum code to pass (GREEN)
4. Refactor under green (REFACTOR)

Task is complete when npm test exits 0 and lint passes.

Pre-commit Linting

MegaLinter runs automatically on staged files when pre-commit install has been run. It catches TypeScript lint errors, YAML issues, unpinned GitHub Actions, accidental secrets, and copy-paste duplication. Docker must be running.

If the pre-commit hook fails, read the error output, fix flagged files, and retry. Do NOT use --no-verify.

To run MegaLinter manually: npm run test:megalinter (requires Docker).

Thorough Testing

After every implementation change:

1. Run npm test. Never skip.
2. Write a regression test for every bug fix.
3. Test adversarial inputs for any function embedding user strings in URLs, paths, or queries (../, ?#, empty strings, special characters).
4. Assert on full RequestInit configuration (e.g. redirect, credentials), not just response output.
5. Do not copy test patterns from existing code without auditing for gaps.
6. After unit tests pass, build (npm run build) and verify end-to-end manually.
7. Check for lint errors on all modified files.
8. Never weaken assertions, skip cases, or change expected values to get green. The test is the spec; fix the code.
9. Never add special cases or dead code in production code solely to satisfy a bad test. Fix the test instead.
10. Test all code paths: missing input, empty input, null, wrong types, boundary values, HTTP error codes, redirects, timeouts.
11. Run the code you wrote. For a CLI command, run it. For a request builder, trace the actual HTTP request.
12. In CI, all tests must pass on all JS runtime versions declared in the package.json engines field. If you have access mise, nvm, asdf, etc. to validate your changes by running tests on multiple versions of Node, do that during your final pass. If you do not, make a best-effort attempt to support them all using whatever knowledge you have access to about them.

Security Checklist

When constructing URLs, sending credentials, or making HTTP requests:

• Encode path parameters: Use encodeURIComponent() or an equivalent wrapper. Never bare String(value).
• Validate URL schemes: Reject anything other than http:// or https://. Warn on plaintext http:// for non-localhost targets.
• Audit before copying: If replicating URL-building logic from another file, check it for encoding and validation gaps.
• Set redirect: 'error' (or 'manual') on every fetch call that sends credentials. Never rely on the default 'follow'.
• Assert on RequestInit in tests: verify redirect, method, and headers.
• Adversarial input test: Add at least one test with malicious input for any function that embeds user strings in URLs or paths.

Generic Abstractions: Lessons Learned

1. Enumerate all variants upfront. A now-retired field-unwrapping helper only handled two shapes out of the many the input source could produce, and the rest fell through silently. The current equivalent is @elastic/schemas' use of JSON Schema composition ($ref, anyOf, oneOf, allOf, root-level $ref): inspect the full set of shapes the upstream schemas can produce (see src/lib/json-schema-refs.ts, src/kb/apis.ts's flattenComposition) and add explicit handling or a loud failure for unrecognized ones.

   The same applies to upstream x- annotations. The current set is x-found-in, x-body-root, x-api, x-method, x-path, x-urls, x-body-format, x-destructive, x-response-type, x-availability, x-deprecated. New ones appear without warning; classify each as routing (must be stripped from output) or user-facing (may be surfaced) rather than matching on the x- prefix. See ARCHITECTURE.md → "Schema Composition and x- Metadata".

2. Fail loudly on unrecognized input. A catch-all fallback once silently returned a garbage field type instead of erroring. src/lib/json-schema-refs.ts's resolveRootRef() follows the fix: it throws when a root ref doesn't resolve to an object schema with properties, rather than letting a silently-empty schema reach downstream flag derivation.
3. Test with real schemas from the actual source. Hand-crafted toy schemas miss shapes that only appear in real @elastic/schemas output (e.g. Kibana's per-rule-type allOf/oneOf bodies).
4. Generic request builders need extension points for endpoint-specific semantics. The bulk API needs NDJSON; the index API needs body promotion. Add explicit extension points (bodyFormat, and the upstream x-body-root annotation) rather than special-casing later.
5. Diagnose common mistakes in user-facing errors. Map known error patterns (TLS mismatch, auth failure, DNS) to actionable hints instead of propagating raw messages.
6. Treat upstream schema output as untrusted input. Validate assumptions about which JSON Schema shapes appear with tests that use the actual @elastic/schemas definitions, not synthetic ones.
7. Read external type definitions before setting properties. TransportRequestParams has bulkBody for NDJSON, not body + custom headers. JavaScript silently ignores extra properties; only tsc --noEmit or CI catches this. Run npx tsc --noEmit before pushing.
8. Guard clauses that discard data are dangerous. A guard that returned early whenever a Cloud POST body wasn't a recognized shape once silently dropped all stdin/--input-file input for those commands. When a guard returns early with no data, ask what happens to the caller's input. Prefer forwarding with passthrough semantics over silent discard.
9. Trace the full data flow for every mode combination. --json broke cat APIs because the handler returned raw text and the factory blindly called JSON.stringify(). When two layers cooperate (handler + formatter, request builder + transport), enumerate all mode combinations and verify each.
10. Review upstream command names for UX. Names sourced directly from @elastic/schemas (e.g. list-deployments) are precise but verbose. Add short aliases where unambiguous so users can discover commands intuitively.

Agent repair loop

On a GitHub Actions CI failure or a Buildkite functional failure, take the first failed job only. Patch the same-repo PR if the fix is in this repo. Comment and stop if the cause is the spec, generator, CI infra, Cloud QA, or a flake.

/bad (optional reason) marks a bad bot review. Owners and maintainers only. Conversation comment or inline reply on the finding. A model turns the finding and your note into one "Do not re-flag" line on .github/skills/ai-review-memory.md (new bot PR, or a commit if that PR already exists).

auto-loop is the switch that lets the bot commit. Without it the loops still triage (sticky comment). With it they apply in-repo fixes for the first CI or Buildkite failure, and for bot reviews (AI review, CodeQL). Adding the label re-dispatches both loops. The label job is issues: labeled so it always runs from the default branch. Apply writes repair-loop-patch (not .repair-loop; upload-artifact drops hidden paths). Comment /stop to halt and pin skip-auto-loop. skip-ai-review silences the review bot.

On a buildkite/elastic-cli/pr failure, GitHub Actions posts <!-- bk-repair-loop -->. Apply uses an ephemeral GitHub App token from Vault so the push retriggers CI. Buildkite does not get a GitHub token.

Do not edit .github/workflows/.

Review comments from a trusted reviewer (OWNER, MEMBER, CodeQL, or the AI review bot) are another pass of the same loop. Do not repeat an approach a reviewer already rejected.

Durable lessons from a run belong in this file.

Conventional Commits

All commit messages and PR titles MUST follow Conventional Commits v1.0.0. PR titles are validated in CI.

Format

<type>(<optional scope>)<optional !>: <description>

<optional body>

<optional footers>

Types

Scope

Use a scope when the change is confined to one area. Omit for cross-cutting changes.

Common scopes: cli, config, auth, cloud, es, serverless.

feat(cli): add --format flag to search command
fix(config): handle missing YAML key gracefully
ci: add PR title validation step

Breaking Changes

Indicate with ! before the colon, a BREAKING CHANGE footer, or both:

feat(cli)!: rename --output to --format

BREAKING CHANGE: --output is removed; use --format instead.

Common Mistakes

• Types and scopes must be lowercase: feat, not Feat; feat(cli), not feat(CLI).
• No space before the colon: feat:, not feat :.
• Space after the colon: feat: add, not feat:add.
• Description must not be capitalized or end with a period.
• Empty scope parentheses are invalid: omit them entirely.
• Every commit must have a type prefix.