Schema Coverage (ec_* entity vs acceptance tests)

Goal

Given one Terraform entity (ec_* resource or data source), compare its Plugin Framework schema to what the acceptance tests configure and assert, then emit a ranked (high-risk-first) coverage-gap report.

This skill is read-only analysis. It does not change provider code, write tests, or run them.

Read reference.md before extracting schema or matching tests. Follow it for Framework shapes, acc-config discovery, excluded steps, import, path normalization, and ranking tags.

Safety

• Never set TF_ACC or run make testacc. Acc hits the real, paid Elastic Cloud API.
• Do not execute tests to "see what is covered". Coverage is inferred from source (ec/acc/ Go + testdata, plus colocated unit tests as an annotation only).
• For the ec_deployment resource, analyze one scope per run unless the user asks for the whole resource:
  • root — top-level attributes on DeploymentSchema() (name, region, version, traffic_filter, credentials, …)
  • one component subtree: elasticsearch, kibana, apm, integrations_server, or enterprise_search

Inputs (infer if not provided)

Primary contract: one entity. Do not enumerate the whole provider unless the user asked for that.

• Entity name (e.g. ec_deployment_traffic_filter, ec_stack) and kind (resource or data_source)
• Or a schema file / package path
• Or an open acc test — infer from resName := "ec_..." / resource.ParallelTest

ec_deployment is registered as both a resource and a data source (ec/provider.go). They have different packages, schemas, and state paths. If the user names ec_deployment without a kind, ask. Never merge their acc files into one report.

If the user did not name an entity, ask. Fallback only: locate TypeNames via ec/provider.go Resources() / DataSources() (see reference for the suffix-grep recipe), then ask which one (and which kind when the name is on both lists).

Workflow

1) Locate and parse the schema

Find the owning package (reference: suffix-grep response.TypeName). Parse the current Framework schema (resp.Schema = … or helper such as v2.DeploymentSchema()). Skip */v1/ packages — those are state upgraders.

For each attribute/block, record:

• Path (see reference path normalization — ec_deployment resource v2 is elasticsearch.hot.size; the data source uses indexed paths like elasticsearch.0.…. Do not mix them.)
• Required / Optional / Computed
• Type (String, Bool, Int64, List, Set, Map, SingleNested, …)
• Validators, plan modifiers, defaults
• Nesting depth

Exclude timeouts unless it is declared in that entity's Attributes or Blocks map.

2) Locate acceptance tests and collect usage

Acc lives in ec/acc/, not next to the entity package. Match the type name and kind in test files, testdata HCL, and helpers (ec_deployment_traffic_filter appears in acc/docs; schema packages set request.ProviderTypeName + "_suffix"). For ec_deployment, keep resource tests (deployment_*.go) separate from data-source tests (datasource_deployment_*.go).

Follow both Config: and Check: helpers to their definitions (they may live in another file in ec/acc/). Dominant config pattern is Config: + Go helper + os.ReadFile + fmt.Sprintf; ConfigDirectory is rare.

Collect two signals from included steps only:

• Configured: set in that step's HCL (including nested blocks). Import steps have no Config of their own — they inherit the previous included step's config.
• Asserted: TestCheckResourceAttr, TestCheckResourceAttrPair, TestCheckResourceAttrWith, TestCheckResourceAttrSet, TestCheckNoResourceAttr, TestMatchResourceAttr, type-set helpers, "block.#" / "tags.%" — including those inside Check: helpers.

Also record import (see reference). ImportStateVerify: true is not "every schema path is asserted". It only supports paths that were non-null in the imported state (i.e. configured in the inherited config, or computed-and-present). It never lifts a never-configured attribute out of no-coverage.

Do not count ExpectError, PlanOnly: true, ExternalProviders (old provider), or an unconditional test-level t.Skip("reason"). The requiresAPIConn / TF_ACC != "1" skip is the acc gate, not an exclusion.

Attributes that appear only in excluded steps get the reason excluded-step-only, not "covered".

3) Build a coverage matrix

For each schema path:

• Configured? Asserted? Assertion quality (value-specific / set-only / absence)
• Value diversity across included steps (do not flag env-derived region / version / template / instance-configuration IDs as single-value — see reference)
• Optional-unset, empty-collection, update coverage (update is N/A for data sources)
• Import-verify / import-ignore (and any ImportPlanChecks next to an ignore)
• Unit-test annotation (offline vs needs-live-api) — never reduces the acc coverage class

4) Classify and rank

No coverage: never configured in an included step and never referenced by an included check (TestCheckResourceAttr, …Pair, …With, …Set, TestCheckNoResourceAttr, …Match…, type-set, #/%). Import-verify of a null/absent path does not count as a check. An omit + TestCheckNoResourceAttr is not no coverage (that is the optional-unset case; it may still be poor if the attribute is never set to a value).

Poor coverage: configured and/or asserted, but weak (configured-never-asserted, set-only where a value is deterministic, single value only, optional never unset, collection never empty only if empty is schema-valid, no in-place update on a resource except RequiresReplace fields, import-ignored).

Rank high-risk first. Do not assign numeric scores. Sort using the tags in the reference. Computed-only ids/endpoints sort last unless they have validators or plan modifiers.

5) Zero-acc entities

If nothing under ec/acc/ references the type name, say so once. Treat the entity as wholly uncovered. Rank only entry-point gaps worth a first test (required/writable top-level attributes and the main nested block), not every leaf.

6) Produce the report

Use the templates below. Markdown is the 4.4-shaped deliverable (stable headings). Append one fenced JSON block that mirrors the same fields — not a separate versioned API. No generated_at, no risk_score. Put close_via on every gap (none and poor).

Report template (markdown)

## Schema coverage report: <entity>

### Entity
- **Name**: <ec_…>
- **Kind**: resource | data_source   (required; `ec_deployment` is both)
- **Implementation directory**: <path from repo root>
- **Schema**: <file(s) or function(s)>
- **Acceptance tests**: <file(s), or "none">

### Attributes with no coverage
- `<attr_path>`: <Required/Optional/Computed>, <type>. Tags: <reason tags>. **Close via**: `offline` | `needs-live-api`. **Gap**: …

### Attributes with poor coverage
- `<attr_path>`: <schema flags/type>. Tags: <reason tags>
  - **Observed**: <configured/asserted/import, example values>
  - **Gaps**: <set-only | single value | no unset | no empty collection | no update | import-ignored | excluded-step-only | …>
  - **Close via**: `offline` | `needs-live-api`
  - **Suggested improvement**: <concrete step or assertion>

### Prioritized top 5
1. `<attr_path>` — <why this is first>
2. …

### Concrete test additions
- <smallest diffs that would close the top gaps; name TestAcc vs unit-test file>

If there are no acc files, keep the same headings; under no-coverage state that the entity has zero acceptance coverage, and keep top 5 / concrete additions to a first-test sketch.

Report template (JSON, advisory)

Append after the markdown, same analysis:

{
  "entity": {
    "name": "ec_example",
    "kind": "resource",
    "implementation_dir": "ec/ecresource/example",
    "schema_files": ["ec/ecresource/example/schema.go"],
    "acc_files": ["ec/acc/example_test.go"]
  },
  "gaps": [
    {
      "path": "name",
      "required": true,
      "optional": false,
      "computed": false,
      "type": "String",
      "coverage": "none",
      "reasons": ["no-coverage", "required", "writable"],
      "suggestion": "…",
      "close_via": "needs-live-api"
    }
  ],
  "top_5": ["name"]
}

close_via is offline or needs-live-api on every gap. coverage is none or poor. reasons uses the closed tag set in the reference.

Rules of thumb

• Prefer value-specific assertions over “is set”.
• Optional: at least one included step that omits the attribute, plus absence/default assertion.
• Collections: empty (or omitted) case + .# / .% assertion only if empty is valid (no SizeAtLeast(1) / equivalent). ec_deployment_traffic_filter.rule cannot be empty — do not suggest that case.
• Resources: at least one multi-step in-place update with a post-update value assertion. Skip no-update when the attribute has RequiresReplace(). Data sources: skip update.
• Computed-only: set-only is acceptable when the value is non-deterministic.
• Configured-but-never-asserted is poor, not good.
• Unit tests never mark an acc gap as covered.
• Suggestions that need live CRUD are acc stubs for a human/Buildkite; never instruct anyone to run TF_ACC from this skill.