Holon Runtime Operations Skill

Summary

Use this skill to inspect Holon runtime health, diagnose agent and lifecycle failures, maintain incremental error checkpoints, produce explicitly authorized patrol reports, and prepare sanitized upstream bug reports.

This skill extends the platform-neutral ops workflow. It does not grant credentials, remediation authority, scheduled execution, direct database access, or GitHub publication authority.

Safe daemon restart contract

A restart is a destructive lifecycle operation because it terminates the current daemon cgroup and can interrupt active turns. Never run systemctl --user restart holon.service, holon daemon restart, or an equivalent synchronous stop/start from a turn executing inside that service.

Before the restart, create one durable operation id and persist its phase as planned, then atomically advance it to scheduled together with the owning turn's destructive-operation fence. Execute the restart from outside the daemon cgroup, preferably with a transient unit:

systemd-run --user --unit=holon-restart-<operation-id> --collect \
  --on-active=2s systemctl --user restart holon.service

The scheduling command must return before the current turn settles. After startup, perform only readiness/version/status verification and advance the operation to verified. If the operation id already exists, treat it as already scheduled and never issue a second stop/start. Do not use nohup, shell backgrounding, or a fixed sleep as a cgroup escape.

When To Use

• diagnosing Holon daemon, agent, WorkItem, task, wait, timer, event-ingress, provider, tool, workspace, or delivery behavior
• preparing an authorized Holon restart, upgrade, rollback, or recovery
• counting and deduplicating new Holon runtime errors
• running an explicitly configured report-only patrol
• preparing or publishing a sanitized holon-run/holon issue under the applicable bug-reporting policy

Do Not Use

• for general server or service administration unrelated to Holon
• to read task objectives, messages, prompts, transcripts, memory, or model request bodies for an activity report
• to turn a diagnosis or patrol authorization into repair authority
• to access the runtime database by default or depend on a private schema
• to publish security, privacy, credential, personal-data, or corruption findings automatically

Authority Gates

Treat these as separate authorizations:

1. read-only runtime diagnosis
2. maintenance or remediation
3. scheduled patrol
4. anomaly notification
5. direct runtime-database read
6. direct runtime-database write
7. GitHub duplicate search
8. issue draft creation
9. issue publication

Record the scope, source, expiry, and revocation conditions of standing authorizations. If a gate is absent or ambiguous, stop at the last authorized step and report what is needed next.

Source Priority

Collect evidence in this order:

1. native runtime tools for their declared responsibility
2. declared machine-readable holon CLI commands
3. authoritative service or deployment logs
4. deployment configuration and version metadata
5. separately authorized read-only runtime-database queries as a final deep-diagnostic fallback

Use holon commands to discover the current CLI contract and holon context to inspect caller provenance. Do not manually construct caller-context environment variables and do not use recursive holon run or holon prompt as a control-plane substitute.

Direct database writes are not a diagnostic technique. They are H3 recovery actions requiring exact approval, a snapshot, a bounded mutation, rollback, and verification.

Live Runtime Database Diagnosis

Keep the runtime database readable under its existing deployment permissions, but do not treat direct access to a live runtime.sqlite as the default diagnostic path. Prefer native runtime tools and APIs.

On Linux, inspect runtime_db_protection in runtime status or readiness:

• starting is limited to startup and migration before the guard is published
• protected means the process-lifetime OFD sidecar guard is active
• quarantined means sidecar integrity failed and runtime DB access is permanently fail-stopped for the process
• unsupported means the platform does not provide the Linux OFD guarantee

DB-dependent requests in quarantine return 503 Service Unavailable with error code runtime_db_quarantined and Retry-After: 1. Do not loop on that response or assume the runtime will reconstruct the sidecars automatically.

If direct structural inspection is necessary:

• require the direct runtime-database read authority gate
• inspect a verified offline backup unless separate maintenance authority has been granted to stop the Holon service
• if maintenance authority has not been granted and no verified backup exists, request authorization instead of stopping the service
• an unmanaged SQLite open/query/close must not remove live sidecars while protection is active, but it remains a last-resort diagnostic path and unmanaged writes are unsupported
• never delete, rename, replace, restore, or copy over live -wal or -shm sidecars

If Holon reports quarantined, a deleted-open WAL/SHM file, or an inode divergence:

1. do not delete files or immediately restart over the evidence
2. record the protection evidence, database path, sidecar path, process ID, FD, device, and inode
3. preserve the main database and both sidecars before recovery
4. perform recovery or restart only under explicit maintenance authority, using the preserved files and an offline verification step

These are operational safety rules, not access-control requirements. Do not change directory ownership or permissions unless the operator separately authorizes that deployment change.

Installation Inventory

Keep one record per Holon installation:

work/inventory/installations/<installation-id>/info.yaml

Recommended fields:

schema: holon.ops.installation.v1
id: local-dev
environment: development
owner: operator
deployment_mode: local-binary
version:
  reported: 0.1.0
  commit: "<git-sha-or-null>"
runtime:
  endpoint_ref: local-control-plane
  service_ref: null
authoritative_sources:
  - kind: runtime
    ref: native-tools
  - kind: deployment
    ref: "<safe-reference>"
data_boundaries:
  activity_reports: runtime-metadata-only
  database_access: disabled
notes: null

Do not place credentials, callback URLs, secret values, private keys, or raw payloads in inventory.

Standard Diagnosis

1. Identify — installation, environment, version, deployment mode, impact, reporting window, and authority.
2. Snapshot — current agent/runtime state and deployment state using the highest-priority available sources.
3. Bound — choose the smallest relevant agents, WorkItems, tasks, errors, components, and time range.
4. Collect — lifecycle metadata, statuses, timestamps, exit classes, redacted errors, and authoritative log references.
5. Correlate — order evidence by time and explicit state transitions.
6. Classify — expected behavior, configuration, dependency, deployment, suspected Holon defect, security/privacy, or unknown.
7. Report — facts, inference, confidence, impact, workaround, missing evidence, and recommended next action.
8. Change only if authorized — use the ops preflight, execution, rollback, operation-record, and verification workflow.

Do not treat queued, yielded, blocked, waiting, and current WorkItems as interchangeable. Use the runtime's declared lifecycle views rather than reconstructing scheduler state from incidental logs.

Incremental Error Analysis

Maintain one checkpoint per installation and source:

work/checkpoints/<installation-id>-<source-id>.json

Example:

{
  "schema": "holon.ops.checkpoint.v1",
  "source": "runtime-events",
  "cursor_kind": "timestamp_and_id",
  "cursor": {
    "timestamp": "2026-09-06T00:00:00Z",
    "id": "event-redacted"
  },
  "last_successful_run": "2026-09-06T00:10:00Z"
}

Rules:

• use a deterministic boundary such as (timestamp, stable_id) when possible
• query with overlap, then deduplicate, so equal timestamps and late records are not lost
• fingerprint from sanitized component, error class, normalized message shape, relevant state transition, and version; never hash a secret-bearing raw body and publish the hash as if it were safe
• distinguish occurrence count, affected agents or operations, first seen, last seen, and new-versus-known status
• state explicitly when sources are missing, truncated, reset, or inconsistent
• write the new checkpoint only after collection, analysis, report persistence, and required delivery all succeed
• on failure, retain the previous checkpoint and record the failed run

Patrol Configuration

Patrol is disabled until the operator approves and persists a policy:

work/policies/patrol.yaml

Suggested contract:

schema: holon.ops.patrol.v1
enabled: false
mode: report-only
timezone: Etc/UTC
schedule:
  kind: daily
  at: "09:00"
scope:
  installations: []
  environments: []
  agents: []
  exclude_agents: []
window:
  kind: since-last-success
checks: []
data_policy: runtime-metadata-only
notifications:
  report_destination: null
  anomaly_destination: null
  silence_windows: []
review:
  expires_at: null
  next_review_at: null

Confirm the IANA timezone, daylight-saving behavior, scope, exclusions, reporting window, checks, timeout, concurrency, destinations, retention, checkpoint initialization, review date, and revocation conditions before scheduling.

An enabled patrol remains report-only unless a separate named remediation authorization exists. Never infer automatic repair, restart, cancellation, upgrade, issue submission, or database access from the patrol policy.

Metadata-Only Activity Reports

Approved report data may include:

• installation and runtime version
• agent ID, lifecycle state, creation time, last approved activity timestamp, and state transition counts
• aggregate WorkItem, task, wait, timer, and delivery counts or outcomes
• newly observed error fingerprints and trends
• recorded deployments, restarts, maintenance actions, findings, and incidents

Excluded data:

• WorkItem objective or plan content
• task command or prompt content
• operator, external, or model messages
• prompts, transcripts, memory, briefs, and model request or response bodies
• tool input/output payload bodies unless separately needed for a diagnosis and excluded from the report
• environment values, credentials, callback capabilities, and secret-manager material

Define “active” in the policy using approved metadata signals, for example a lifecycle transition, task start/completion, or runtime-recorded activity timestamp inside the report window. Do not infer activity from content access.

Recommended report outline:

# Holon daily operations report
## Executive summary
## Runtime and daemon health
## Agent current-state distribution
## Agent activity in reporting window
## WorkItem, task, wait, timer, and delivery lifecycle health
## New errors and trend versus previous window
## Open findings and incidents
## Deployments, restarts, and maintenance actions
## Recommended operator actions
## Data coverage and limitations

Persist reports under work/reports/daily/YYYY-MM-DD.md. Include the exact timezone, UTC bounds, policy revision, sources, excluded sources, and checkpoint outcome.

Finding and Incident Records

Use findings for bounded observations that need tracking:

work/findings/FIND-<timestamp>-<slug>.md

Include installation, time window, affected component and version, sanitized evidence, fingerprint, occurrence count, impact, confidence, classification, workaround, missing evidence, and recommended action.

Use incidents for active or materially impactful operational events:

work/incidents/INC-<timestamp>-<slug>.md

Follow the ops incident and operation-record rules. Link findings, reports, authorized operations, and issue drafts instead of duplicating their full contents.

Bug Escalation Policy

Persist the policy at:

work/policies/bug-reporting.yaml

Suggested contract:

schema: holon.ops.bug-reporting.v1
mode: disabled
repository: holon-run/holon
allowed_categories: []
rate_limit:
  max_issues: 0
  per_hours: 24
expires_at: null
require_duplicate_search: true
forbidden_auto_submit:
  - security
  - privacy
  - credential
  - personal-data
  - data-loss

Modes:

• disabled: local findings only
• draft-only: create a sanitized draft and wait for operator review
• scoped-submit: publish only while an explicit repository-, category-, duration-, frequency-, and revocation-bounded policy is active

First enablement must be draft-only. This skill defines scoped-submit for a possible future authorization but grants none.

Sanitized Issue Workflow

1. Confirm the bug-reporting mode and publication authority.
2. Establish affected version, environment class, frequency, impact, and a minimal reproducer or strong evidence chain.
3. When GitHub read access is authorized, search open and closed holon-run/holon issues for duplicates.
4. Classify security, privacy, credential, personal-data, corruption, or data-loss findings for private escalation; do not auto-publish them.
5. Replace identifying or secret-bearing values with stable placeholders.
6. Write the issue body to work/issue-drafts/<timestamp>-<slug>.md.
7. Review the rendered body and every attachment for leakage.
8. In draft-only, stop and request operator approval.
9. In scoped-submit, revalidate repository, category, expiry, rate limit, revocation, duplicate result, and forbidden categories immediately before publication.
10. Publish using ghx file-based payload guidance and record the resulting issue reference without copying authentication data.

Sanitize at least:

• installation, agent, WorkItem, task, event, request, and provider IDs
• usernames, home directories, absolute paths, hostnames, IP addresses, and internal repository or branch names
• callback URLs and capability-bearing query strings
• tokens, cookies, headers, environment values, secrets, and key material
• message, prompt, transcript, memory, objective, brief, and model payload content

Do not assume hashing makes sensitive data publishable. Prefer descriptive placeholders such as <agent-id>, <local-path>, and <request-id>.

Recommended issue outline:

## Summary
## Environment and Holon version
## Minimal reproduction
## Expected behavior
## Actual behavior
## Sanitized evidence
## Impact and frequency
## Workaround
## Duplicate search
## Additional context

Completion Criteria

A diagnosis or patrol run is complete only when:

• scope, authorization, sources, and data limitations are recorded
• observations and inferences are separated
• errors are deduplicated and checkpoint handling is explicit
• the report or finding is persisted and delivered as configured
• no excluded content or secret-bearing material was retained
• any change has an ops operation record, rollback result, and verification
• issue publication either stopped at a draft or recorded the exact applicable publication authority

If any criterion is unmet, report partial completion and do not advance the checkpoint.