adamant-cosmos-suite-results
Skill4 starsChanged 3 months agoApache-2.0View on GitHub
Run a COSMOS test suite headless from automation/CI and gate on pass/fail. Use when launching a suite via openc3cli script run (or the Script Runner REST API), streaming and parsing its results, and gating a pipeline on the outcome. Covers the version-stable stream method and the COSMOS 6.x (MinIO) vs 7.x (versitygw) object-store rename.
- Reads credentials
- Sends data out
In this repository27 other files it gives its agents.
- AGENTS.mdAGENTS.md
- CLAUDE.mdCLAUDE.md
- adamant-assembly-devSkill
- adamant-build-systemSkill
- adamant-code-reviewSkill
- adamant-component-devSkill
- adamant-cosmos-integrationSkill
- adamant-cosmos-testingSkill
- adamant-cosmos-tool-creationSkill
- adamant-debuggingSkill
- adamant-formal-verificationSkill
- adamant-framework-componentsSkill
- adamant-framework-internalsSkill
- adamant-generator-devSkill
- adamant-project-setupSkill
- adamant-regression-suiteSkill
- adamant-skill-campaignSkill
- adamant-skill-creationSkill
- adamant-skill-selectorSkill
- adamant-styleSkill
- adamant-subassembliesSkill
- adamant-testingSkill
- adamant-toolsSkill
- adamant-type-systemSkill
- high-assurance-designSkill
- knowledge-acquisitionSkill
- task-planningSkill
The file
We split this skill into the rules it gives an agent
4 rules. About 0 are common, 1 is distinctive, and 3 are very specific to this project.
≈0
Common · estimate
Many files like this have a rule of this kind.
≈1
Distinctive · estimate
This author's own choices. Worth a look.
3
Very specific
Tied to their project's names, files or tools. Probably skip.
- Common
- Distinctive
- Very specific
- Very specific: line 8
Run a COSMOS test suite without the GUI and gate a pipeline on its result.
- Very specific: line 32
Run the suite via the CLI inside openc3-cosmos-cmd-tlm-api, unbuffered, and pipe stdout to a parser.
- Distinctive: line 91
Prefer Method A's exit-code gate over retrieving and parsing an object-store log.
- Very specific: line 130
Run inside openc3-cosmos-cmd-tlm-api.
≈ “Common” is an estimate: at least 1 in 5 of the 350,495 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.
Files
COSMOS Suite Execution + Result Verification
Run a COSMOS test suite without the GUI and gate a pipeline on its result. The
version-stable path runs the suite via openc3cli script run inside the
cmd-tlm-api container and reads the result from the streamed output plus the
process exit code -- no object-store retrieval. (Earlier setups pulled the suite
log from the MinIO bucket; COSMOS 7.x replaced MinIO with versitygw, so the stream
method is both simpler and store-independent.)
Architecture
Driver (bash/python)
|
|--> openc3cli script run (inside openc3-cosmos-cmd-tlm-api, $stdout.sync=true)
| | suite emits result/report/summary lines on stdout
| v
|--> stream parser (tee + classify) --> filtered log + live progress
| |
| v
+--> exit code: 0 = all pass, non-zero = a failure <-- the authoritative gate
(on failure: docker compose logs <service> for debugging)Method A -- openc3cli script run + stream (preferred, version-stable)
Run the suite via the CLI inside openc3-cosmos-cmd-tlm-api, unbuffered, and pipe
stdout to a parser. The pipeline's exit status is the authoritative gate.
docker compose -f <compose> exec -T openc3-cosmos-cmd-tlm-api \
ruby -e '$stdout.sync=true; load "/openc3/bin/openc3cli"' \
-- script run "<SCRIPT_PATH>" \
--suite "<SuiteClass>" \
--method start \
--options "continueAfterError" \
--scope DEFAULT \
2>&1 | tee suite_results.log
STATUS=${PIPESTATUS[0]} # 0 = all tests passed; non-zero = a failure occurred$stdout.sync=truemakes the suite output unbuffered, so a parser sees results live.--method startruns the suite's start entrypoint;--optionspasses suite-runner flags (continueAfterErrorruns every test even after a failure;manualskips interactive prompts).- The suite emits structured lines on stdout (per-test results, report lines, a final summary). Gate on the exit code, not on grep counts -- the run exits non-zero iff any test failed.
- A richer parser can classify the stream into started / result / summary events for live progress and a filtered log; the exit code stays the gate. Keep the parser project-specific.
Stop a running suite
docker compose -f <compose> exec -T openc3-cosmos-cmd-tlm-api \
ruby /openc3/bin/openc3cli script running --scope DEFAULT # list running ids (col 1)
docker compose -f <compose> exec -T openc3-cosmos-cmd-tlm-api \
ruby /openc3/bin/openc3cli script stop <id> --scope DEFAULTDebugging: per-service container logs (store-independent)
for s in $(docker compose -f <compose> config --services); do
docker compose -f <compose> logs "$s" > "${s}_$(date +%Y%m%d-%H%M%S).log" 2>/dev/null || true
doneThis enumerates services dynamically, so it adapts to renamed/added 7.x services
(openc3-buckets, openc3-tsdb) automatically.
Method B -- Script Runner REST API (run + poll)
An alternative HTTP trigger (base http://localhost:2900, header
Authorization: openc3service). The endpoint surface is COSMOS-version-dependent --
verify against your deployment before relying on it.
BASE=http://localhost:2900 ; AUTH="Authorization: openc3service" ; SCOPE=DEFAULT
# load (expect 200) -> lock -> run
curl -s -w '%{http_code}' -H "$AUTH" "$BASE/script-api/scripts/$SCRIPT/?scope=$SCOPE" -o /dev/null
curl -s -X POST -H "$AUTH" "$BASE/script-api/scripts/$SCRIPT/lock?scope=$SCOPE" -o /dev/null
curl -s -X POST -H 'Content-Type: application/json' -H "$AUTH" \
"$BASE/script-api/scripts/$SCRIPT/run?scope=$SCOPE" \
--data-raw '{"environment":[],"suiteRunner":{"method":"start","suite":"TestSuite","options":["continueAfterError"]}}'
# poll until empty, then read completion info (incl. log file names)
curl -s -H "$AUTH" "$BASE/script-api/running-script?scope=$SCOPE" # [] == finished
curl -s -H "$AUTH" "$BASE/script-api/completed-scripts?scope=$SCOPE"Prefer Method A's exit-code gate over retrieving and parsing an object-store log.
Object store: COSMOS 6.x MinIO vs 7.x versitygw
COSMOS keeps tool logs in the openc3-logs bucket; the backing service changed at 7.0:
| 6.x | 7.x | |
| Service | openc3-minio (MinIO) | openc3-buckets (versitygw, S3-compatible) |
| Volume | openc3-bucket-v | openc3-object-v |
| Credentials | MINIO_ROOT_USER / MINIO_ROOT_PASSWORD | ROOT_ACCESS_KEY / ROOT_SECRET_KEY |
| Bucket URL host | openc3-minio:9000 | openc3-buckets:9000 |
| In-container client | mc with a preconfigured local alias | none -- use an S3 client (e.g. aws-cli) |
A bare image-tag bump to 7.x fails (openc3-minio publishes no 7.x tag). Because
versitygw ships no mc/local, any 6.x retrieval that did
docker exec <minio> mc cp local/openc3-logs/... must move to an S3 client -- or, better,
gate on Method A's stream + exit code so result verification does not depend on the store.
Suite-log path (both versions): DEFAULT/tool_logs/sr/YYYY_MM_DD/ (underscored dates). The
Script Runner log lines are timestamped:
YYYY/MM/DD HH:MM:SS.SSS | SCRIPTRUNNER | <test_name> | PASS
YYYY/MM/DD HH:MM:SS.SSS | SCRIPTRUNNER | <test_name> | FAIL | <error message>COSMOS container names (compose stack)
openc3-cosmos-cmd-tlm-api -- API server + the openc3cli entrypoint (run scripts here)
openc3-operator -- manages microservices
openc3-traefik -- reverse proxy (port 2900)
openc3-buckets -- object store (versitygw; 6.x: openc3-minio)
openc3-tsdb -- time-series telemetry store (7.x; mandatory)Resolve names dynamically: docker compose -f <compose> config --services.
Guidelines
- Gate CI on the run's exit code (Method A), not on grep counts or bucket availability.
continueAfterError-> partial results beat an early abort.- Run inside
openc3-cosmos-cmd-tlm-api; stop a run viaopenc3cli script stop <id>. docker compose logs <service>is the version-independent debug source; only collect on failure.- On 7.x, MinIO is gone (versitygw /
openc3-buckets) -- do not rely onmc/local. - Give the suite a generous timeout for large assemblies; poll/stream rather than block blindly.
Does it work?
Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.
No reviews yet
People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.
- It worked
- Partly
- It didn't
Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.
Your trail
Files you open will collect here, so you can find them again.
Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.
Now do this for your own.
Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.

