OpenCRE Agent Instructions

Cursor agents working in this repo must follow the rules in .cursor/rules/.

Quick start

1. Requirements gate — If goal, success criteria, @ file refs, or constraints are missing, stop and ask (requirements-gate.mdc).
2. Plan first — Non-trivial or multi-file work requires a plan and user approval before coding (plan-first-workflow.mdc, multi-agent-workflow.mdc).
3. Verify — After code changes, run checks and iterate until green (verifiable-goals.mdc):
   • make lint
   • make mypy
   • make test
   • make frontend (if frontend touched)
4. Review — Substantive work needs independent judge/subagent review (multi-agent-workflow.mdc).
5. Production gap analysis — On Heroku/opencreorg, gap analysis is cache-only: serve precomputed rows from Postgres; do not compute GA on production (cache miss → 404).

Operational scripts

Prefer existing Makefile targets and scripts/ helpers over ad-hoc Docker, GA, import, or prod-DB setup. Read script headers or --help for env vars and flags; do not reimplement their logic.

Local Docker services

Use Makefile targets — do not hand-roll docker run:

make docker-neo4j      # Neo4j (7474/7687)
make docker-redis      # Redis Stack (6379/8001)
make docker-postgres   # local Postgres+pgvector (5432, cre/password)
make start-containers  # neo4j + redis only
make migrate-upgrade   # after postgres is up

Reset volumes: make docker-neo4j-rm / make docker-redis-rm / make docker-postgres-rm. Local Postgres is pgvector/pgvector:pg16 by default (POSTGRES_IMAGE override). Plain postgres images are not supported for app DB use — embedding_vec / chat / Librarian need the vector extension.

Imports and Neo4j populate

make import-all        # full standards import via scripts/import-all.sh
make import-projects   # skip core CRE import (CRE_SKIP_IMPORT_CORE=1)
make import-neo4j      # populate Neo4j from cache

scripts/import-all.sh handles parallel importers, SQLite export, and verification. For embeddings-only SQLite → Postgres push, use scripts/sync_embeddings_table.py (see its docstring).

Gap analysis (local compute, prod verify)

Local backfill (starts containers, workers, migrations — see scripts/backfill_gap_analysis.sh):

make backfill-gap-analysis              # parallel workers + --ga_backfill_missing
make backfill-gap-analysis-sync         # Neo4j populate + backfill without queue
make sync-gap-analysis-table-local      # upsert material rows sqlite → local Postgres

Prod / staging checks (HTTP only — never compute GA on opencreorg):

make verify-ga-complete-prod    # scripts/verify_ga_completeness.py
make monitor-ga-health-prod     # scripts/monitor_ga_health.py (503 / empty result alerts)
make monitor-chatbot-health-prod # scripts/monitor_chatbot_health.py (SPA + /completion 401)
make verify-ga-parity-local     # scripts/verify_ga_postgres_neo_parity.py

Table sync between databases: scripts/sync_gap_analysis_table.py, scripts/sync_embeddings_table.py.

Production DB (Heroku opencreorg)

Use scripts/db/* — never raw destructive psql against prod without these wrappers. All flows capture and wait for a fresh Heroku backup first (scripts/db/common.sh).

scripts/db/backup-opencreorg.sh                                    # backup only
scripts/db/surgery-opencreorg.sh --sql-file path/to/change.sql     # targeted SQL
scripts/db/surgery-opencreorg.sh --sql-file … --destructive        # DELETE/DROP/TRUNCATE
scripts/db/sync-local-to-opencreorg.sh [--table node]…             # local → prod sync

Destructive surgery requires CONFIRM_DESTRUCTIVE=I_UNDERSTAND_OPENCREORG_PROD_DB_DESTRUCTIVE_ACTION (exact phrase). Override app: APP_NAME=opencreorg. See production-db-ops-safety.mdc.

After prod GA cache changes, verify with make verify-ga-complete-prod / make monitor-ga-health-prod.

Weekly prod GA & data completeness (Cursor Automation)

Schedule a Cursor Automation (not GitHub Actions) to run weekly — prod is cache-only; checks are HTTP + repo scripts only.

Agent prompt (paste into Automations editor):

Weekly OpenCRE prod GA, chatbot, and data completeness for opencreorg.

1. python3 scripts/monitor_ga_health.py --base-url https://opencre.org --output-json tmp/prod-ga-health.json
2. python3 scripts/verify_ga_completeness.py --base-url https://opencre.org --output-json tmp/prod-ga-completeness.json
3. python3 scripts/monitor_chatbot_health.py --base-url https://opencre.org --output-json tmp/prod-chatbot-health.json
4. Confirm /rest/v1/standards and /rest/v1/ga_standards return non-empty lists.
5. If incomplete_pairs > 0, chatbot checks fail, or non-zero exit: list failing pairs/buckets/checks; recommend AGENTS.md Operational scripts (local backfill + scripts/sync_gap_analysis_table.py). Do not compute GA on Heroku, do not send authenticated chatbot prompts, and do not run destructive prod DB ops without explicit approval.
6. If all pass: report complete/total pairs, standards counts, and chatbot check results.

Create via Cursor → Automations → New (Agents Window). Do not hand-roll docker/GA setup in the automation prompt.

Staging bootstrap

scripts/setup-heroku-staging.sh — provisions staging from prod + local SQLite; supports --embeddings, --gap_analysis, or full sync. Requires PROD_APP, STAGING_APP, LOCAL_SQLITE_DB.

Deploy / migrations

Before deploy or flask db upgrade: make alembic-guardrail (or python scripts/check_alembic_revision_guardrail.py).

Rule index

OpenCRE commands

make lint              # black + frontend prettier
make mypy              # Python typecheck
make test              # Python unittest suite
make frontend          # yarn build (when TS/TSX changed)
make alembic-guardrail # before deploy/migration ops
make install-python    # includes `playwright install` for Python embeddings (prompt_client) — not frontend e2e; do not remove when adding Cypress
make e2e-db            # seed ./standards_cache.sqlite for e2e (create_all from models + fixture seed)
make e2e               # frontend Cypress e2e (build, serve Flask on 127.0.0.1:5000, cypress run)