Liquefy — AI Agent Interface

You are an AI agent. This file tells you everything you need to operate Liquefy. Read this FIRST. It replaces reading the README, docs, or source code for 95% of tasks.

For what each component does, what has been demonstrated and what is not established, see REVIEW.md (generated from evidence/claims.json; validate with node scripts/check-claims-registry.mjs).

One-Line Setup

macOS / Linux:

make setup

Or if make isn't available:

bash install.sh

Windows (PowerShell):

.\setup.ps1

Windows (CMD):

setup.bat

pip install (into existing project):

pip install git+https://github.com/Parad0x-Labs/openclaw-skills.git

Optional extras: pip install "liquefy-openclaw[vision,cloud,anchor,api,all] @ git+https://..."

Docker:

docker compose run liquefy tools/tracevault_pack.py ./data --org default --out ./vault/output --json

All paths create a virtualenv, install dependencies, run a smoke test, and print "READY".

What Liquefy Does

Liquefy compresses, redacts, and vaults AI agent workspace data (session logs, JSONL traces, artifacts, screenshots, memory files). It has 24 specialized compression engines that auto-detect data format and pick the best one. Everything is verified bit-perfect (MRTV — Mandatory Round-Trip Verification).

Quick Commands (Copy-Paste Ready)

All commands work from the repo root after make setup.

Compress a directory into a vault

make quick DIR=./path/to/data

Or explicitly:

python tools/tracevault_pack.py ./path/to/data --org default --out ./vault/output

Compress an OpenClaw workspace

python tools/liquefy_openclaw.py --workspace ~/.openclaw --out ./vault/openclaw --json

Works with any agent framework

Liquefy is framework-agnostic. OpenClaw, NanoClaw, LangChain, CrewAI, Claude Agent SDK, custom scripts — if it writes files, Liquefy handles it. No adapter needed.

# NanoClaw container output
python tools/tracevault_pack.py ./nanoclaw-runs/latest --org dev --out ./vault/latest

# LangChain traces
python tools/tracevault_pack.py ./langchain-output --org dev --out ./vault/latest

# Any agent output directory
make quick DIR=./my-agent-output

Restore from a vault

python tools/tracevault_restore.py ./vault/output --out ./restored/

Search inside compressed vaults (no restore needed)

python tools/tracevault_search.py ./vault/output --query "error"

Scan for leaked secrets

python tools/liquefy_leakhunter.py scan ./path/to/data --deep --json

Visualize vault contents

python tools/liquefy_viz.py timeline ./vault/
python tools/liquefy_viz.py web ./vault/ --port 8377

Run the archiver daemon

python tools/liquefy_archiver.py once --watch ~/.openclaw --out ./vault
python tools/liquefy_archiver.py daemon --watch ~/.openclaw --out ./vault

Ingest telemetry JSONL

cat telemetry.jsonl | python tools/liquefy_telemetry_sink.py pipe --out ./vault/telemetry

Sandbox-test a skill

python tools/liquefy_sandbox.py run ./skills/some_skill --timeout 60 --json

Sync to Obsidian

python tools/liquefy_obsidian.py sync --vault-root ./vault --obsidian ~/Obsidian/MyVault

Native OpenClaw integration (zero-config)

make openclaw-hook       # Install hooks — auto-triggers on every session close
make status              # Show integration status
# Users won't know it's there until they type: make status

AI Intelligence Layer

make predict DIR=~/.openclaw         # "This agent will hit 2 GB in 3 days"
make suggest DIR=~/.openclaw         # "Switch to ratio profile, enable archiver"
make score VAULT=./vault             # Value-score every trace (high/med/low)
make prune DIR=./vault               # Auto-prune low-value, keep high-value (dry-run)
make summarize VAULT=./vault         # LLM-powered "what actually mattered today"
make migrate SRC=./old.tar.gz OUT=./vault  # Import from tar/zstd/gzip backups

Fleet Coordination (Multi-Agent)

make fleet-register AGENT=agent-47 QUOTA_MB=500 PRIORITY=20
make fleet-status                    # Dashboard: all agents, usage, health
make fleet-quota AGENT=agent-47      # Check quota headroom
make fleet-ingest AGENT=agent-47 SRC=./data  # Compress for a specific agent (quota-enforced)
make fleet-merge TARGET=main-agent SOURCES='agent-1 agent-2 agent-3' STRATEGY=last_write
make fleet-gc MAX_AGE=30             # Remove old vaults, enforce quotas

All agents share one vault root (~/.liquefy/fleet/). Each gets a namespace partition. File-level locking ensures safe concurrent access from multiple processes. Conflict resolution: last_write, largest, priority, or both (keep-both).

Compliance & Audit

make audit-verify                    # Verify tamper-evident hash chain is intact
make compliance VAULT=./vault ORG=acme TITLE="Q1 Audit"  # Generate HTML compliance report
make compliance-verify VAULT=./vault  # Chain integrity check (pass/fail)
make compliance-timeline VAULT=./vault # Chronological event timeline (HTML)

Or directly:

python tools/liquefy_compliance.py report --vault ./vault --org acme --title "Q1 Audit" --output report.html
python tools/liquefy_compliance.py verify --vault ./vault --json
python tools/liquefy_compliance.py timeline --vault ./vault --output timeline.html

Vision — Screenshot Dedup (Engine #24)

Agents capture redundant screenshots. Vision deduplicates near-identical images using perceptual hashing (aHash), storing only unique frames.

make vision-scan DIR=./agent-screenshots         # Report dedup potential
make vision-pack DIR=./agent-screenshots          # Pack into VSNX vault (deduplicated)
make vision-restore SRC=./vault/vision.vsnx       # Restore all images
make vision-stats SRC=./vault/vision.vsnx         # Show dedup stats

Or directly:

python tools/liquefy_vision.py scan  ./agent-screenshots --json
python tools/liquefy_vision.py pack  ./agent-screenshots --out ./vault/vision.vsnx --json
python tools/liquefy_vision.py restore ./vault/vision.vsnx --out ./restored --json
python tools/liquefy_vision.py stats ./vault/vision.vsnx --json

Install Pillow for perceptual dedup (pip install Pillow). Without it, falls back to exact SHA-256 dedup.

Policy Enforcer (Active Kill Switch)

Goes beyond audit-mode: actively BLOCKS operations and can HALT agent processes when critical violations are detected.

Three escalation levels:

make policy-audit DIR=./agent-output     # Report only (safe, no side effects)
make policy-enforce DIR=./agent-output   # Block on critical/high — non-zero exit
make policy-kill DIR=./agent-output SIGNAL=./agent.halt PID=1234  # Halt signal + SIGTERM

Or directly:

python tools/liquefy_policy_enforcer.py audit   --dir ./agent-output --json
python tools/liquefy_policy_enforcer.py enforce --dir ./agent-output --json
python tools/liquefy_policy_enforcer.py kill    --dir ./agent-output --signal ./agent.halt --pid 1234 --json
python tools/liquefy_policy_enforcer.py watch   --dir ./agent-output --signal ./agent.halt --interval 5

What it catches:

• secret_leak (critical) — API keys, tokens, passwords, private keys, AWS creds, JWTs
• forbidden_ext (high) — .exe, .dll, .so, .msi, .vbs, etc.
• forbidden_path (critical) — .ssh/, .gnupg/, .aws/credentials, .kube/config
• oversized (warning) — files exceeding 50MB (configurable via policy JSON)

Kill switch workflow: watch mode continuously scans the agent output directory. On critical violations, it writes a .liquefy-halt JSON file (agents watch for this) and optionally sends SIGTERM to the agent process group.

Hardened halt channel:

• HMAC signing — halt signals signed with LIQUEFY_SECRET (SHA-256 HMAC). Tampered signals rejected.
• Nonce — unique per signal, prevents replay attacks
• TTL/expiry — signals expire after 5 minutes (configurable). Stale commands rejected.
• Process group kill — os.killpg() terminates the entire process tree, not just one PID. Falls back to single-PID if group unavailable.
• Verify command — liquefy policy verify-halt --signal ./agent.halt validates signature, nonce, and TTL

# Verify a halt signal is authentic and not expired
python tools/liquefy_policy_enforcer.py verify-halt --signal ./.liquefy-halt --json

Custom policies: pass --policy policy.json with max_file_size, forbidden_extensions, etc.

Content-Addressed Storage (Cross-Run Dedup)

Blobs stored once by SHA-256. Vaults become lightweight manifests over shared blobs. Agent runs that produce the same screenshots, prompts, configs, tool outputs share storage automatically.

# First run: all blobs are new
make cas-ingest DIR=./agent-run-1

# Second run: only changed files stored, identical blobs deduped
make cas-ingest DIR=./agent-run-2

# See savings
make cas-status

# Restore any vault from its manifest
make cas-restore MANIFEST=<id> OUT=./restored

# Clean up unreferenced blobs
make cas-gc

Or directly:

python tools/liquefy_cas.py ingest  --dir ./agent-output --trace-id task-42 --label "run #5" --json
python tools/liquefy_cas.py restore --manifest abc123 --out ./restored --json
python tools/liquefy_cas.py status  --json
python tools/liquefy_cas.py gc      --json

How it works:

1. Each file is hashed (SHA-256)
2. Blob stored in sharded directory (~/.liquefy/cas/blobs/ab/abcd1234...)
3. If blob already exists (from any previous run), skip — instant dedup
4. Vault = manifest JSON pointing to blob hashes
5. Restore reconstructs any vault from shared blobs

Supports: --trace-id for multi-agent correlation, --label for human-readable vault names.

Unified CLI (liquefy)

One Python-first entry point for all operations:

liquefy pack       --workspace ~/.openclaw --out ./vault --apply
liquefy restore    ./vault/run_001 --out ./restored
liquefy search     ./vault --query "error" --json
liquefy policy     audit --dir ./agent-output --json
liquefy safe-run   --workspace ~/.openclaw --cmd "openclaw run" --sentinels SOUL.md
liquefy cas        ingest --dir ./agent-output --json
liquefy tokens     scan --dir ./agent-output --json
liquefy telemetry  push --webhook https://my-siem/api --json
liquefy events     emit --agent-id a1 --session-id s1 --event model_call
liquefy guard      save --dir . --json
liquefy anchor     --vault-dir ./vault --json
liquefy --version

Also installable as pip install liquefy-openclaw — the liquefy command is registered as a console script.

Agent Event Schema (Structured Traces)

Canonical trace model for agent-native operations:

# Emit events (from agent code or CLI)
python tools/liquefy_events.py emit \
    --agent-id researcher-1 \
    --session-id sess-42 \
    --event model_call \
    --model gpt-4o \
    --input-tokens 1500 \
    --output-tokens 300 \
    --prompt "Analyze this dataset" \
    --trace-id task-chain-99 \
    --json

# Query all events in a session
python tools/liquefy_events.py query --session-id sess-42 --json

# Build parent-child span tree
python tools/liquefy_events.py spans --session-id sess-42 --json

# Session statistics (tokens, cost, duplicate prompts)
python tools/liquefy_events.py stats --session-id sess-42 --json

Event fields:

• agent_id, session_id, span_id, parent_span_id — who did what
• trace_id — cross-agent correlation (same as --trace-id everywhere)
• model, input_tokens, output_tokens, cost_usd, duration_ms — model call metadata
• tool_name, tool_input_ref, tool_output_ref — tool call tracking
• prompt_hash, context_hash — deduplicate without storing raw prompts
• error, retry_count — failure tracking
• metadata — arbitrary key-value pairs

Event types: model_call, tool_call, tool_result, agent_start, agent_end, session_start, session_end, error, retry, escalation, handoff, checkpoint, policy_violation, custom.

Span trees: events linked by parent_span_id form a directed tree — reconstruct exactly how an agent reasoned through a task.

Safe Run (Automated Rollback + Sentinel Monitoring)

Wraps any agent execution with pre-flight state capture and automatic rollback on policy violations, crashes, or sentinel file tampering.

The pattern: snapshot workspace -> execute agent -> enforce policies -> check sentinels -> auto-restore if anything went wrong.

# Basic: snapshot ~/.openclaw, run agent, restore on policy violation
make safe-run WORKSPACE=~/.openclaw CMD="openclaw run task.md"

# With sentinel monitoring: SOUL.md, HEARTBEAT.md, auth-profiles.json
make safe-run WORKSPACE=~/.openclaw CMD="openclaw run" SENTINELS=SOUL.md,HEARTBEAT.md,auth-profiles.json

# With policy + trace-id for multi-agent chains
make safe-run WORKSPACE=~/.openclaw CMD="python agent.py" POLICY=./policies/strict.yml TRACE_ID=task-42

Or directly:

python tools/liquefy_safe_run.py \
    --workspace ~/.openclaw \
    --cmd "openclaw run task.md" \
    --sentinels SOUL.md,HEARTBEAT.md,auth-profiles.json \
    --policy ./policies/strict.yml \
    --trace-id task-42 \
    --json

Five phases:

1. Snapshot — full workspace state captured (file hashes + copies)
2. Heartbeat — optional Dead Man's Switch (--heartbeat): writes .liquefy-heartbeat every 5s so agents can verify monitoring is alive
3. Execute — agent command runs with timeout protection
4. Enforce — policy enforcer + token cost check (--max-cost) + sentinel comparison
5. Rollback — if anything failed, workspace is restored to pre-run state

# With cost cap: kill + rollback if agent burns more than $5
python tools/liquefy_safe_run.py \
    --workspace ~/.openclaw \
    --cmd "python my_agent.py" \
    --max-cost 5.00 \
    --heartbeat \
    --json

# Agent can check if monitoring is alive:
# cat ~/.openclaw/.liquefy-heartbeat  →  {"pid": 12345, "ts": "...", "interval_s": 5}
# If file is older than 2× interval, monitoring died — agent should self-halt.

Token cost enforcement (--max-cost): scans agent output logs for LLM token usage metadata after the run. If estimated cost exceeds the limit, triggers automatic rollback — prevents economic denial-of-service attacks where a rogue skill loops expensive API calls.

Sentinel files: critical identity files (SOUL.md, HEARTBEAT.md, auth-profiles.json) are hashed before the run. If an agent or a malicious skill modifies them during execution, the tampering is detected and the workspace is rolled back automatically.

Dead Man's Switch (--heartbeat): writes a heartbeat file every 5 seconds while the agent runs. Agents or external watchers can check this file — if it goes stale (older than 2× interval), they know Liquefy monitoring crashed and should self-halt. This prevents unmonitored execution.

Use --no-restore to report violations without auto-restoring (forensic mode).

Known limitation: multi-modal context poisoning (hidden text in images used for prompt injection) is out of scope for Liquefy's policy engine. Liquefy audits files, not pixel content. LLM providers are responsible for vision model safety filters.

Docker Jail Pattern

Run OpenClaw inside a container, Liquefy outside on the host — so a compromised agent can't escape to your host machine.

# docker-compose.override.yml
services:
  openclaw-agent:
    image: openclaw/agent:latest
    volumes:
      - ./openclaw-workspace:/workspace
    network_mode: none  # no network access unless explicitly needed

  liquefy-auditor:
    build: .
    volumes:
      - ./openclaw-workspace:/audit-target:ro  # read-only mount
    command: >
      python tools/liquefy_safe_run.py
        --workspace /audit-target
        --cmd "echo audit-only"
        --sentinels SOUL.md,HEARTBEAT.md
        --json

Key principle: the agent writes to /workspace, Liquefy reads from the same volume (read-only). If the agent gets compromised, it can't break out of the container, and Liquefy safely archives the evidence from a secure vantage point on the host.

Multi-Agent Chain of Custody (trace-id)

Pass a correlation ID between agent handoffs so you can trace a prompt from researcher -> executor -> verifier across separate vaults.

# Agent A (researcher) packs its output with a trace ID
python tools/liquefy_openclaw.py --workspace ~/.openclaw --out ./vault-a --apply --trace-id "task-42-researcher"

# Agent B (executor) receives the same trace ID
python tools/liquefy_openclaw.py --workspace ~/.openclaw --out ./vault-b --apply --trace-id "task-42-executor"

# Policy enforcer carries the same trace ID
python tools/liquefy_policy_enforcer.py enforce --dir ./vault-b --trace-id "task-42-executor" --json

# Query the audit chain by trace_id to reconstruct the full multi-agent graph
python -c "
from api.liquefy_audit_chain import get_audit_chain
chain = get_audit_chain()
for e in chain.query():
    if e.get('trace_id','').startswith('task-42'):
        print(e['event'], e.get('trace_id'), e['ts'])
"

• --trace-id flag on liquefy_openclaw.py, liquefy_policy_enforcer.py
• Also reads LIQUEFY_TRACE_ID env var (for agent frameworks that set env)
• Written into .liquefy-trace-id file inside the vault for offline correlation
• Logged into the tamper-evident audit chain with every operation
• Forwarded through Telemetry Forwarder to SIEM systems

Telemetry Forwarder (SIEM Streaming)

Push audit chain events to Splunk, Datadog, ELK, PagerDuty, Slack, or any SIEM.

make telemetry-push WEBHOOK=https://splunk:8088/services/collector TOKEN=xxx
make telemetry-stream WEBHOOK=https://hooks.slack.com/xxx INTERVAL=10
make telemetry-push SYSLOG=10.0.0.1:514
make telemetry-push FILE=/var/log/liquefy-events.jsonl
make telemetry-test WEBHOOK=https://my-siem/api/events
make telemetry-status

Or directly:

python tools/liquefy_telemetry_forward.py push   --webhook https://splunk:8088/services/collector --token xxx --json
python tools/liquefy_telemetry_forward.py stream --webhook https://hooks.slack.com/xxx --interval 10
python tools/liquefy_telemetry_forward.py push   --syslog 10.0.0.1:514 --json
python tools/liquefy_telemetry_forward.py push   --file /var/log/liquefy-events.jsonl --json
python tools/liquefy_telemetry_forward.py test   --webhook https://my-siem/api --json
python tools/liquefy_telemetry_forward.py status --json

Destinations: Webhook (HTTP POST JSON), Syslog (RFC 5424 UDP/TCP), File (JSONL append).

Cursor-based: tracks what's been sent so push only forwards new events. stream mode tails the audit chain continuously.

Token Ledger [EXPERIMENTAL]

Track, budget, and audit LLM token usage across agent runs. Parses OpenAI, Anthropic, LangChain, and generic JSONL traces.

EXPERIMENTAL: Token counts are extracted from agent logs on a best-effort basis. Actual billing may differ from estimates. Use provider dashboards for exact costs.

make token-scan DIR=./agent-output                              # Scan logs for usage
make token-budget ORG=acme DAILY=500000 MONTHLY=10000000        # Set limits
make token-report ORG=acme PERIOD=today                         # Usage report
make token-audit DIR=./agent-output                             # Detect waste

Or directly:

python tools/liquefy_token_ledger.py scan   --dir ./agent-output --json
python tools/liquefy_token_ledger.py budget --org acme --daily 500000 --monthly 10000000
python tools/liquefy_token_ledger.py report --org acme --period today --json
python tools/liquefy_token_ledger.py audit  --dir ./agent-output --json

What it detects:

• Duplicate prompts — identical prompts sent multiple times (wasted tokens)
• Oversized context — inputs exceeding 100K tokens
• Model overkill — small tasks routed to expensive models (GPT-4 for 50-token outputs)
• High input/output ratio — sending too much context for small responses

Cost estimates for GPT-4, GPT-4o, GPT-4o-mini, GPT-3.5-turbo, Claude 3/3.5/4 Opus/Sonnet/Haiku. Unknown models use a conservative default.

Budget alerts: set daily/monthly token or cost limits per org. Reports show usage percentage and warn when approaching limits.

Auto-detection:

• Unknown models — scan and audit automatically flag models not in the cost table with the exact command to add them
• Model switches — audit detects when agents switch models mid-trace (e.g. gpt-4o → gpt-5) and flags for review

28 built-in models (GPT-3.5/4/4o/5, o1/o3, Claude 3/3.5/4/4.5/4.6, Gemini 1.5/2.0, DeepSeek V3/R1, Llama 3.3, Mistral). Expandable:

make token-models                                               # List all models + costs
python tools/liquefy_token_ledger.py models --add 'gpt-6:0.01:0.03'  # Add/update a model

Or drop a model_costs.json at ~/.liquefy/tokens/model_costs.json or set LIQUEFY_MODEL_COSTS env var.

All usage data is logged to the Liquefy audit chain for tamper-evident tracking.

Config Guard (Update Protection)

Never lose your customizations to a framework update again. Config Guard snapshots your configs, skills, prompts, and env files before an update and restores them after.

# Before update — save everything
make guard-save DIR=./my-agent LABEL="pre-v2.0"

# Run your update (git pull, npm update, pip install --upgrade, etc.)

# After update — see what got overwritten
make guard-diff DIR=./my-agent

# Restore your customizations
make guard-restore DIR=./my-agent

# Check current state
make guard-status DIR=./my-agent

Or directly:

python tools/liquefy_config_guard.py save    --dir ./my-agent --label "pre-v2.0" --json
python tools/liquefy_config_guard.py diff    --dir ./my-agent --json
python tools/liquefy_config_guard.py restore --dir ./my-agent --json
python tools/liquefy_config_guard.py status  --dir ./my-agent --json

What it guards: .yaml, .json, .toml, .env, .py, .ts, .sh, Makefile, Dockerfile, requirements.txt, skill files, prompt files — anything config-like.

What it skips: node_modules/, .git/, __pycache__/, .venv/, dist/, build/.

Conflict handling: If the update changed a file AND you had customizations, Config Guard saves a .update-backup copy so you can merge manually. Use --force to skip backups. Use --dry-run to preview without changes.

Works with any framework: OpenClaw, NanoClaw, LangChain, CrewAI, or any project directory.

On-Chain Anchoring (Solana)

Anchor vault integrity proofs on Solana. Anyone with a Solana explorer can verify your data hasn't been tampered with — without seeing a single byte of it.

What goes on-chain (80 bytes):

• vault_hash — SHA-256 of all vault file hashes (32 bytes, truncated to 16 hex)
• chain_tip — latest audit chain hash (32 bytes, truncated to 16 hex)
• key_fingerprint — SHA-256 of encryption key (16 hex chars)

Cost: ~0.000005 SOL per anchor via SPL Memo program.

make vault-proof VAULT=./vault                    # Compute proof (free, offline)
make vault-anchor VAULT=./vault KEYPAIR=~/.config/solana/id.json  # Anchor on Solana
make vault-verify VAULT=./vault                   # Verify vault matches anchor
make vault-show PROOF=./vault/.anchor-proof.json  # Display proof details

Or directly:

python tools/liquefy_vault_anchor.py proof  --vault ./vault --json
python tools/liquefy_vault_anchor.py anchor --vault ./vault --keypair ~/.config/solana/id.json --json
python tools/liquefy_vault_anchor.py verify --vault ./vault --json
python tools/liquefy_vault_anchor.py show   --proof ./vault/.anchor-proof.json --json

Install solders and httpx for on-chain anchoring: pip install solders httpx. Proof computation works without any Solana dependencies.

Key Backup (Disaster Recovery)

If your machine dies and LIQUEFY_SECRET was only an env var, your encrypted cloud backups are bricks. Back up your key:

make key-backup                        # Export key (passphrase-protected)
make key-card                          # Printable recovery card
make key-recover SRC=./backup.enc      # Recover key on new machine
make key-verify SRC=./backup.enc       # Verify backup is valid

Or directly:

python tools/liquefy_key_backup.py export --output key_backup.enc
python tools/liquefy_key_backup.py recover --input key_backup.enc
python tools/liquefy_key_backup.py card --output RECOVERY_CARD.txt
python tools/liquefy_key_backup.py verify --input key_backup.enc

The backup file is encrypted with your passphrase (AES-256-GCM, PBKDF2 600k iterations). Store it on a USB, in a password manager, or print the recovery card and put it in a safe.

Cloud Sync (S3 / R2 / MinIO)

Sync encrypted vaults to S3-compatible storage. Cloud provider sees only opaque blobs — sovereign means encrypted everywhere.

make cloud-push VAULT=./vault BUCKET=my-backups                          # Push (incremental)
make cloud-push VAULT=./vault BUCKET=my-r2 ENDPOINT=https://xxx.r2.cloudflarestorage.com  # R2
make cloud-pull VAULT=./vault BUCKET=my-backups                          # Restore from cloud
make cloud-status VAULT=./vault BUCKET=my-backups                        # Compare local vs remote
make cloud-verify VAULT=./vault BUCKET=my-backups                        # Verify remote integrity

Or directly:

python tools/liquefy_cloud_sync.py push   --vault ./vault --bucket my-backups --json
python tools/liquefy_cloud_sync.py pull   --vault ./vault --bucket my-backups --json
python tools/liquefy_cloud_sync.py status --vault ./vault --bucket my-backups --json
python tools/liquefy_cloud_sync.py verify --vault ./vault --bucket my-backups --json

Environment variables: LIQUEFY_S3_ENDPOINT, LIQUEFY_S3_BUCKET, LIQUEFY_S3_PREFIX, AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY.

Presets (Choose Your Risk Level)

Liquefy defaults to maximum safety. Use presets to match your risk tolerance:

SAFE (default) — recommended for production

python tools/liquefy_openclaw.py --workspace ~/.openclaw --out ./vault --json

• Credentials, keys, envs: BLOCKED
• MRTV verification: FULL
• Profile: default (balanced ratio/speed)

POWER — faster, still safe, includes more file types

python tools/liquefy_openclaw.py --workspace ~/.openclaw --out ./vault \
  --mode balanced --profile speed --verify-mode fast --json

• Most credentials still blocked
• MRTV: fast (sampled verification)
• Profile: speed-first

YOLO — everything included, your responsibility

python tools/liquefy_openclaw.py --workspace ~/.openclaw --out ./vault \
  --mode off --profile ratio --verify-mode full \
  --include-secrets "I UNDERSTAND THIS MAY LEAK SECRETS" --json

• Nothing blocked
• Max compression ratio
• Full verification still on (non-negotiable safety net)

Custom policy file

python tools/liquefy_openclaw.py --workspace ~/.openclaw --out ./vault \
  --policy ./policies/my_policy.yml --json

Policy files live in ./policies/. Examples: strict.yml, balanced.yml, demo_risky.yml.

Interactive Setup

python tools/liquefy_setup.py

Walks you through:

1. What directories to watch
2. Risk tolerance (safe/power/yolo)
3. Encryption (on/off + secret generation)
4. Notifications (telegram/discord/off)
5. Daemon vs manual mode
6. Writes config to ~/.liquefy/config.json

JSON Output (Machine-Readable)

Every command supports --json for structured output:

python tools/tracevault_pack.py ./data --org dev --out ./vault --json

Returns:

{
  "schema_version": "liquefy.tracevault.cli.v1",
  "command": "pack",
  "ok": true,
  "result": {
    "total_original_bytes": 104857600,
    "total_compressed_bytes": 15728640,
    "overall_ratio": 6.67,
    "files_processed": 42,
    "verify_mode": "full",
    "mrtv_all_pass": true
  }
}

Schema contracts are in ./schemas/.

Health Checks

# Are all dependencies installed?
python tools/liquefy_cli.py doctor --json

# Do all engines load and roundtrip?
python tools/liquefy_cli.py self-test --json

# What version is this?
python tools/liquefy_cli.py version --json

Capabilities Summary

Compression Engines (24 total, auto-selected)

Environment Variables

File Layout

liquefy/
├── api/                    # Core engines + orchestrator + safety + security
│   ├── orchestrator/       # Engine routing, registry, contracts
│   ├── containers/         # .null vault format + bloom filters
│   ├── engines/core/       # 23 engine manifests (engine.json)
│   ├── json/               # JSON family engines (4)
│   ├── apache/             # Apache engines (2)
│   ├── syslog/             # Syslog engines (2)
│   ├── sql/                # SQL engines (3)
│   ├── k8s/                # Kubernetes engines (2)
│   ├── aws/                # CloudTrail + VPC Flow (2)
│   ├── nginx/              # Nginx engines (2)
│   ├── scm/                # GitHub engine (1)
│   ├── windows/            # Windows EVTX (1)
│   ├── vmware/             # VMware (1)
│   ├── netflow/            # NetFlow (1)
│   ├── universal/          # Universal + Fallback (2)
│   ├── vision/             # Screenshot perceptual dedup (1) — Engine #24
│   ├── liquefy_safety.py       # MRTV verification
│   ├── liquefy_security.py     # LSEC v2 encryption
│   ├── liquefy_primitives.py   # Shared varint/zigzag/bloom
│   ├── liquefy_audit_chain.py  # Tamper-evident hash-chained audit log
│   ├── liquefy_resilience.py   # Graceful degradation + self-healing
│   ├── liquefy_fleet.py        # Multi-agent fleet coordination core
│   └── orchestrator/
│       └── plugin_loader.py    # Community engine/pattern auto-discovery
├── tools/                      # CLI tools (all commands above)
│   ├── liquefy_openclaw_plugin.py  # Native OpenClaw integration
│   ├── liquefy_intelligence.py     # AI intelligence layer
│   ├── liquefy_fleet_cli.py        # Multi-agent fleet CLI
│   ├── liquefy_compliance.py       # HTML compliance report generator
│   ├── liquefy_vision.py           # Screenshot dedup CLI
│   └── liquefy_cloud_sync.py       # S3/R2/MinIO vault sync
├── api/engines/community/      # Drop-in community engines (auto-registered)
├── patterns/community/         # Drop-in leak patterns (auto-registered)
├── skills/                     # ClawHub skills
├── policies/                   # YAML policy files
├── schemas/                    # JSON schema contracts
├── tests/                      # 201 tests
└── bench/                      # Benchmarks

For AI Agent Developers

If you're building an AI agent that uses Liquefy:

1. Install: git clone <repo> && cd liquefy && make setup
2. Integrate: Call CLI tools with --json flag, parse JSON output
3. Schemas: Validate against ./schemas/liquefy.*.json
4. Presets: Use SAFE for production, POWER for development, YOLO for testing
5. Monitor: Use liquefy_viz.py web for dashboards or liquefy_obsidian.py sync for Obsidian
6. Automate: Use liquefy_archiver.py daemon for background archival
7. Secure: Run liquefy_leakhunter.py scan --deep before sharing any data
8. Extend: Drop engines into api/engines/community/ or patterns into patterns/community/ — auto-discovered
9. OpenClaw Native: Run make openclaw-hook once — Liquefy becomes the invisible default session store
10. Intelligence: Use make predict / make suggest / make summarize for proactive insights
11. Compliance: make audit-verify checks tamper-evident hash chain integrity. make compliance VAULT=./vault generates a beautiful HTML report for auditors
12. Fleet: Running multiple agents? Use make fleet-register + make fleet-ingest for shared namespace with quotas
13. Vision: Agent screenshots eating storage? make vision-pack DIR=./screenshots deduplicates near-identical frames (estimated 80-95% savings on repetitive captures — varies by content)
14. Cloud Sync: make cloud-push VAULT=./vault BUCKET=x syncs encrypted vaults to S3/R2/MinIO — cloud sees only opaque blobs
15. Agent Blueprints: liquefy agents list shows 15 ready-made templates. liquefy agents scaffold <id> generates a full workspace with guardrails, handoff contracts, and safe-run wiring. See docs/OPENCLAW_AGENT_BLUEPRINTS.md
16. State Guard: Agent crashed and forgot its wallet balance? liquefy state-guard init ~/.openclaw --files wallet-state.json --strict declares critical state. check verifies before each run, checkpoint backs up after, recover restores on crash. No more session-reset amnesia
17. History Guard: Agent has email/calendar/social access? liquefy history-guard init sets up continuous backup pulls + approval-gated destructive commands. Pre-action snapshots + auto-recovery. See docs/OPENCLAW_HISTORY_GUARD.md
18. PII Redaction: Strip emails, IPs, API keys, phone numbers, SSNs, wallet addresses before LLM ingestion. liquefy redact scan ./data for dry-run, liquefy redact apply ./data --out ./clean for redacted copy
19. Log De-Noise: Kill the Context Window Tax. liquefy denoise stats ./logs estimates noise ratio, liquefy denoise filter ./logs --out ./signal strips heartbeats/200s/health checks, keeps errors + context. Can cut token cost 60-95% depending on log composition — run denoise stats first to see yours