Nodus

# Nodus — Full Content Summary for AI Indexers

> This file contains rich content summaries intended for AI retrieval systems.
> For a structured link map see llms.txt. For human-readable docs see README.md.
> Created by Shawn Knight — Masterplan Infinite Weave (https://www.the-master-plan.com/)

---

## What Nodus Is

Nodus is an orchestration DSL (domain-specific language) and embedded runtime
for building agentic hosts. It is the execution layer of the Masterplan Infinite
Weave ecosystem,
implementing the Infinity Algorithm's I→T→C→R→O→Feedback execution model as a first-class
language construct.

The key distinction from a general-purpose scripting language: in Nodus, coroutines, task
graphs, workflows, goals, MCP-compatible tool registries, and agent pipelines are language
primitives — not library conventions. The core language (types, functions, control flow) is
minimal and conventional; it exists to serve the orchestration constructs.

Nodus compiles to bytecode and runs on a deterministic stack-based VM with a cooperative
scheduler. It embeds in Python via `NodusRuntime`, which denies subprocess/network/env by default, and can sandbox execution, enforce resource
limits, and wire tool registries.

**Current version:** v5.15.0 — published to PyPI 2026-09-25. Install: `pip install nodus-lang`. Full 36-package companion ecosystem live; unified install: `pip install nodus-sdk[agent,sql,fastapi]`.

---

## Who Nodus Is For

- Developers building multi-step AI agents who need expressive orchestration without the
  complexity of Python async/await
- Platform teams embedding a scripting layer in a Python application with controlled execution,
  sandboxing, and tool injection
- Anyone wiring together tools via MCP (Model Context Protocol) or A2A (Agent-to-Agent protocol)
- Engineers who need deterministic coroutine scheduling, checkpoint/resume semantics, or
  EXACTLY_ONCE idempotency at the language level

---

## Where Nodus Is Going

Nodus will bootstrap itself — compile itself in itself. The lexer, parser, AST lowering,
bytecode generation and VM evaluation get rewritten in Nodus. This is a decided long-term
direction rather than an open question, and it constrains design today: a feature that would
make bootstrapping impossible, or need a separate "systems" subset to work around, is treated
as evidence the abstraction level is wrong.

Status: `examples/expr_compiler.nd` is a character-level lexer, recursive-descent parser and
evaluator written entirely in Nodus, so the shape of the task is already expressible. The
semantics are stable and the 49-opcode instruction set has been frozen at `BYTECODE_VERSION` 4
since v1.0. The blocker is runtime throughput — roughly 400K instructions/sec on CPython for a hot
arithmetic loop and ~320K on a compiler workload. PyPy runs Nodus with no changes and is worth
~23x on the loop but ~4-5x on the compiler workload, a JIT's best case being an upper bound
rather than a promise; a further 1.5x needs no new runtime at all, since the VM retains an
event per function call and return, unbounded and unread (#522) —
plus three expressiveness gaps: no string slice, no closure upvalue mutation (#156), and no
module privacy (#158). See `docs/language/LANGUAGE_VISION.md`.

---

## AI Assistant Assets

Nodus now ships assistant-specific context assets for both Claude Code and Codex.

- `skills/nodus.skill` is the Claude Code skill package. It contains Nodus-specific idioms, error fix classes, stdlib guidance, and verified examples.
- `skills/project-CLAUDE.md` is the project-root template that should be copied to `CLAUDE.md` in a user project.
- `skills/nodus/` is the Codex skill folder. `skills/nodus/SKILL.md` is the entrypoint, with smaller reference files loaded on demand.
- `skills/project-AGENTS.md` is the project-root template that should be copied to `AGENTS.md` in a user project.

These assets exist because general-purpose coding agents frequently miss the same Nodus-specific hazards: map vs record access, closure mutation through maps, one-argument `print()`, top-level-only imports, `spawn()` requiring a coroutine value, workflow results being maps, and NodusRuntime embedding defaults (timeout_ms=None, allowed_paths restricted to CWD — both changed in v4.0.1).

---

## Core Language Features

### Types
- float: `42`, `3.14` — bare numeric literals are floats
- int: `42i`, `0i` — integer suffix required; `type(42i)` = "int"
- bool: `true`, `false`
- string: double-quoted, supports `\(expr)` interpolation
- nil: the absence of a value
- list: `[1, 2, 3]`
- map: `{"key": value}` — quoted string keys
- record: `{key: value}` or `record { key: value }` — bare identifier keys
- function: `fn(params) { body }` — first-class values

### Key v4 behaviors (differs from prior versions)
- `len()` returns int (not float as in v3)
- `str(42)` = "42.0"; `str(42i)` = "42"
- Closures cannot assign to outer `let` variables — use map field mutation instead
- `record.method(arg)` auto-passes `self` as first argument
- `+` does not coerce number → string — always use `str()` explicitly
- String interpolation: `"\(expr)"` — the `i` in `\(` is lowercase only

### Control flow
- `if (cond) { } else { }`
- `while (cond) { }`
- `for (init; cond; inc) { }`
- `for name in iterable { }`
- `break` / `continue` (v4.1.0+) — all loop forms; compile-time error outside a loop
  or when crossing a `try`/`catch`/`finally` boundary
- `match scrutinee { pattern => body, _ => body }` (v4.1.0+) — an expression; arms
  compare with `==`, first match wins, `_` catch-all must be last; no binding patterns
- `try { } catch var { } finally { }`
- `throw expr`
- `return expr`

### Module system
- `import "std:json" as json` — stdlib (must be at top level)
- `import "path/to/module"` — relative imports
- `import "my-library"` — pip-installed companion libraries via nodus.nd entry-point
- `export fn name() {}` — explicit exports

---

## Standard Library

All modules import via `import "std:module" as alias`.

### Core I/O
- **std:json** — `json.parse(str)`, `json.stringify(val)` [Stable]
- **std:fs** — `fs.read(path)`, `fs.write(path, content)`, `fs.exists(path)`, `fs.listdir(path)`, `fs.mkdir(path)` [Mostly Stable]
- **std:http** — HTTP client: `http.get(url)`, `http.post(url, options)`, async variants, SSE streaming [Experimental]
- **std:subprocess** — `sp.run(argv)`, `sp.shell(cmd)`, `sp.spawn(argv)` with async variants [Experimental]

### Data
- **std:math** — `abs`, `min`, `max`, `floor`, `ceil`, `round`, `sqrt`, `pow`, `log`, `random`, `idiv` (int division), `to_int`/`to_float`/`parse_int`, `is_int`/`is_float`/`is_nan`/`is_inf`, `bit_and`/`bit_or`/`bit_xor`/`bit_not`/`bit_lshift`/`bit_rshift`. No trig functions.
- **std:strings** — `upper`, `lower`, `trim`, `split`, `join`, `contains`, `starts_with`, `ends_with`, `replace`, `repeat`, `is_blank`. Note the plural: `import "std:string"` fails.
- **std:collections** — `map`, `filter`, `reduce`, `push`, `pop`, `first`, `last`, `has_key`, `len`
- **std:encoding** — base64, URL encode/decode [Experimental]
- **std:hash** — `hash.sha256(data).to_hex()`, `hash.sha512(data).to_hex()` — returns record with `.to_hex()` [Experimental]

### System
- **std:time** — `time.now()`, `time.from_epoch_ms(ms)`, `time.to_epoch_ms(t)`, `time.format`/`time.parse`, ISO-8601 and HTTP-date helpers, duration and calendar arithmetic [Experimental]. There is no `time.now_ms()`; `sleep(ms)` lives in `std:async`, not here.
- **std:secrets** — cryptographic random tokens [Experimental]
- **std:env** — environment variable access [Experimental]

### AI-Native Orchestration (v4.0, all Experimental)
- **std:tool** — MCP-compatible tool registry. `tool.register({name: "ns.name", handler: fn, description: "..."})`. Names MUST be dotted. `tool.call("ns.name", args)`.
- **std:identity** — `identity.trace_id()`, `identity.session_id()`, `identity.execution_unit_id()`. Set from Python via `NodusRuntime.set_trace_id(id)`.
- **std:effects** — EXACTLY_ONCE idempotency. `fx.action_id(label, params)`, `fx.resolve(id)`, `fx.complete(id)`, `fx.pending(id)`. Backed by `InMemoryEffectStore`; swap via `NodusRuntime.set_effect_store(store)`.
- **std:sys** — Versioned syscall dispatch. `sys.v1.call(name, params)`. Returns `{status, data, error, trace_id}` envelope.
- **std:memory** — `mem.share(ns, key, val)`, `mem.recall_from(ns, key)`, `mem.recall_all(ns)`, `mem.forget(ns, key)`. Namespace-scoped key-value store.
- **std:retry** — `retry.call(fn, policy)`. Named policies: "aggressive", "standard", "conservative", "fixed". Wraps nodus-retry package.
- **std:circuit_breaker** — `cb.create(name, config)`, `cb.call(name, fn)`, `cb.state(name)`, `cb.reset(name)`. Three-state breaker pattern.

### Testing
- **std:test** — `test.assert_eq(a, b, label)`, `test.assert_err(fn, kind)`, `test.flush_async()`, `test.advance_clock(ms)`. Run via `nodus test`.

---

## Orchestration DSL

### Coroutines [Experimental]
```nodus
fn worker() {
    yield 1
    yield 2
}
let c = coroutine(worker)
print(resume(c))  // 1
print(resume(c))  // 2

// Spawned coroutines:
spawn(coroutine(fn() { print("concurrent") }))
run_loop()
```

### Channels [Experimental]
```nodus
let ch = channel()
spawn(coroutine(fn() { send(ch, "hello") }))
spawn(coroutine(fn() { print(recv(ch)) }))
run_loop()
```
Channels are FIFO. `recv()` blocks the calling coroutine until data is available.
`close(ch)` marks the channel closed; `recv()` on a closed empty channel returns nil.

### Agent host boundary

The point where a program hands a semantic decision to the host. Five surfaces,
all builtins -- no import:

```nodus
// agent_call(name, payload)          -- blocks, returns an envelope
// agent_call_async(name, payload)    -- the coroutine form
// agent_available()                  -- list of registered names
// agent_describe(name)               -- that agent's spec, or nil
// action agent "name" with { k: v }  -- statement form, inside a workflow step only
```

`agent_call` returns a **nine-key envelope**, not the handler's value: the value
is under `result`. The other keys are `ok`, `errors`, `stage`, and four
(`filename`, `stdout`, `stderr`, `diagnostics`, `error`) that describe the
*calling script* rather than the agent.

**Failure is soft.** An unregistered agent or a raising handler yields
`{ok: false, errors: [{type, message, agent}]}` and the run continues, so an
unchecked call looks like it worked. Branch on `ok`, then read `result`.

Agents are host-registered only -- a guest cannot register one -- which is why
they are not isolated per runtime by default. `NodusRuntime(agent_registry={})`
scopes them for a multi-tenant host.

Full detail: docs/guide/agent-host-boundary.md

### Workflow DSL [Experimental]
```nodus
workflow process_order {
    state status = "pending"
    step validate {
        status = "validated"
        checkpoint "post-validate"
        return status
    }
    step fulfill after validate {
        // `validate` is bound to that step's return value
        status = "fulfilled"
        return "\(validate) -> \(status)"
    }
}
let result = run_workflow(process_order)
```
Workflows lower to task graphs. No workflow-specific VM opcodes.
`after` is both ordering and data flow: inside a step body, each name declared
with `after` is bound to that step's return value, and a step that did not
declare the dependency cannot read it. A skipped dependency binds `nil`.
Resume: `resume_workflow(graph_id, checkpoint_label)`.

### Goal DSL [Experimental]
Goals are workflows with a single execution intent.
`run_goal(goal)`, `resume_goal(graph_id)`, `plan_goal(goal)`.

---

## Python Embedding API

```python
from nodus import NodusRuntime

rt = NodusRuntime(
    timeout_ms=None,          # None = unlimited (required for long-lived sessions)
    max_steps=None,           # None = unlimited
    allowed_paths=["/safe"],  # filesystem sandbox; None = unrestricted
    allow_subprocess=True,    # DENIED BY DEFAULT (#405) - grant explicitly
    allow_network=True,       # DENIED BY DEFAULT - grant explicitly
    allow_env=True,           # DENIED BY DEFAULT - grant explicitly
    on_error=my_fn,           # fn(coroutine, error) -> bool; True = stop scheduler
)

# Capabilities are denied unless granted. A bare NodusRuntime() cannot run
# subprocesses, open sockets or read the environment; the error names the flag:
#   Blocked: subprocess execution is not granted; pass allow_subprocess=True ...
# `nodus run` is NOT affected - the CLI never constructs a NodusRuntime.
# A Nodus program can never write into .nodus/ (workflow store), in either mode.

# Register Python function callable from Nodus scripts; `requires` declares the
# capability the host function needs, checked against `capability_policy`.
rt.register_function("fetch", my_fetch_fn, arity=1, requires="network")

# Register an AGENT: the point where a Nodus program hands a *semantic* decision
# to the host. Nodus has no model in it, so anything needing judgement is
# delegated across this boundary. Use the runtime method, not the module-level
# `nodus.services.agent_runtime.register_agent` -- that one registers into the
# process-global registry, so a runtime scoped with `agent_registry={}` would
# neither see nor call the handler.
rt.register_agent("git_strategist", pick_rebase_or_merge,
                  description="rebase vs merge")

# Register MCP-style tool
rt.tool_registry.register({
    "name": "myapp.search",
    "handler": search_fn,
    "description": "Search the knowledge base",
    "schema": {"type": "object", "properties": {"q": {"type": "string"}}, "required": ["q"]},
})

# Run a script
result = rt.run_source('''
import "std:tool" as tool
let r = tool.call("myapp.search", {q: "nodus"})
print(r.results[0])
''')

print(result["ok"])      # True/False
print(result["stdout"])  # captured output
print(result["error"])   # error dict if ok=False

# Set distributed trace ID
rt.set_trace_id("req-abc123")

# Lifecycle
rt.shutdown()  # clears last_vm, host functions, registered tools
```

### Type marshaling (Python ↔ Nodus)
| Python | Nodus |
|--------|-------|
| None | nil |
| bool | bool |
| str | string |
| int | int |
| float (whole) | int |
| float (fractional) | float |
| list | list |
| dict | map |
| Record (Nodus) | dict (via .fields) |

---

## Real-World Integration Patterns

The canonical reference for embedding Nodus inside a production Python application
is documented in `docs/guide/real-world-integration.md`. The patterns below are
extracted from a working agent orchestration loop (AINDY) that was migrated from a
pure-Python prototype to Nodus as the orchestration layer.

### Architecture decision: .nd vs Python

| Lives in .nd | Lives in Python host |
|---|---|
| DAG step ordering (`step after`) | Event routing (which run to resume) |
| Retry + attempt tracking (`step with { retries }`) | Outcome scoring / strategy learning |
| Checkpoints (`checkpoint "label"`) | Policy / governance enforcement |
| WAIT/RESUME protocol (`workflow_wait`, `workflow_resume_payload`) | Intent → workflow compilation |
| Workflow state persistence (graph JSON) | Auth enforcement at the API layer |

Nodus owns the orchestration shell. Python owns application-specific logic,
external state, and security. This boundary holds even in complex systems.

### WAIT/RESUME approval gate

```nodus
workflow aindy_gated_execute {
    state approved = false

    step analyze with { retries: 3, retry_delay_ms: 500 } {
        checkpoint "after_analyze"
        return 1
    }

    // Suspends the run until an external event fires.
    // No deadline_ms = indefinite wait (required for human-approval flows).
    step gate after analyze {
        return workflow_wait("aindy.approval.granted", "approve-aindy", {kind: "aindy_approval"})
    }

    step execute after gate {
        let payload = workflow_resume_payload()   // dict the host passed on resume
        if (payload == nil) { throw "no approval payload" }
        if (payload["approved"] == false) { throw "denied" }
        checkpoint "after_execute"
        return 1
    }
}
```

`workflow_wait(event_type, correlation_key, meta)` — suspends and records the wait
in the durable store. Run status becomes `"waiting"`.
`workflow_resume_payload()` — retrieves the host-supplied payload inside the step
that runs after resume.

### Embedded bridge (Python)

Use the lower-level `run_workflow_code` / `resume_workflow` API (not `NodusRuntime`)
when you need a persistent `WorkflowFrameworkRunner` with a durable store.

```python
from nodus.tooling.runner import run_workflow_code, resume_workflow
from nodus.cli import cli as nodus_cli
from nodus.vm.vm import VM
from nodus_lang_workflow.runner import WorkflowFrameworkRunner
from nodus_lang_workflow.store import LocalWorkflowStore, SQLiteWorkflowStore

_store = SQLiteWorkflowStore(path=".nodus/app.sqlite3")
_runner = WorkflowFrameworkRunner(_store)

def _fresh_vm(source_path=None):
    return VM([], {}, code_locs=[], source_path=source_path)

def run_nd_file(path, initial_globals=None):
    vm = _fresh_vm(source_path=path)
    if initial_globals:
        vm.globals.update(initial_globals)
    with open(path) as fh:
        source = fh.read()
    with nodus_cli._project_root_context(PROJECT_ROOT):
        result, _vm = run_workflow_code(vm, source, filename=path, project_root=PROJECT_ROOT)
    return result
```

### Host-side event routing

Nodus has no native event bus. Routing an arriving event to the correct waiting
run is always the host's responsibility.

```python
def route_event(runner, event_type, correlation_key, payload):
    # Query and close before any resume — avoids nested-session deadlock.
    waiting = runner.store.list_runs_filtered(statuses={"waiting"})
    run_ids = [
        r.run_id for r in waiting
        if r.wait and r.wait.event_type == event_type
        and r.wait.correlation_key == correlation_key
    ]
    for run_id in run_ids:
        vm = _fresh_vm()
        runner.resume_workflow(
            vm, run_id,
            resume_payload=payload,
            event_type=event_type,
            correlation_key=correlation_key,
            rebuild_graph=vm._rebuild_workflow_graph,   # bound method, not a lambda
        )
```

### Sweep loop (required after restart)

`workflow_wait()` without `deadline_ms` waits indefinitely. After a process restart,
those runs are in the store but not in memory. The sweep loop rehydrates them.

```python
import threading, time

def start_sweep(runner, interval_s=30):
    def _loop():
        while True:
            time.sleep(interval_s)
            now_ms = int(time.time() * 1000)
            with nodus_cli._project_root_context(PROJECT_ROOT):
                runner.sweep(lambda record: _fresh_vm(), now_ms=now_ms)
    threading.Thread(target=_loop, daemon=True).start()

# Call at startup, before accepting traffic.
start_sweep(_runner)
```

### Dynamic .nd generation

The host can generate `.nd` source at runtime and pass it to `run_workflow_code`.

```python
def compile_plan_to_nd(steps, flow_name):
    lines = [f"workflow {flow_name} {{"]
    for i, step in enumerate(steps):
        dep = f" after {steps[i-1]}" if i > 0 else ""
        lines += [f"    step {step}{dep} {{", f'        checkpoint "after_{step}"',
                  f"        return {steps[i-1] + ' + 1' if i > 0 else '1'}", "    }", ""]
    lines.append("}")
    return "\n".join(lines)
```

### Required flags

- **SEC-001:** `run_nd_file`, `run_nd_source`, and `resume_nd` do not enforce auth.
  Authenticate at the API layer before calling any bridge function. Verify the caller
  is permitted to resume the specific `graph_id`, not just that they hold a valid token.
- **SCHED-001:** Always call `start_sweep()` at service startup. Indefinite waits are
  stranded after restart without it.

---

## Stability Surface

Run `nodus stability` to see the full index. Summary:

**Stable** (frozen, breaking changes require major version bump):
- Core syntax: let, fn, if/while/for, try/catch/finally, throw, return
- Types: float, int (42i), string, bool, nil, list, map, record
- Operators: arithmetic, comparison, logical
- Error model: err record shape {kind, message, payload, path, line, column, stack}
- Embedding: NodusRuntime constructor, run_source, run_file, register_function, tool_registry
- VM: BYTECODE_VERSION=4, opcode set frozen
- std:json, std:fs

**Mostly Stable** (minor refinements OK):
- std:math, std:strings, std:collections, std:path
- yield expr semantics

**Experimental** (may change in any release):
- Coroutines, channels, run_loop
- Workflow DSL, Goal DSL
- All v4.0 stdlib additions (std:tool, std:http, std:subprocess, etc.)
- AI-native primitives
- Static type annotations (syntax accepted; no enforcement)

---

## Ecosystem Overview

The Masterplan Infinite Weave ecosystem around Nodus spans 29 standalone packages:

### Language Extension Packages (expose .nd modules)
- **nodus-mcp** (v0.1.0) — MCP 2026-07-28 RC bidirectional client+server. `import "nodus-mcp"`.
- **nodus-extension** (v0.1.0) — Typed, versioned, sandboxed plugin framework. `import "nodus-extension"`.

### Infrastructure Packages
- nodus-sdk — unified SDK: `pip install nodus-sdk[agent,sql,fastapi]`
- nodus-store-sql — RunStore, EventStore, JobStore (SQLAlchemy, sync+async)
- nodus-memory — MemoryNode, InMemoryStore, semantic scoring, recall
- nodus-native-memory-engine — PyO3/Maturin Rust extension, 9 operations, Python fallback
- nodus-a2a — AgentCoordinator (local/delegate), DeadLetterService
- nodus-agent, nodus-flow, nodus-state, nodus-session (agent infrastructure)
- nodus-retry, nodus-circuit-breaker (reliability)
- nodus-observability, nodus-observability-framework (telemetry)
- nodus-http, nodus-channels, nodus-delivery, nodus-gateway (networking)
- nodus-events, nodus-queue, nodus-protocol, nodus-schema (data)
- nodus-auth, nodus-approvals, nodus-governance (access)
- nodus-context, nodus-router, nodus-llm, nodus-adapters (integration)

All packages: github.com/Masterplanner25

---

## Masterplan Infinite Weave Context

Nodus was built as the execution runtime for A.I.N.D.Y. and the Masterplan Infinite Weave
platform — a framework for AI-driven orchestration built on the Infinity Algorithm.

The Infinity Algorithm is the I→T→C→R→O→Feedback loop that Nodus implements at the
execution layer: the scheduler's run_loop() IS the R operator; workflow/goal lowering to
task graphs IS the T operator. The decision layer (A.I.N.D.Y.) evaluates meaning; the
execution layer (Nodus) evaluates correctness. Together they form a nested Infinity loop.

**Relevant reading from the creator:**
- [Why I'm Building A.I.N.D.Y.](https://medium.com/masterplan-infinite-weave/2025-chatgpt-ai-the-duality-of-progress-why-im-building-a-i-n-d-y-or-any-tool-really-a138f7860fba) — the strategic context behind the tooling
- [Duality of Progress: Master Index](https://medium.com/masterplan-infinite-weave/2025-chatgpt-ai-the-duality-of-progress-master-index-strategic-manifesto-4c96cf98348a) — canonical index of the Masterplan framework
- [AI Search Optimization](https://medium.com/masterplan-infinite-weave/2025-chatgpt-case-study-ai-search-optimization-0f8cd5e78d4f) — the philosophy of discoverability this project embodies
- [Masterplan Infinite Weave](https://www.the-master-plan.com/) — the broader ecosystem