agentleFS
JoviDeCroockJoviDeCroock/pracht/skills/add-observability/SKILL.md

add-observability

Skill106 starsChanged 7 months agoMITView on GitHub

Wire Sentry or OpenTelemetry into pracht loaders, middleware, and API routes, report Web Vitals, and record every capability call with an audit sink. Use for "add observability", "wire Sentry", "set up tracing", "add OpenTelemetry", "monitor Web Vitals", "track errors", "log capability calls".

  • Reads credentials
  • Installs packages

1 other project uses a version of this.

Adopted by

Most recently taken up by

The file

We split this skill into the rules it gives an agent

11 rules. About 0 are common, 5 are distinctive, and 6 are very specific to this project.

  • ≈0

    Common · estimate

    Many files like this have a rule of this kind.

  • ≈5

    Distinctive · estimate

    This author's own choices. Worth a look.

  • 6

    Very specific

    Tied to their project's names, files or tools. Probably skip.

  • Common
  • Distinctive
  • Very specific
  1. Distinctive:

    Only relevant when the app registers capabilities.

    line 28
  2. Very specific:

    Add a global middleware that calls initObservability() once (from inside the handler, never at module scope) and wraps the downstream call:

    line 81
  3. Very specific:

    Create a SDK init module that runs at server entry — for Node, use the --require ./otel.cjs flag.

    line 148
  4. Distinctive:

    Use the framework hook from a component mounted by a shared shell:

    line 172
  5. Very specific:

    Import the module from an eagerly loaded server module.

    line 249
  6. Distinctive:

    Never send loader return values verbatim — they often contain user data.

    line 313
  7. Very specific:

    If an audit sink was added: call a capability in dev and confirm the event reaches both the sink and, when the adapter exposes it, the …

    line 319
  8. Very specific:

    Run pracht typegen if any routes were added (the beacon API route does not affect page-route types, but re-run when in doubt).

    line 324
  9. Very specific:

    Run pracht verify --json and confirm no failures.

    line 326
  10. Distinctive:

    Never send raw cookies, auth headers, or full loader payloads to a third-party SaaS.

    line 341
  11. Distinctive:

    Keep the synchronous part of an audit sink cheap: no CPU-heavy work or synchronous network call.

    line 345

≈ “Common” is an estimate: at least 1 in 5 of the 350,495 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.

Keep what fits and drop the rest. Your version, ready for your agents.
Files
SKILL.md

Pracht Add Observability

Four layers, each opt-in:

  1. Server error tracking — capture loader/middleware/API exceptions.
  2. Request tracing — span per request with child spans per loader/db call.
  3. Web Vitals (LCP/CLS/INP/FCP/TTFB) — client-side, posted to a beacon endpoint.
  4. Agent traffic — one structured audit event per capability dispatch. Only relevant when the app registers capabilities; skip it otherwise.

MCP: if pracht dev-mcp is registered, prefer its inspect_*, generate_*, doctor, and verify tools to the CLI.

Step 1: Pick the stack

Use AskUserQuestion:

  • Sentry — easiest end-to-end (errors + traces + Web Vitals).
  • OpenTelemetry + your backend (Honeycomb, Grafana, Datadog, Jaeger).
  • Custom beacon — minimal fetch('/api/telemetry') setup, no SaaS.

The skill below shows Sentry and OTel patterns. Custom beacon is mentioned but trivial.

Step 2: Server error tracking

pracht already prints one line per 5xx to the server console — phase, route id, source file, request path, message, and the stack — so the app is not silent before any of this is wired. What that default does not give you is a request id, a duration, the successful requests, or delivery to a sink off the box. Say so when the user only wants "errors in my logs": they may already have it.

Sentry path (Node adapter)

The pattern below uses @sentry/node and works on the Node adapter only — see the caveat box below for Cloudflare and Vercel Edge before installing anything.

pnpm add @sentry/node    # Node adapter only

Create src/server/observability.ts:

import { serverEnv } from "@pracht/core/env/server";
import * as Sentry from "@sentry/node";

let initialized = false;
export function initObservability() {
  if (initialized) return;
  initialized = true;
  // serverEnv is read INSIDE this function, not at module scope — module-level
  // env reads break on runtimes where env arrives per request (https://pracht.resynapse.dev/docs/env).
  Sentry.init({
    dsn: serverEnv.SENTRY_DSN,
    tracesSampleRate: Number(serverEnv.SENTRY_TRACES_SAMPLE_RATE ?? 0.1),
    environment: serverEnv.NODE_ENV,
  });
}

Add a global middleware that calls initObservability() once (from inside the handler, never at module scope) and wraps the downstream call:

// src/middleware/observability.ts
import type { MiddlewareFn } from "@pracht/core";
import * as Sentry from "@sentry/node";
import { initObservability } from "../server/observability";

export const middleware: MiddlewareFn = async ({ request, route }, next) => {
  initObservability();
  return Sentry.startSpan(
    {
      name: `${request.method} ${route.path}`,
      op: "http.server",
    },
    () => next(),
  );
};

Cloudflare / Vercel Edge caveat — be honest here. @sentry/cloudflare requires wrapping the worker's fetch handler with withSentry(), but pracht's Cloudflare adapter owns that handler — there is no user hook to wrap it today, so the middleware-init pattern above cannot work with @sentry/cloudflare, and @sentry/node does not run on Workers at all. Do not scaffold a pattern that can't work. Options on Cloudflare:

  1. Plain fetch-based event forwarding: catch errors in a wrap-around middleware and fetch them to Sentry's store/envelope endpoint (or any HTTP sink) yourself. Read the DSN via serverEnv inside the middleware.
  2. Wait for pracht to expose a handler-wrap hook for the adapter, then use Sentry's Cloudflare SDK properly.

The same applies to @sentry/vercel-edge: verify how the init hooks into the runtime before installing; if it needs to own the handler, fall back to option 1. (This mirrors the OTel-edge honesty note below.)

pracht middleware is wrap-around: await next() invokes the rest of the request and resolves to the final Response, so the span naturally covers the loader/handler and ends when they finish.

Register it in defineApp({ middleware: { observability: "./..." } }) (the top-level middleware field is a registry keyed by name — not an ordered chain). To actually wrap requests, place "observability" first in every chain that should cover them:

defineApp({
  middleware: { observability: "./middleware/observability.ts", auth: "./middleware/auth.ts" },
  api: { middleware: ["observability"] },           // all API routes
  routes: [
    group({ middleware: ["observability"] }, [      // all pages
      group({ middleware: ["auth"] }, [ /* protected routes */ ]),
    ]),
  ],
});

Ordering lives in these middleware: [...] arrays — always place observability first so it spans the rest of the chain.

OpenTelemetry path

pnpm add @opentelemetry/api @opentelemetry/sdk-trace-node @opentelemetry/auto-instrumentations-node

Create a SDK init module that runs at server entry — for Node, use the --require ./otel.cjs flag; for Cloudflare/Vercel edge, OTel is more limited (use HTTP exporter directly). Surface this trade-off; don't pretend OTel edge is plug-and-play.

Step 3: Loader/API tracing

For each loader and API handler, wrap the body in a span.

import * as Sentry from "@sentry/node";

export async function loader({ request }) {
  return Sentry.startSpan({ name: "loader: dashboard", op: "function" }, async () => {
    return { /* ... */ };
  });
}

Auto-injection is out of scope; provide a snippet, recommend wrapping the 5-10 slowest loaders (cross-reference with audit-bundles perf hotspots).

Step 4: Web Vitals on the client

Use the framework hook from a component mounted by a shared shell:

import { useWebVitals } from "@pracht/core";

export function Vitals() {
  useWebVitals((metric) => {
    navigator.sendBeacon?.(
      "/api/telemetry/vitals",
      JSON.stringify({ name: metric.name, value: metric.value, id: metric.id, path: location.pathname }),
    );
  });
  return null;
}

The hook is safe during SSR, lazy-loads web-vitals after mount, and shares a single observer set across callers. No separate dependency, hydration guard, or client-only module is needed, and apps that never call it ship no metrics runtime.

Step 5: Beacon endpoint

// src/api/telemetry/vitals.ts
import type { ApiRouteArgs } from "@pracht/core";

export async function POST({ request }: ApiRouteArgs) {
  const body = await request.text();
  // Forward to your destination (Sentry, Honeycomb, custom store).
  // Keep body small; do not block on the upstream.
  console.log("vitals", body);
  return new Response(null, { status: 204 });
}

For Sentry users, Sentry's browser SDK can capture Web Vitals natively — prefer that over a custom beacon if you've gone the Sentry route.

Step 6: Agent traffic (capability apps only)

Skip when pracht inspect agents --json reports an empty capabilities list. That only means there are no capability operations to audit; llms.txt, MCP, or Web Bot Auth may still expose other agent-facing surfaces. When capabilities do exist, every dispatch already emits a structured CapabilityAuditEvent — nothing is instrumented per capability, a sink just has to be registered.

import { addCapabilityAuditListener } from "@pracht/core/server";

const stopAuditLog = addCapabilityAuditListener("audit-log", (event) => {
  console.log(
    JSON.stringify({
      msg: "capability",
      at: new Date().toISOString(),
      capability: event.capability,
      effect: event.effect,
      transport: event.transport, // "http" | "webmcp" | "mcp" | "server"
      via: event.via, // causal transport for nested invokeCapability()
      outcome: event.outcome, // "ok" or the envelope error code
      status: event.status,
      durationMs: Math.round(event.durationMs),
      agent: event.agent?.agentDomain ?? event.agent?.keyId ?? null,
    }),
  );
});

if (import.meta.hot) {
  import.meta.hot.dispose(stopAuditLog);
}

The OTel version records a counter and a histogram keyed on capability/transport/outcome, and backdates a span with startTime: Date.now() - event.durationMs (the dispatch has already finished when the sink runs). The full snippet is on the agent-trust docs page.

Import the module from an eagerly loaded server module. The adapter's configured createContextFrom module is one portable option: add import "./audit.ts" there so the generated entry registers the sink before request handling. A custom server entry can import it directly. Do not rely on an unrelated route, API route, middleware, or src/server/ registry module; those modules are lazy and can miss earlier capability calls.

Key properties to state when scaffolding this:

  • Sinks are invoked synchronously, so keep work before the callback returns or reaches its first await cheap. A returned promise is never awaited, and a synchronous throw is swallowed (first failure per sink reported via console.warn, naming it), so a broken exporter cannot fail the call.
  • Always pass a stable name as the first argument. Registering the same name again replaces that sink, which is what keeps a module-scope call safe under dev HMR: Vite re-executes importers on every save, so an unkeyed registration would add a fresh closure per keystroke and deliver every event N times. Never compute the name. Register the returned unsubscribe with import.meta.hot.dispose() too, so removing the module or renaming the sink cannot leave the old name active until the dev server restarts.
  • setCapabilityAuditHook() is a single slot — a second call replaces the first. Use addCapabilityAuditListener() whenever more than one sink exists; it returns an unsubscribe handle that removes only its own registration.
  • Warning suppression is per named registration, so differently named sinks still report independently when they reuse one callback.
  • Delivery snapshots the registered sinks before callbacks run. Adding or replacing a sink from inside a callback takes effect on the next dispatch, so the current event is never delivered twice to one name.
  • A sink's returned promise is handed to the request's waitUntil(), so an async sink finishes after the response on every adapter (Cloudflare ctx.waitUntil, Netlify/Vercel context.waitUntil, Node's shutdown drain) and its rejection is reported. A batching exporter that flushes on its own schedule should be flushed from middleware with args.waitUntil(exporter.flush()) — never reach for context.executionContext.waitUntil.
  • Audit events cover dispatch only. A cross-origin 403, an unknown-capability 404, and an unknown MCP tool name all return before dispatch and emit nothing, so do not build a reconnaissance alert on these events — use the HTTP access log for that.

In dev the same events are already collected: the Agents section of /_pracht shows recent dispatches with transport, via, verified identity, outcome, and duration, and /_pracht.json exposes all of them under agentTraffic. The page counts verified identities, MCP, and MCP-caused composition as agent-attributed; shows top-level unsigned HTTP, HTTP-caused composition, and client-declared WebMCP markers separately as unverified client dispatches; and hides only invokeCapability() work with no served-request provenance behind a first-party toggle, so the panel's visible count can be lower than the sink's. Counts and empty-state conclusions only cover the retained window when older events have been dropped. Use it to confirm the sink sees what the panel sees before wiring a paid backend. Adapter-owned dev servers do not register this middleware: on Cloudflare workerd, /_pracht and /_pracht.json return 404. Validate the sink from its own output there; a missing panel is not a failed audit hook.

Step 7: Sampling and PII

  • Set SENTRY_TRACES_SAMPLE_RATE to a small number (0.05–0.10) in production.
  • Scrub auth headers and cookies from breadcrumbs:
    Sentry.init({ beforeSend(event) { delete event.request?.headers?.cookie; return event; } });
  • Never send loader return values verbatim — they often contain user data.

Step 8: Verify

  • Trigger a deliberate error in dev and confirm it lands in Sentry/OTel.
  • Open a route, check the Web Vitals beacon fires (Network tab).
  • If an audit sink was added: call a capability in dev and confirm the event reaches both the sink and, when the adapter exposes it, the Agents panel at /_pracht. On Cloudflare's adapter-owned dev server, validate the sink output directly because the panel does not exist.
  • Confirm pnpm test and pnpm e2e still pass.
  • Run pracht typegen if any routes were added (the beacon API route does not affect page-route types, but re-run when in doubt).
  • Run pracht verify --json and confirm no failures.

Rules

  1. Confirm adapter compatibility before installing the SDK package (Sentry has separate packages per runtime), and never scaffold a pattern the runtime can't actually run — see the Cloudflare/Vercel-edge caveat in Step 2. Read SENTRY_DSN and friends via serverEnv inside functions, never process.env at module scope.
  2. Top-level middleware in defineApp is a name→path registry, not an ordered chain. Place "observability" first in every group({ middleware: [...] }) and in api.middleware so it wraps the rest.
  3. Web Vitals only matter for SSR/SSG/ISG routes that hydrate; SPA-only routes still benefit but the values reflect the post-bootstrap state.
  4. Sample traces (≤ 10%) in production; full sampling in dev.
  5. Never send raw cookies, auth headers, or full loader payloads to a third-party SaaS. The same applies to audit events: log the capability name, effect, transport, outcome, and agent domain — not the capability's input, which is application data.
  6. Keep the synchronous part of an audit sink cheap: no CPU-heavy work or synchronous network call. Returned promises are fire-and-forget; the runtime does not await them or catch their rejections.

$ARGUMENTS

Download now (ZIP)

Does it work?

Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.

No reviews yet

People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.

  • It worked
  • Partly
  • It didn't
What did you use it for? e.g. Reviewing pull requests in a Rails app

Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.

Your trail

Files you open will collect here, so you can find them again.

Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.

Now do this for your own.

Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.