CLAUDE.md

Project Overview

Unofficial, community-maintained Ruby SDK for Claude Agent (gem: claude-agent-sdk). Wraps the Claude Code CLI as a subprocess, communicating via stream-JSON over stdin/stdout. Requires Ruby 3.2+ and Claude Code CLI 2.0.0+.

Common Commands

RUN_INTEGRATION=1 bundle exec rspec         # Run the real-CLI integration suite too (self-skips if `claude` CLI or ANTHROPIC_API_KEY is absent)

Architecture

Layered Design

User code
  ├── ClaudeAgentSDK.query()     ← One-shot/streaming, defined in lib/claude_agent_sdk.rb
  └── ClaudeAgentSDK::Client     ← Bidirectional sessions, defined in lib/claude_agent_sdk.rb
        │
        ▼
      Query (lib/claude_agent_sdk/query.rb)
        - Bidirectional control protocol (control_request / control_response routing)
        - Hook callback dispatch with typed inputs (PreToolUseHookInput, etc.)
        - Permission callback handling (can_use_tool)
        - SDK MCP server request routing (tools/call, resources/read, prompts/get)
        - Message queue (Async::Queue) separating control messages from SDK messages
        - When stdin may close: Query::RunLifecycle (lib/claude_agent_sdk/query/run_lifecycle.rb)
        │
        ▼
      SubprocessCLITransport (lib/claude_agent_sdk/subprocess_cli_transport.rb)
        - Spawns `claude` CLI via Open3.popen3
        - CLI command built from ClaudeAgentOptions by CommandBuilder (lib/claude_agent_sdk/command_builder.rb)
        - Reads stdout as newline-delimited JSON, writes stdin for streaming mode
        - Stderr handling in a separate Thread
        │
        ▼
      Claude Code CLI (external Node.js process)

Two API Entry Points

• query() — Simple function interface, run in its own Async { }.wait. Like Client, it always uses streaming mode with a Query handler and the control protocol, so hooks, can_use_tool and SDK MCP servers work (an optional transport: replaces the default SubprocessCLITransport). It writes the prompt (a String, or an Enumerator streamed in the background), then closes stdin. When hooks / can_use_tool / SDK MCP servers still need the channel it waits for the end of the run (Python #1279), which Query::RunLifecycle decides (see Run Lifecycle below). The difference from Client is lifecycle, not protocol: no follow-up queries and no outbound control calls (interrupt, set_model, ...). ClaudeAgentSDK.ask is a thin wrapper over query() that returns the last ResultMessage (raising CLIConnectionError if none arrives) and adds no error policy of its own.
• Client — Full-featured bidirectional sessions. Accepts optional transport_class (defaults to SubprocessCLITransport) and transport_args for custom transports. Creates transport, instantiates a Query handler that runs read_messages in an async task, routes control messages internally, and exposes SDK messages via Async::Queue. Supports hooks, permission callbacks, SDK MCP servers, interrupt, model switching, and file rewind.

Both are lifecycle adapters over the @api private SessionAssembly (lib/claude_agent_sdk/session_assembly.rb; Dispatch, in the same file, holds a session's observers / callback_scheduling / callback_wrapper so no call site passes them by hand — Client keeps one for its lifetime and puts the observers it resolves in on each connect, so each notification reads the current ones): the one shared implementation of acquisition (materialize a store-backed resume → repoint the options → construct and connect the transport → the only Query.new → transcript mirror → start → handshake), prompt writes, delivery (the only receive loop, #deliver) and disposal (#close_resources), which never notifies on_error or on_close (it does dispatch on_message, in #deliver) and rescues nothing — when those two fire, which error wins and whether stdin closes stay in the two adapters, and so does Client's state across a teardown (#close_resources yields once both closes are behind and before the temp dir is removed, where Client marks itself disconnected; a session whose teardown raised is kept until a disconnect completes; spec/unit/client_teardown_interleavings_spec.rb) (spec/unit/entry_point_lifecycle_spec.rb pins those differences; spec/unit/entry_point_parity_spec.rb runs a set of scenarios through both and pins them as identical).

Run Lifecycle (when stdin may close)

Query::RunLifecycle (lib/claude_agent_sdk/query/run_lifecycle.rb, @api private) owns the one decision when may stdin close?: the CLI's session_state_changed idle after a result (the transport sets CLAUDE_CODE_SDK_READS_SESSION_STATE=1, and Query drops the sdk_host_only frames that brings), bounded between turns by CLAUDE_CODE_PRINT_BG_WAIT_CEILING_MS (10 min; a main-thread turn, requires_action or a tracked agent stops the clock); from a CLI that sends no state, the first result with no tracked task in flight (the #1088 ledger).

Query only feeds it: frame for every frame the read loop does not route elsewhere (after the mirror flush for a result, before the sdk_host_only drop), message_will_write (after serialization, before the write), wait, stdin_closing, empty_input (final only — the #1226 trade-off), reader_gone. It supplies the timer (sleeper:, by default Query#sleep_on_read_task: a child of the read task, never a spawn_task entry) and three live, non-suspending predicates (bidirectional_needs:, closed:, control_requests_in_flight:). close_now never calls into it — it can run off the reactor — and the lifecycle is reactor-only, with no mutex.

The orderings live inside the module, and its class comment states them: a run is a per-run RunEnd, marked ended before its waiters are signalled and before the sleeper is stopped (each of those can hand the reactor to an input writer, whose message must then get a run of its own); a ceiling that wakes checks its generation, detaches, then acts, and re-arms instead of ending while a control request is in flight; every exit path clears the sleeper. spec/unit/query/run_lifecycle_spec.rb tables them through the interface with a FakeSleeper (spec/support/fake_sleeper.rb; Query.new(sleeper:) takes one too); specs read query.run_lifecycle (ended?, final?, ceiling_armed?, inflight?, session_state), never its instance variables. It is the next TLA+ target (formal/tla/README.md).

CLI Discovery / Installer

SubprocessCLITransport#find_cli probes, in order: CLAUDE_CLI_PATH (absolutized against the process cwd before use — the CLI is later spawned with chdir: options.cwd, where a relative path would name a different file) → CLIInstaller.installed_path (project-local vendored binary, deliberately ahead of PATH so a pinned install is hermetic) → claude on the process PATH, searched by the SDK itself (#executable_on_path; no which — it would be found through PATH too, relative entries included, and its answer would have to be believed; with no PATH at all, DEFAULT_DISCOVERY_SEARCH_PATH, Ruby's default without its .) → common install locations.

One absolute path, settled in #initialize and again at the top of #connect (#settle_cli_path, from @given_cli_path — the lookup happens when spawn used to do it): the version probe runs in the process cwd and the spawn in options.cwd, so both must be handed the same absolute path. The input is options.cli_path || find_cli (any falsy value means discovery), .to_s (it may be a Pathname). A path with a separator is anchored to the process cwd — an absolute one is kept as given, a leading ~ becomes the home directory — and never normalized: link/.. must resolve through the filesystem, so no File.expand_path (it collapses .. lexically, which names another file when link is a symlink; CLAUDE_CLI_PATH's branch still uses it). A bare name is searched by the SDK itself (#executable_on_path: an executable regular file; relative entries, . and the empty entry are anchored to the process cwd, not skipped; a ~ or ~/x entry is the home directory, as in Ruby's own command lookup, so deliberately not one of them — #path_entry_dir). Each lookup uses the PATH of the mechanism it replaced: discovery the process's (what which claude saw), an explicit bare cli_path the one the child will get (#spawn_search_path: options.env's, else the process's, else Ruby's built-in default — what spawn searched). A path that cannot be settled is kept as given and the SDK never executes it: #connect skips the probe and, when that path is the program of the argv, raises CLINotFoundError through the same Errno::ENOENT mapping as a missing file (the argv of a subclass whose #build_command wraps the CLI in another program is spawned as built). Such a subclass — detected in #connect because the argv does not start with the settled path — gets cli_path as the caller gave it, not settled: it names a file on the far side (docker exec, ssh), where a path anchored to this host's cwd or found on its PATH would not exist.

CLIInstaller (lib/claude_agent_sdk/cli_installer.rb) downloads a pinned CLI into vendor/claude (relative to CLIInstaller.root when set — the Railtie sets it to Rails.root — else to Dir.pwd, resolved at call time; root is a plain frozen-String ivar, absolutized on set, read lock-free) for Docker/CI deploys: dist-tag → concrete version, manifest SHA-256 + declared size, streamed download (bounded by that size) to a random-named sibling temp file (O_EXCL), verify, chmod 0755, atomically write VERSION = version + verified checksum, and finally rename the temp file into place. Stdlib only; every failure (filesystem errors included) is wrapped in CLIInstallError with cause preserved.

Publication order is load-bearing: the rename is last precisely because nothing can fail after it. A failed upgrade (download, checksum, metadata, or the rename itself) leaves the previously installed binary intact and runnable, and the next install redoes it cleanly — the earlier rename-then-record order let a metadata failure delete the freshly renamed binary and take the working install with it. That same invariant is why discovery (installed_path / find_cli) is deliberately lock-free: a reader only ever sees a complete binary.

formal/tla/CLIInstaller.tla model-checks this ordering, the in-lock dist-tag resolve and the in-lock sweep (formal/tla/run.sh; see its README). Changing any of them means changing the model and run.sh's expectations in the same change.

The whole sequence runs under an exclusive flock on <dir>/.install.lock: sweep stale temp files (abandoned by an install killed before its ensure) → resolve the dist-tag → idempotency check → download → record → publish. Dist-tag resolution is deliberately inside the lock — resolving it outside let a slow installer read an old version, wait for a faster one to publish a newer one, and then downgrade it; the semantics are now "last resolver wins". Only the local version-format check runs before the lock, so malformed input never creates a directory. The idempotency shortcut re-hashes the binary against the recorded checksum and makes no network call (offline re-boot must work, given a pinned version); a mismatch or a legacy single-line VERSION triggers a clean reinstall.

Four nested modules keep it testable without an HTTP stubbing library: Http (all network IO, redirect/timeout/size caps), Platform (host probing: RUBY_PLATFORM mapping, musl, Rosetta 2), Release (dist-tag + manifest reads), Metadata (the VERSION file). Specs stub those modules directly.

SDK MCP Servers

Custom tools run in-process (no subprocess). The flow:

1. User defines tools via ClaudeAgentSDK.create_tool → returns SdkMcpTool
2. create_sdk_mcp_server wraps tools in SdkMcpServer, which creates dynamic MCP::Tool subclasses and delegates to the official MCP::Server
3. The server config hash ({ type: 'sdk', instance: server }) is stored in ClaudeAgentOptions.mcp_servers
4. Client.connect extracts SDK server instances and passes them to Query
5. Query.handle_control_request dispatches mcp_message subtypes to the server's list_tools / call_tool / etc.

Message Flow

CLI outputs newline-delimited JSON. SubprocessCLITransport.read_messages parses it and yields raw hashes. In both entry points, Query.read_messages intercepts control_response and control_request types, putting regular messages on @message_queue. MessageParser.parse converts raw hashes into typed objects (UserMessage, AssistantMessage, SystemMessage, ResultMessage, StreamEvent).

Control Protocol

Only active in streaming/Client mode. Uses Async::Condition for request-response coordination:

• Outbound requests (SDK → CLI): send_control_request writes JSON, waits on condition, returns response
• Inbound requests (CLI → SDK): handle_control_request dispatches to can_use_tool, hook_callback, or mcp_message handlers, writes response back

The outbound waiter protocol (execution mode detected before the write, registration atomic with the stream-error check, slot checked before every wait, level-triggered ThreadWaiter) is model-checked by formal/tla/ControlProtocol.tla; keep it in step with send_control_request / await_control_response.

Observer / Instrumentation

Optional observability layer for tracing agent sessions (e.g., Langfuse via OpenTelemetry).

• Observer module (lib/claude_agent_sdk/observer.rb) — base interface with no-op defaults: on_user_prompt(prompt), on_message(message), on_error(error), on_close
• OTelObserver (lib/claude_agent_sdk/instrumentation/otel.rb) — emits OTel spans with gen_ai.* + OpenInference semantic conventions. Lazy-requires opentelemetry-api; not loaded unless user explicitly require 'claude_agent_sdk/instrumentation'
• Observers registered via ClaudeAgentOptions.new(observers: [...]). Supports callable factories (lambdas) for thread-safe global defaults in Rails
• resolve_observers materializes callables into fresh instances per query/session; notify_observers calls methods with rescue StandardError so observers never crash the pipeline
• on_user_prompt is called before the prompt is sent to stdin (before CLI responds with InitMessage), so the OTel observer buffers it and applies to the root span in start_trace

SessionStore / Transcript Mirroring

Optional adapter for mirroring session transcripts to external storage (the subprocess still writes the local-disk transcript; the store gets a secondary copy).

• SessionStore (lib/claude_agent_sdk/session_store.rb) — only #append and #load are required; the SDK probes optional methods via SessionStore.implements? (duck typing — adapters need not subclass)
• Keys/entries cross the adapter boundary as Hashes with string keys (e.g., { 'project_key' => ..., 'session_id' => ... })
• TranscriptMirrorBatcher buffers transcript_mirror stdout frames and flushes to #append per-turn (batched, default) or near-realtime (eager); StandardError failures retry with backoff but never raise — they surface as MirrorErrorMessage (process-exit exceptions — exit, Interrupt, signals — from a store adapter propagate)
• Retried batches may overlap prior writes, so adapters should dedupe by entry["uuid"] (with a fiber-native :inline adapter, a cooperatively cancelled append may also be half-applied — same dedupe covers it)
• lib/claude_agent_sdk/testing/session_store_conformance.rb provides a shared conformance suite for custom adapters

Session Management

sessions.rb / session_resume.rb / session_mutations.rb implement session listing (SDKSessionInfo), resume (can materialize the transcript from a SessionStore when the local file is absent), forking, and mutations.

The public functions in lib/claude_agent_sdk.rb (list_sessions, rename_session, …) route on an optional session_store: — nil to the disk implementation (Sessions.list_sessions), a store to the store one (Sessions.list_sessions_from_store). The root-level *_from_store / *_via_store twins are deprecated shims (issue #126; kept through 1.x, removed in 2.0) that warn once via Deprecation.warn_once and call the store implementation directly.

FiberBoundary (fiber safety)

async installs a Fiber scheduler, but most Ruby libraries (pg, mysql2, ActiveRecord pools) key state on Thread.current and are thread-safe, not fiber-safe. FiberBoundary.invoke (lib/claude_agent_sdk/fiber_boundary.rb) hops every user-supplied callback (tool handlers, hooks, permission callbacks, message blocks, observers) to a plain thread before invoking it. Control-request callbacks (hooks, can_use_tool, SDK MCP tool/resource/prompt handlers) go through FiberBoundary.invoke_callback: a SystemExit / SignalException raised by the callback is carried out of the hop, Query#handle_control_request writes the normal error response, then re-raises the original — the CLI always gets its answer and process termination is never swallowed (#119). Consequence: the thread hop severs break/return/next from the surrounding method — SDK loops yielding user callbacks must keep loop control outside the invoked block (see SessionAssembly#deliver, the receive loop behind query() and Client#receive_messages / #receive_response; user break is bridged via .invoke_iteration).

Opt-in ClaudeAgentOptions#callback_scheduling: :inline (issue #47) skips the hop and runs callbacks in place on the reactor fiber — for hosts that are fiber-isolated end to end (solid_queue fiber workers with IsolatedExecutionState.isolation_level = :fiber). The mode is plumbed per-call (Query kwarg; the SDK-MCP dispatch path carries it via fiber storage — Fiber[FiberBoundary::DISPATCH_KEY] — so a server shared by concurrent sessions is never mutated), never via a thread-local. Timeout-bounded store-adapter calls hop by default regardless of session mode — unless the ADAPTER declares itself fiber-native via an optional callback_scheduling method returning :inline (probed by SessionStores.store_callback_scheduling at batcher construction / resume-materialization entry; issue #47 phase 3), in which case they run in place under a cooperative timeout that still surfaces as JoinTimeout. Inline hook timeouts are cooperative (with_timeout). Public ClaudeAgentSDK.offload { } hops one heavy block manually. ClaudeAgentOptions#callback_wrapper (issue #47 phase 2) is optional middleware composed around every user-callback dispatch BEFORE the hop, so it runs on the callback's own execution context (worker thread in :thread, reactor fiber in :inline) — in Rails, ClaudeAgentSDK::Railtie.callback_wrapper (see Rails Integration; a bare executor.wrap lambda deadlocks under dev code reloading); it travels the SDK-MCP dispatch path inside the same fiber-storage scope as the mode, and also composes around timeout-bounded store-adapter dispatch (batcher appends, resume-materialization calls) inside the bound. SdkMcpServer#callback_scheduling= / #callback_wrapper= validate like their ClaudeAgentOptions counterparts (String coerced, invalid values raise).

Rails Integration

lib/claude_agent_sdk.rb requires railtie.rb only if defined?(Rails::Railtie) — nothing Rails-related loads otherwise (spec/unit/require_surface_spec.rb). ClaudeAgentSDK::Railtie sets CLIInstaller.root ||= Rails.root in an initializer ordered before: :load_config_initializers (so vendored-CLI discovery ignores the process cwd, and app initializers can still override), loads the claude_agent_sdk:install_cli rake task (lib/claude_agent_sdk/tasks/claude_agent_sdk.rake, also loadable from a plain Rakefile via require 'claude_agent_sdk/tasks'; guarded against double definition; it does not boot the app, so it anchors to Rails.root itself unless CLIInstaller.root is already set) and hosts Railtie.callback_wrapper: executor-wrapped callbacks, except with a locking executor (reloading enabled / allow_concurrency == false, mirroring railties' configure_executor_for_concurrency), where a callback thread entering the executor deadlocks against its waiting caller — there it runs unwrapped and releases AR connections; already-active executor (:inline) → straight through. The generator lives at lib/generators/claude_agent_sdk/install/ (explicit namespace: Thor would derive claude_agent_s_d_k). Rails specs live in spec/rails/, excluded from the default run by the root .rspec; run them with BUNDLE_GEMFILE=gemfiles/rails_8.gemfile bundle exec rspec --options spec/rails/.rspec (rails_7_1.gemfile for the floor).

Global Configuration

ClaudeAgentSDK.configure { |c| c.default_options = { ... } } (lib/claude_agent_sdk/configuration.rb) sets defaults merged into every request — the Rails-initializer pattern. Per-call ClaudeAgentOptions override the defaults.

Key Conventions

• Types use plain Ruby classes with attr_accessor and keyword args (no Struct/Data)
• Type definitions live in lib/claude_agent_sdk/types/, one file per area: base (Type, Type::OptionValue), attributes (attribute declarations and access rules, loaded by base), content_blocks, messages (incl. SystemMessage subclasses, rate-limit types), hooks (HOOK_EVENTS, inputs, outputs), permissions, mcp, option_values (value types ClaudeAgentOptions holds), sessions, options (ClaudeAgentOptions). Shared constants sit with the types that use them. types.rb only require_relatives the parts; add new types to the matching part file (each part requires base itself)
• Attributes are the names declared through attr_accessor / attr_reader / attr_writer on a Type subclass (Type overrides them to record the names, inherited) plus Type.declare_attributes for a hand-written reader (RateLimitEvent#data); the rules live in types/attributes.rb. #[], #[]= and camelCase method_missing reach attributes (and attr? predicates) plus any method user code defined (owner outside the ClaudeAgentSDK:: namespace and below Type in the ancestors, or a singleton method); positive registry hits are cached per class keyed by the caller's spelling. A name resolving to any other public method (to_h, freeze) is treated as undefined (#[] nil, #[]= no-op, camelCase NoMethodError, respond_to? false; 0.37 warned, 1.0 enforces). Types the user builds and passes IN (option values, HookMatcher, hook outputs, PermissionResult*, PermissionUpdate, PermissionRuleValue) declare strict_attributes: an unknown key on .new / #[]= raises ArgumentError naming the class, the key and the known attributes. CLI-parsed types stay lenient, and so does anything built through .from_hash / .wrap (a fiber-local lenient scope that also covers nested values) — SDK code hydrating CLI data into a strict type must use those, never .new. spec/unit/strict_attributes_spec.rb pins the strict classification; spec/unit/type_attribute_access_spec.rb pins that every public reader/writer is a declared attribute
• Hook inputs are typed classes inheriting from BaseHookInput; hook outputs use to_h for serialization with camelCase keys for CLI compatibility
• ClaudeAgentOptions is the central config object (~30 fields); uses dup_with for immutable-style updates
• to_h methods on config types convert Ruby snake_case to CLI camelCase (e.g., auto_allow_bash_if_sandboxed → autoAllowBashIfSandboxed)
• RBS signatures for the public API (docs/ + YARD without @api private) live in sig/ (sig/claude_agent_sdk.rbs; sig/claude_agent_sdk/types/ mirrors lib/claude_agent_sdk/types/) and ship in the gem. Any public API change must update sig/ in the same change and pass bundle exec rake rbs:validate rbs:test (the latter runs the suite under rbs's runtime type checker; examples that deliberately pass out-of-signature values carry rbs_incompatible: '<reason>' metadata and are skipped there). Never add signatures for @api private objects. Conventions: CONTRIBUTING.md, "RBS signatures"
• Tests use expect syntax only (no should), disable_monkey_patching! enabled
• Test helpers in spec/support/test_helpers.rb provide sample_* message fixtures and mock_transport