AIRelays

# AIRelays

> AIRelays is a local OpenAI-compatible HTTP server backed by your own subscription logins, with account balancing for OpenAI and Claude. Claude uses isolated local CLI profiles. It ships as a CLI/server package and as a cross-platform desktop tray app.

## Document Index

- README.md: overview, install (CLI and desktop), quick start, compatibility layer
- docs/getting-started.md: setup, custom CLI configuration, multi-account sign-in, verification
- docs/configuration.md: config file shape and environment overrides
- docs/security.md: relay auth, open local relay mode, Claude guardrails
- docs/api.md: routes, compatibility adaptations, provider limits
- docs/architecture.md: request flow and module boundaries
- docs/subscription-status.md: usage reporting for OpenAI and Claude
- docs/faq.md: common questions and limits
- docs/troubleshooting.md: symptoms, causes, fixes
- desktop/README.md: combined connection/access card, hide-on-blur dashboard, tray controls, build, supervision
- docs/adr/README.md: durable design decisions

---

## Overview (from README.md)

AIRelays is a local OpenAI-compatible HTTP server with provider-scoped runtimes.

- The default runtime uses an AIRelays-owned ChatGPT subscription login.
- The Claude runtime uses isolated local `claude` CLI profiles and retains the existing default CLI/token sign-in.
- AIRelays protects the relay with its own bearer token by default; open local relay mode (`--no-auth`) disables only the client-token gate.
- Traffic logs rotate hourly and at a size limit. By default, AIRelays keeps
  up to 7 days and 1024 MiB of managed traffic logs.
- AIRelays is an independent third-party project designed for a single user running a local relay for personal convenience. It is not a shared, pooled, multi-user, or resale service (see DISCLAIMER.md).

## Install

Headless relay and CLI (macOS, Linux), one line, no sudo; installs the newest PyPI release with uv or into a Python 3.11+ virtualenv linked as `~/.local/bin/airelays` (installs uv first when neither uv nor Python 3.11+ is present):

```bash
curl -fsSL https://raw.githubusercontent.com/lpalbou/AIRelays/main/scripts/install-headless.sh | bash
```

Or directly from PyPI:

```bash
python -m pip install airelays
```

Desktop app, one line, no sudo or admin; downloads the newest GitHub Release installer, verifies its SHA-256 digest, installs, and starts the app. macOS (Apple Silicon) installs `AIRelays.app` into `/Applications`; Linux (x86_64) installs the AppImage as `~/.local/bin/airelays-desktop` with a menu entry; Windows (x64) runs the NSIS setup silently for the current user:

```bash
curl -fsSL https://raw.githubusercontent.com/lpalbou/AIRelays/main/scripts/install-desktop.sh | bash
```

```powershell
irm https://raw.githubusercontent.com/lpalbou/AIRelays/main/scripts/install-desktop.ps1 | iex
```

Set `AIRELAYS_VERSION=X.Y.Z` to pin a release and `AIRELAYS_NO_LAUNCH=1` to skip starting the app. Run the same command again to update. Desktop builds are not notarized or code-signed; see docs/troubleshooting.md "Desktop app install" for Gatekeeper, SmartScreen, and AppImage FUSE notes. Intel Macs and non-x86_64 Linux use the headless installer.

Desktop app details: a Tauri tray app under `desktop/` with a dashboard for relay start/stop, auth and network modes, OpenAI and Claude sign-in/sign-out, per-account usage bars, a model list with copy-ready ids, live traffic, and diagnostics. The tray icon shows connection state and pulses on request activity; the app can start at login, starts the relay when it opens, and restarts a crashed relay automatically. Both installs share the same config (`~/.config/airelays`) and data (`~/.airelays`).

In Overview, **Connect Your App** combines the endpoint and relay key with authentication and network access controls. Access changes apply immediately and restart a running relay. Clicking outside the dashboard hides it while preserving the relay, page, and unsaved edits. On macOS and Windows, left-click the tray icon to reopen it; right-click opens the menu. **Open Dashboard** remains available in the tray menu on all platforms. Automatic hiding is disabled if tray initialization fails; opening the already-running macOS app also reopens the dashboard.

## First Run (CLI)

The CLI and desktop share saved accounts and configuration by default. Use
`--config ./relay.toml` consistently with `init`, sign-in, account commands,
and `serve` to select another configuration. Shared options (`--config`,
`--data-dir`, `--logs-dir`, `--auth-storage`, `--bearer-token-file`) work
before or after subcommands; a later explicit value wins. For example:
`airelays --config ./relay.toml claude accounts --json`.

OpenAI runtime:

```bash
airelays init
airelays login          # repeat with another account to enroll it too
airelays doctor
airelays serve --port 8080
```

Headless / server (SSH, no browser): `airelays login --device` — approve from a browser on any device. Auto-selected on SSH sessions and displayless Linux.

Claude runtime:

```bash
airelays init
airelays claude login       # repeat to add another account
airelays serve --port 8080
```

Claude headless: run `claude setup-token` on any browser-equipped machine, then `airelays claude set-token` on the relay machine (stores the default account token in a 0600 file that survives service managers and reboots). This token does not override browser-added profiles. List accounts with `airelays claude accounts`, renew one with `airelays claude login --replace ACCOUNT`, and sign out one with `airelays claude logout ACCOUNT`. Renewal suspends a managed profile until its original identity is verified; renewing `default` creates a new isolated profile and keeps the existing credentials. Signing out `default` also affects other tools using that CLI profile. Multiple-account logout requires a target or `--all`.

## Multiple Claude Accounts

`airelays claude login` delegates browser authentication to Claude Code in a
unique `CLAUDE_CONFIG_DIR`. Credentials and refresh remain CLI-owned. Metadata
lives under `data_dir/claude/accounts/<id>/account.json`, with a stable adjacent
`config/` directory. Do not move profile directories: macOS Keychain entries
are scoped to their paths. See docs/adr/0006-isolated-claude-subscription-accounts.md.

The desktop provides Add account, per-account usage, renewal, and sign-out.
Accounts join and leave routing without restart. Duplicate subscriptions count
once; a profile signed in to a different identity is not eligible for routing.
`[providers.claude].balance` defaults to `balanced`: prefer available execution
slots, then lowest fresh weekly usage. Without fresh usage, accounts take turns.
`round_robin` and `ordered` are alternatives. `max_concurrent_requests` applies
per account. Known exhausted short and model-specific windows are skipped;
account failures fail over only before response bytes reach the client.
`account_cooldown_seconds` defaults to 300 when no reset time is reported;
transient failures use a short cooldown. CLI profiles remain stateless for relay
requests, without conversation affinity.

Claude usage accepts `?provider=claude&all_accounts=true` for per-account
`{slug, email, status}` or `{slug, email, error}` entries, and
`?provider=claude&account=<email-or-id>` for one account. Usage caches, credential
resolution, and persisted usage-endpoint cooldowns are profile-specific.
`POST /v1/relay/accounts/refresh?provider=claude` respects upstream backoff.
`providers.claude.accounts` in relay status exposes profile readiness and request
cooldowns. Environment overrides are `AIRELAYS_CLAUDE_BALANCE` and
`AIRELAYS_CLAUDE_ACCOUNT_COOLDOWN_SECONDS`.

## Client Configuration

- Base URL: `http://127.0.0.1:8080/v1` (desktop app default port: 8317)
- API key: the relay token (`airelays token show`); any placeholder in open mode
- List accepted model ids: `airelays models` or `GET /v1/models`

## Multiple OpenAI Accounts

`/v1/models` returns the union of enrolled accounts' catalogs. Balancing,
conversation affinity, and failover stay inside each model's supporting
subset, even if every supporting account is cooling down. Per-model
`airelays.account_availability` reports supporting and total account counts;
`catalog_visibility` and `description` preserve upstream metadata, including
entries hidden in the upstream picker. Unlisted configured overrides remain
eligible across the pool, unless a discovered subset establishes support.

One person can enroll several of their own OpenAI subscriptions; `airelays login` is additive. By default the relay routes each request to the account with the most remaining quota in its longest usage window — the weekly budget; windows are identified by duration because which windows a plan reports is upstream policy — among those that serve the requested model (`balance = "balanced"`), so consumption equalizes as a percentage of each plan's capacity; `balance = "round_robin"` sends strictly equal request counts and `balance = "ordered"` drains the first account before the next. An account at its usage limit is benched until its window resets and rejoins rotation automatically; at launch the relay probes each account's capacity and model catalog so balancing is correct from the first request. Manage with `airelays accounts` (list, order, refresh), sign out one with `airelays logout <email>`. Conversations stick to the account that served their first turn.

## Routes

- `GET /v1/models` — models from all enabled providers, with an `airelays` extension block per record. OpenAI queries its authenticated upstream catalog using the installed Codex version (`client_version = "auto"`, tested floor `0.153.4`; the former `0.124.0` default also uses automatic mode). Claude queries its CLI initialization catalog without submitting generation prompts and exposes both aliases and concrete model ids. Configured overrides extend discovery; `extra_models` defaults to empty. `airelays.discovery_source` distinguishes provider catalogs from configured overrides, and `airelays.resolved_model` shows the concrete Claude model behind an alias. `GET /v1/models?refresh=true` bypasses provider caches, including all OpenAI account catalogs; the desktop Refresh button uses it, and the Models tab reloads every five minutes while reachable. Failed Claude discovery retains configured ids and the last successful catalog, with its error in relay status. Catalog listing does not guarantee a successful generation under current account limits.
- `POST /v1/responses`, `POST /v1/chat/completions`, `POST /v1/completions` — text generation
- `GET /v1/subscription/status` (alias `GET /v1/account/rate_limits`) — normalized usage; `?provider=claude`, `?account=`, `?all_accounts=true`, `?raw=true`
- `POST /v1/relay/accounts/refresh` — clear usage-limit holds and re-check capacity
- `GET /v1/relay/status` — diagnostics, provider readiness, `requests_total`; `?activity_only=true` returns only the counter for frequent polling
- `GET`/`PUT /v1/relay/logging` — traffic-log retention policy and usage
- `POST/GET/DELETE /v1/files...`, `POST/GET/DELETE /v1/conversations...` — local files and conversations
- `/no-tools/v1/*` — tool-disabled variants
- `GET /healthz` — minimal public health check

## Compatibility Layer

The verified upstream is the ChatGPT subscription backend, not the public platform API:

- `temperature`, `top_p`, `presence_penalty`, `frequency_penalty` are rejected by the upstream, and output-token limits (`max_tokens`, `max_completion_tokens`, `max_output_tokens`) are not honored; the relay strips them and discloses it via the `x-airelays-ignored-parameters` response header and a `compatibility_adaptation` traffic record. Generation uses the upstream's own defaults and runs to the model's natural stop. The Claude routes apply the same strip-and-disclose adaptation (the local `claude` CLI has no equivalent controls).
- `reasoning_effort` (chat) and `reasoning.effort` (responses) pass through verbatim to OpenAI models; on Claude models `reasoning_effort` maps to the CLI's `--effort` flag. Supported modes and defaults are read from provider catalogs when available and published in `/v1/models` under `airelays.reasoning`; they vary by model and can include `max` or `ultra`. Omitting effort uses the provider default; Claude can choose adaptively. An empty Claude modes list advertises no effort parameter.
- `store=true`, `n>1`, and `best_of`/`echo`/`logprobs`/`suffix` are rejected loudly instead of silently adapted.
- Failed upstream OpenAI calls are retried automatically with exponential backoff (default 3 retries, 5s/20s/60s; `retry_attempts = 0` disables) while no response byte has reached the client; each retry re-runs account failover. Final failures return OpenAI-shaped `{"error": {...}}` JSON with the real HTTP status and the upstream's own error code. After a stream has started, failures surface as an in-band `data: {"error": ...}` event (chat/completions) or verbatim `response.failed` events (responses passthrough).
- Claude runtime: discovered `claude:*` aliases and concrete `claude-*` ids, plus configured overrides; text chat/completions only, stateless, loopback-only, no tools/files/images. Structured outputs are supported on chat completions: `response_format` `json_schema`/`json_object` map to the claude CLI's `--json-schema` (native enforcement); supported types per model are published in `/v1/models` under `airelays.structured_output`.

## Subscription Usage

Claude reports explicit scoped limits, including Fable's weekly cap when
present, separately from all-model windows. The desktop groups accounts under
OpenAI and Anthropic headings. The **?** beside each account name opens details
on hover, click, or keyboard activation; Escape closes them. It names exhausted
scopes, exposes credit/spend details with their declared currency scale, and
does not infer missing balances. OpenAI preserves `model_usage`, credits,
spend controls, and the distinction between available and currently usable
limit-reset credits. Quota percentages are not token counts; the token
breakdown only covers responses observed through this relay.

Snapshots retain their fetch timestamp on cache reads. The Overview reloads
usage every five minutes and derives countdowns from absolute reset times.
Unknown or expired percentages display as awaiting fresh data, not zero.
Claude's five-minute cache and persisted upstream cooldowns protect its
usage endpoint; stale fallbacks explicitly report their age and reason.
Claude capacity observations older than fifteen minutes do not rank accounts;
known exhausted windows remain blocked until reset or newer capacity evidence.

`GET /v1/subscription/status` returns per-window `used_percent`, `window_label` ("5h", "weekly", derived from each window's duration), and reset times in one shape for both providers, plus credits, spend control, and any named extra quotas (for example code review) the plan reports. Which windows appear is plan-dependent upstream policy; only reported windows are returned. OpenAI reads the subscription usage surface; query one account (`?account=`) or all (`?all_accounts=true`, always the list shape — an account whose probe fails carries a per-account `error` instead of a `status`). Claude (`?provider=claude`) returns the 5-hour and weekly windows plus per-model weekly caps when reported; its upstream source is not a publicly documented API, so the relay caches it briefly and degrades gracefully. In the desktop app, an OpenAI account whose stored sign-in was invalidated upstream shows a "Sign-in expired" badge with a one-click "Sign in again" repair that refreshes the account slot in place.

## Security Defaults

- default listener `127.0.0.1:8080` (CLI) / `0.0.0.0:8317` (desktop, with a one-click loopback switch)
- protected routes `/v1/*` and `/no-tools/v1/*`; public: `/` and `GET /healthz`
- rate limit 120 requests/minute (burst 40), 8 concurrent requests per IP, temporary IP block after repeated bad tokens
- the Claude runtime is loopback-only and follows the relay's auth mode

## Configuration

Order: CLI flags → `AIRELAYS_*` environment variables → `~/.config/airelays/config.toml` → defaults. Notable keys: `[server] host/port`, `[security] require_bearer_auth`, `[logging] stream_lines`, `retention_days`, `max_total_mb`, and `max_file_mb`, `[providers.openai] enabled/balance/account_cooldown_seconds/retry_attempts/retry_backoff_seconds`, `[providers.claude] enabled/bin/models`. Saved log-directory retention policy overrides the three logging defaults. See docs/configuration.md for the full file shape and limits.

## Diagnostics

- `airelays logs` — inspect or apply traffic-log retention, including `--retention-days 30` for a month and `--max-total-mb` / `--max-file-mb` limits in MiB. The tray's Settings → Traffic log retention and GET/PUT `/v1/relay/logging` use the same saved policy. Defaults are 7 days, 1024 MiB total, and 50 MiB per file; `0` days disables the age limit only. Cleanup permanently removes oldest managed files, including eligible existing traffic logs. It excludes console output, uploads, and conversations. See docs/configuration.md, docs/api.md, and docs/troubleshooting.md for limits, API response fields, and cleanup-error recovery.

- `airelays status` — local config, token, and provider readiness
- `airelays doctor` — setup checks plus live upstream `/models` and a tiny `/responses` smoke request (`--skip-response` to skip)
- `airelays models` — model ids the running relay accepts, grouped by provider
- Traffic logs: hourly JSONL under `~/.airelays/logs`, with per-request records (tokens, status, serving account, adaptations)

## Paths

- config: `~/.config/airelays/config.toml`
- data: `~/.airelays` (logs, relay token, per-account auth slots, stored Claude token)

## Troubleshooting Pointers

- 401 then 429: wrong/missing relay token; repeated failures trigger a temporary IP block
- 422 on token-limit fields: the subscription backend does not accept them; remove the fields
- browser login URL only works on the relay's own machine; use `airelays login --device` on servers
- Claude not ready in network mode: the runtime is loopback-only; switch to loopback binding
- macOS "AIRelays is damaged" after a browser download: `xattr -dr com.apple.quarantine /Applications/AIRelays.app` (the one-line installer avoids this); Linux AppImage FUSE error: install libfuse2 or set `APPIMAGE_EXTRACT_AND_RUN=1`
- see docs/troubleshooting.md for full workflows