deploy-macos

Use this when standing up mesh-llm on a macOS machine — either installing a release or shipping a locally built dev binary to a remote Mac for testing.

This is the macOS counterpart to deploy-linux-gpu. The current binary embeds the staged llama.cpp runtime: the bundle is a single mesh-llm binary. There is no rpc-server, no llama-server, and no .dylib set anymore — if you see instructions mentioning those, they are outdated.

Related skills/docs:

• deploy-linux-gpu — remote Linux/CUDA nodes
• deploy-windows — Windows nodes
• mesh-join — creating/joining private and public meshes (tokens, NAT, multi-node)
• connect-agents — pointing Goose/Claude Code/OpenCode/Pi at a running mesh
• mesh/docs/USAGE.md — install details, service mode, model storage
• mesh/docs/CLI.md — full command and model-ref reference

The one rule that matters most

mesh-llm resolves and downloads the model itself. Pass --model <ref> and it fetches the GGUF into the standard Hugging Face cache on first use. Do NOT pre-download with hf/huggingface-cli, do NOT scp GGUFs around. (Only --gguf takes a local file path you manage yourself.)

Install path A: official release (most cases)

curl -fsSL https://raw.githubusercontent.com/Mesh-LLM/mesh-llm/main/install.sh | bash

The binary lands at ~/.local/bin/mesh-llm (may not be on a non-interactive SSH PATH — use the full path or bash -lc). Metal is the macOS backend; the installer picks it automatically.

To install as a per-user background service (launchd agent) in the same step:

curl -fsSL https://raw.githubusercontent.com/Mesh-LLM/mesh-llm/main/install.sh | bash -s -- --service

Service files: ~/Library/LaunchAgents/com.mesh-llm.mesh-llm.plist, shared env in ~/.config/mesh-llm/service.env, startup models in ~/.mesh-llm/config.toml.

Install path B: dev build to a remote Mac

Build and bundle locally (from the repo):

just release-build   # serious testing must use the release binary
just bundle          # /tmp/mesh-llm-bundle.tar.gz (single mesh-llm binary)

Ship and unpack:

scp -P <SSH_PORT> /tmp/mesh-llm-bundle.tar.gz user@host:
ssh -p <SSH_PORT> user@host 'mkdir -p ~/bin && tar xzf mesh-llm-bundle.tar.gz -C ~/bin --strip-components=1'

Check quarantine xattrs when another transfer path was used

This scp and tar path does not normally add or propagate com.apple.quarantine. If the binary came through another transfer path and the attribute is present, remove it before retrying:

if xattr -p com.apple.quarantine ~/bin/mesh-llm >/dev/null 2>&1; then
    xattr -d com.apple.quarantine ~/bin/mesh-llm
fi
codesign --verify --verbose=2 ~/bin/mesh-llm   # re-sign ad-hoc only if this fails

No Apple-issued identity is needed for SSH/interactive-shell launches.

Verify the version on the remote matches what you built:

~/bin/mesh-llm --version

Troubleshooting: macOS Local Network privacy (same-LAN join/split failures, relay-only peers)

This is not a deploy precondition. Launching an ad-hoc/linker-signed binary from SSH or an interactive shell is the normal lab path and has worked across our Macs repeatedly. Use this section when a same-LAN join or split fails, two Macs on the same LAN connect via relay instead of directly, or a GUI app / launch agent shows a Local Network alert — check this before diagnosing iroh.

macOS Local Network privacy is keyed to the responsible code and its code signature. Replacing or ad-hoc re-signing a development binary can therefore change the identity whose decision macOS remembers. A GUI app or launch agent may show a blocking Local Network alert on the logged-in desktop even though the same network is reachable from an interactive shell.

If that happens, for the affected app/binary identity:

1. Avoid re-signing it between attempts (a new signature can reset the remembered decision); for GUI apps/launch agents prefer a stable identity. Verify with codesign --verify --verbose=2 <path> and record codesign -dv --verbose=4 <path> 2>&1 in the lab evidence.
2. Launch it once from the same responsible app/service context intended for the test while a user is logged in at the Mac. Trigger a mesh join or mDNS operation, then accept the Local Network alert on that Mac. Repeat on every Mac participating in the test.
3. Check System Settings > Privacy & Security > Local Network. If the responsible app is listed, make sure access is enabled. macOS offers no supported command to query or reset an individual program from the undetermined state, so scripts must not claim this check passed merely because a raw UDP probe worked.
4. Authorize the exact launch context that will run the test. For a CLI-launched repro over SSH, use the held foreground TTY described by remote-observable-process. If a per-user launchd agent or GUI app will run the workload, launch that exact agent/app while a user is logged in and accept its prompt before diagnosis; an SSH TTY does not authorize a different responsible app.
5. Treat the network preflight as passed only after both nodes report an iroh direct path, observed_via_relay=false, and an observed_direct_remote_addr whose IP matches the intended LAN address of the peer. direct_addr_available=true alone only proves that some direct address was observed. An invitation containing a LAN candidate, a raw UDP probe, or a working relay path does not prove the deployed process has Local Network access.

If no alert appears, keep the process alive after its first failed LAN operation: macOS can fail to display the alert for very short-lived processes. Rule this out before debugging iroh candidate selection. See Apple TN3179, Understanding local network privacy.

Launch

Serve a model and join the public mesh:

mesh-llm serve --model unsloth/Qwen3.6-27B-GGUF:UD-Q4_K_XL --auto

• --auto discovers and joins the community mesh; local serving and mesh joining happen together.
• Without --auto (and without --join/--discover) you create a private mesh and an invite token is emitted — see the mesh-join skill.
• --model accepts catalog names, repo:QUANT, repo/file.gguf, or a full HF URL. --gguf /path/file.gguf serves a local file directly.
• API on :9337, management console on :3131 (override with --port / --console).

Notes / gotchas:

• Do NOT use --headless to "go quiet" — it only disables the embedded web UI and does nothing for backgrounding. For machine-readable output use --log-format json.
• Model load takes time. Poll /v1/models until your model appears before concluding anything is broken.
• For background test runs from an agent: bash -c 'nohup mesh-llm serve --model <ref> --auto > /tmp/mesh.log 2>&1 & disown'. For persistence across reboots, prefer the --service install.

Verify it's actually serving

# Ports bound
lsof -nP -iTCP:9337 -iTCP:3131 -sTCP:LISTEN

# Models (union of local + mesh peers)
curl -s http://localhost:9337/v1/models

# Status / peers
curl -s http://localhost:3131/api/status

# Inference — the returned "model" field tells you which node/model answered
curl -s http://localhost:9337/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{"model":"auto","messages":[{"role":"user","content":"hi"}],"max_tokens":16}'

To force your local model specifically, pass its exact id from /v1/models instead of auto.

Logs and state

• ~/.mesh-llm/runtime/<pid>/logs/skippy-native.log — embedded llama.cpp/skippy native logs. Check here first if a model fails to load.
• ~/.mesh-llm/key — persistent node identity.
• ~/.mesh-llm/config.toml — startup models and defaults for bare mesh-llm serve.
• HF cache (~/.cache/huggingface/...) — downloaded GGUFs; you generally never need to touch this.

Stop / clean up

mesh-llm stop        # scoped stop of tracked instances (preferred)
# emergency only: use the PID for the specific instance being validated
kill -9 <mesh-llm-pid>

A clean stop removes the instance runtime dir under ~/.mesh-llm/runtime/.

Troubleshooting