agentleFS
iwesysiwesys/IWE/.claude/skills/local-llm/SKILL.md

local-llm

Skill0 starsChanged 26 days agoMITView on GitHub

Local LLM stack on Mac (Apple Silicon, MLX) for privacy and fallback

5 other projects use a version of this.

Adopted by

Most recently taken up by

The file

We split this skill into the rules it gives an agent

2 rules. 0 are common, 1 is distinctive, and 1 is very specific to this project.

  • 0

    Common

    Most of the 3 edited versions we read keep these.

  • 1

    Distinctive

    This author's own choices. Worth a look.

  • 1

    Very specific

    Tied to their project's names, files or tools. Probably skip.

  • Common
  • Distinctive
  • Very specific
  1. Very specific:

    Verify JOB fit: privacy (offline, no telemetry) and fits Mac memory (min_ram_gb ≤ hardware).

    line 106
  2. Distinctive:

    Sending prompt contents to the cloud: when the local stack is chosen for privacy — the stack exists precisely to prevent this.

    line 114

“Common” is measured: kept by at least 6 in 10 of the 3 edited versions of this file we could read. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.

Keep what fits and drop the rest. Your version, ready for your agents.
Files
SKILL.md

Local LLM Stack (local-llm)

Scope: Natural-language entry point for the local LLM stack (install / start / status / switch model) plus a judgment layer for evaluating new model monitor findings. Not in scope: mechanics (installation, states, switching, archive) — those live in deterministic scripts (Distinction: Script ≠ Agent). The Skill calls them; it does not duplicate their logic. The source of truth for model statuses is the catalog and model-lifecycle.py. Stack JOB: privacy (tasks with personal data that cannot be sent to the cloud) and fallback mode (no network / cloud tokens exhausted). See ADR-001-local-llm-stack.md.

Bundle location

The control scripts are a self-contained bundle (each runs from its own directory). The path is resolved via an env variable; the default is extensions/local-llm/ inside the IWE installation:

BUNDLE="${IWE_LOCAL_LLM_DIR:-$HOME/IWE/extensions/local-llm}"
  • Stack venv: ~/.iwe-local-llm/.venv (installed by the installer).
  • Active model and server state: ~/.iwe-local-llm/ (active-model, server.pid, server.log).

When to use

  • "install local model" → hardware-matched installation (Step 1).
  • "show models" / "what models are available" → list by state (Step 2).
  • "start / stop" / "status" → server management (Step 2).
  • "switch model" / "switch to 14B" → change active model (Step 3).
  • "what new models are out" / "check new models" → monitor and judgment on adopt (Step 4).

Algorithm

Step 1 — installation (mechanics → script)

bash "$BUNDLE/install-local-llm.sh"          # recommended model for the hardware
bash "$BUNDLE/install-local-llm.sh" --max    # heaviest model that fits

The installer is idempotent: detect hardware → select model from catalog by memory → venv + mlx-lm → download → smoke test. Show the user the result (model name, confirmation that the server started).

Step 2 — start / status / list (mechanics → script)

bash "$BUNDLE/iwe-local-llm.sh" start    # start server (OpenAI-compatible, localhost:8080)
bash "$BUNDLE/iwe-local-llm.sh" status   # active model + server state
bash "$BUNDLE/iwe-local-llm.sh" models   # all models across four states + download status
bash "$BUNDLE/iwe-local-llm.sh" stop     # stop server
bash "$BUNDLE/iwe-local-llm.sh" test     # check compatible interface (SHIM_OK)

IWE tools connect to http://127.0.0.1:8080/v1/chat/completions using the same client as for the cloud.

Step 3 — switching models (mechanics → script)

bash "$BUNDLE/iwe-local-llm.sh" use <model-id>    # download + activate + restart
bash "$BUNDLE/iwe-local-llm.sh" pull <model-id>   # download only for testing (→ testing)
bash "$BUNDLE/iwe-local-llm.sh" archive <model-id># move to archive

use moves the previous active model to testing (it remains downloaded). Exactly one model is active at a time. Do not edit model-catalog.yaml by hand — use only iwe-local-llm.sh / model-lifecycle.py (preserves comments, maintains the single-active invariant).

Step 4 — monitoring new models (judgment → LLM work)

BUNDLE="${IWE_LOCAL_LLM_DIR:-$HOME/IWE/extensions/local-llm}"
PY3="$(bash "${IWE_SCRIPTS:-$HOME/IWE/scripts}/lib/find-python3.sh")" && "$PY3" "$BUNDLE/model-monitor.py"    # digest of trending models not yet in the catalog

The monitor only finds candidates. The adopt decision is the agent's judgment (this is the LLM work the Skill exists to support):

  1. Compare the candidate against the current active model on quality (LMArena / benchmarks, if the user provides a link or asks for a comparison).
  2. Verify JOB fit: privacy (offline, no telemetry) and fits Mac memory (min_ram_gb ≤ hardware).
  3. Give a recommendation: add to the catalog (model-lifecycle.py add) and pull for comparison, or skip with a reason.
  4. The decision to change the active model belongs to the user. The Skill recommends; it does not switch silently.

Anti-patterns

  • Duplicating mechanics in the Skill. States, switching, and archiving belong in the scripts. The Skill calls them; it does not rewrite them (OwnerIntegrity: catalog = source of truth).
  • Silently switching the active model based on a monitor finding. Adopt is the user's decision after a quality comparison.
  • Sending prompt contents to the cloud when the local stack is chosen for privacy — the stack exists precisely to prevent this.
  • Editing model-catalog.yaml by hand — breaks comments and the single-active invariant; use lifecycle commands only.
<!-- USER-SPACE --><!-- /USER-SPACE -->

Download now (ZIP)

Does it work?

Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.

No reviews yet

People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.

  • It worked
  • Partly
  • It didn't
What did you use it for? e.g. Reviewing pull requests in a Rails app

Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.

Your trail

Files you open will collect here, so you can find them again.

Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.

Now do this for your own.

Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.