mellea

<!--
AGENTS.md — Instructions for AI coding assistants (Claude, Cursor, Copilot, Codex, Roo, etc.)
-->

# Agent Guidelines for Mellea Contributors

> **Which guide?** Modifying `mellea/`, `cli/`, or `test/` → this file. Writing code that imports Mellea → [`docs/AGENTS_TEMPLATE.md`](docs/AGENTS_TEMPLATE.md).

> **Code of Conduct**: This project adheres to a [Code of Conduct](CODE_OF_CONDUCT.md). All contributors, including AI assistants, are expected to follow these community standards when generating code, documentation, or interacting with the project.

## 1. Quick Reference

**⚠️ Always use `uv` for Python commands** — never use system Python or `pip` directly.
- Run Python scripts: `uv run python script.py` (not `python script.py`)
- Run tools: `uv run pytest`, `uv run ruff` (not `pytest`, `ruff`)
- Install deps: `uv sync` (not `pip install`)
- The virtual environment is `.venv/` — `uv run` automatically uses it

```bash
pre-commit install                    # Required: install git hooks
uv sync --all-extras --all-groups     # Install all deps (required for tests)
uv sync --extra backends --all-groups # Install just backend deps (lighter)
ollama serve                          # Start Ollama (required for most tests)
uv run pytest                         # Default: qualitative tests, skip slow tests
uv run pytest -m "not qualitative"    # Fast tests only (~2 min)
uv run pytest -m slow                 # Run only slow tests (>5 min)
uv run pytest --co -q                 # Run ALL tests including slow (bypass config)
uv run ruff format .                  # Format code
uv run ruff check .                   # Lint code
uv run mypy .                         # Type check
```
**Branches**: `feat/topic`, `fix/issue-id`, `docs/topic`

## 2. Directory Structure
| Path | Contents |
|------|----------|
| `mellea/core/` | Core abstractions: Backend, Base, Formatter, Requirement, Sampling |
| `mellea/stdlib/` | Standard library: Sessions, Components, Context |
| `mellea/backends/` | Providers: HF, OpenAI, Ollama, Watsonx, LiteLLM |
| `mellea/formatters/` | Output formatters for different types |
| `mellea/templates/` | Jinja2 templates |
| `mellea/helpers/` | Utilities, logging, model ID tables |
| `cli/` | CLI commands (`m serve`, `m alora`, `m decompose`, `m eval`) |
| `test/` | All tests (run from repo root) |
| `docs/examples/` | Example code (run as tests via pytest) |
| `.agents/skills/` | Agent skills ([agentskills.io](https://agentskills.io) standard) |
| `scratchpad/` | Experiments (git-ignored) |

## 3. Test Markers
Tests use a four-tier granularity system (`unit`, `integration`, `e2e`, `qualitative`) plus backend and resource markers. The `unit` marker is auto-applied by conftest — never write it explicitly. The `llm` marker is deprecated; use `e2e` instead.

See **[test/README.md](test/README.md)** for classification rules, authoring guide, marker reference, CI tier map, and local workflow.

**Examples in `docs/examples/`** are opt-in — unlike `test/` files (auto-collected, default `unit`), examples require an explicit `# pytest:` comment to be collected. Files without this comment are silently ignored (they won't appear in skip summaries either). This is because examples have variable dependencies and limited setup:
```python
# pytest: e2e, ollama, qualitative
"""Example description..."""
```

**Notebooks in `docs/examples/notebooks/`** opt in through a `mellea` block in their own top-level notebook metadata (`{"markers": [...], "packages": [...]}`), since a notebook has nowhere to put a `# pytest:` comment. They are collected only when `--nbmake` is passed: `uv run poe nbtest`. A notebook without that block is skipped, and `test/test_example_collection.py` fails. `--nbmake-timeout` is per cell; a whole notebook is one pytest item bounded by `--timeout`, so raise both together.

⚠️ Don't add `qualitative` to trivial tests — keep the fast loop fast.
⚠️ Mark tests taking >1 minute with `slow`.

## 4. Agent Skills

Skills live in `.agents/skills/` following the [agentskills.io](https://agentskills.io) open standard. Each skill is a directory with a `SKILL.md` file (YAML frontmatter + markdown instructions).

**Tool discovery:**

| Tool              | Project skills    | Global skills       | Config needed                                                      |
| ----------------- | ----------------- | ------------------- | ------------------------------------------------------------------ |
| Claude Code       | `.agents/skills/` | `~/.claude/skills/` | `"skillLocations": [".agents/skills"]` in `.claude/settings.json`  |
| IBM Bob           | `.bob/skills/`    | `~/.bob/skills/`    | Symlink: `.bob/skills` → `.agents/skills`                          |
| VS Code / Copilot | `.agents/skills/` | —                   | None (auto-discovered)                                             |

**Bob users:** create the symlink once per clone:

```bash
mkdir -p .bob && ln -s ../.agents/skills .bob/skills
```

**Available skills:** `/audit-markers`, `/skill-author`

## 5. Coding Standards
- **Types required** on all core functions
- **Public API docstrings are published in the public API reference** — be specific and accurate.
- **Google-style docstrings** — `Args:` on the **class docstring only**; `__init__` gets a single summary sentence. Add `Attributes:` only when a stored value differs in type/behaviour from its constructor input (type transforms, computed values, class constants). See CONTRIBUTING.md for a full example. **No RST directives inside docstrings** — never use `Example::`, `.. deprecated::`, `:param:`, `:type:`, or other RST markup inside a docstring; Google-style sections (`Example:`, `Raises:`, etc.) use plain Markdown. **Code examples use triple-backtick fences** (` ```python `) — not `>>>` doctest prompts (output is not verified). **Inline code uses single backticks** (`` `name` ``) — never double backticks (`` ``name`` ``); Mellea uses Markdown-style docstrings where double backticks are RST syntax and render incorrectly. See CONTRIBUTING.md for the full rationale and examples.
- **Ruff** for linting/formatting
- Use `...` in `@generative` function bodies
- Prefer primitives over classes
- **Friendly Dependency Errors**: Wraps optional backend imports in `try/except ImportError` with a helpful message (e.g., "Please pip install mellea[hf]"). See `mellea/stdlib/session.py` for examples.
- **CLI command docstrings**: Typer command functions in `cli/` follow an enriched convention with `Prerequisites:` and `See Also:` sections — these feed the auto-generated CLI reference page. See [`docs/CONTRIBUTING_DOCS.md`](docs/CONTRIBUTING_DOCS.md) for the full pattern. Regenerate after changes: `uv run poe clidocs`. Test the generator: `uv run pytest tooling/docs-autogen/test_cli_reference.py -v`. Full pipeline docs: [`tooling/docs-autogen/README.md`](tooling/docs-autogen/README.md).
- **Backend telemetry fields**: All backends must populate `mot.generation.usage` (dict with `prompt_tokens`, `completion_tokens`, `total_tokens`), `mot.generation.model` (str), and `mot.generation.provider` (str) in their `post_processing()` method. These fields live on `mot.generation`, a `GenerationMetadata` dataclass. `mot.generation.streaming` (bool) is set in `astream()`; `mot.generation.ttfb_ms` (float | None) is stamped at the provider's first-chunk receipt inside `send_to_queue()` — backends set neither manually. `mot.generation.prompt_template` (str | None) and `mot.generation.prompt_template_variables` (dict | None) are set at render time in `_generate_from_context` (not `post_processing`), beside the early `model`/`provider` assignment, via `self.formatter.prompt_template_for(action)` — only on the chat path (`generate_from_raw` has no single unified template). Metrics are automatically recorded by `TokenMetricsPlugin`, `LatencyMetricsPlugin`, and `ErrorMetricsPlugin` — don't add manual `record_token_usage_metrics()`, `record_request_duration()`, or `record_error()` calls.
- **Adding or editing telemetry (spans/metrics)**: Telemetry is emitted by **hook-fired plugins**, not direct calls. Core fires lifecycle hooks; a `*TracingPlugin` in `mellea/telemetry/tracing_plugins.py` emits spans and a `*MetricsPlugin` in `mellea/telemetry/metrics_plugins.py` emits metrics, both subscribing to those hooks. Core does **not** call `start_*_span`/`finish_*_span` from `mellea/telemetry/tracing.py` directly (the only exception is sync code that can't fire paired hooks). Matching helper names in `tracing.py` is not enough — read the plugins, and use the existing one whose span shape matches yours as the template. Emitting a span needs a start/pre hook to open it and a matching end/post hook to close it; a hook that only fires at completion, with no paired opener, can feed a metric but cannot anchor a span.

## 6. Commits & Hooks
[Angular format](https://github.com/angular/angular/blob/main/CONTRIBUTING.md#commit): `feat:`, `fix:`, `docs:`, `test:`, `refactor:`, `release:`

Pre-commit runs: ruff, mypy, uv-lock, codespell, license-headers

**Pull request template**: opening a PR fills the body from [`.github/pull_request_template.md`](.github/pull_request_template.md), which ends with four type checkboxes - `Component`, `Requirement`, `Sampling Strategy`, `Tool`. If your PR adds or modifies one of those, check the matching box; the `PR Bot` workflow ([`.github/workflows/pr-update.yml`](.github/workflows/pr-update.yml)) then posts a comment with the type-specific review checklist from `.github/PULL_REQUEST_TEMPLATE/`:

| Checked box | Checklist template |
|-------------|--------------------|
| `Component` | `.github/PULL_REQUEST_TEMPLATE/component.md` |
| `Requirement` | `.github/PULL_REQUEST_TEMPLATE/requirement.md` |
| `Sampling Strategy` | `.github/PULL_REQUEST_TEMPLATE/sampling.md` |
| `Tool` | `.github/PULL_REQUEST_TEMPLATE/tool.md` |

This matters when a PR is opened outside the GitHub UI (`gh pr create --body`, from a fork, or by an agent): the template isn't applied automatically. When you open such a PR and it adds or modifies one of the four types, build the body from `.github/pull_request_template.md` with the matching box checked (if relevant) so the bot posts the checklist.

**Review states**: when reviewing a PR, see [CONTRIBUTING.md → Review States](CONTRIBUTING.md#review-states) for when to use `APPROVE` vs `REQUEST CHANGES` vs `COMMENT`.

For AI attribution trailers, see [Section 7 (AI Attribution)](#7-ai-attribution).

## 7. AI Attribution

Commits require a Signed-off-by trailer from the human author (added by running `git commit -s`). AI agents must not add a Signed-off-by in the tool's own name — instead, always add an `Assisted-by:` trailer to the commit footer:

```text
Assisted-by: Claude Code
Assisted-by: IBM Bob
```

Use the tool's common name (e.g., GitHub Copilot, Cursor, etc.).

## 8. Timing
> **Don't cancel**: `pytest` (full) and `pre-commit --all-files` may take minutes. Canceling mid-run can corrupt state.

## 9. Common Issues
| Problem | Fix |
|---------|-----|
| `ComponentParseError` | Add examples to docstring |
| `uv.lock` out of sync | Run `uv sync` |
| Ollama refused | Run `ollama serve` |
| Telemetry import errors | Run `uv sync` to install OpenTelemetry deps |
| Silent empty strings from async backends | Check for `asyncio.gather(..., return_exceptions=True)` — exceptions become values silently; use `return_exceptions=False` unless callers explicitly handle `BaseException` values |
| GitHub Actions workflow injection warning | Never use `${{ expression }}` directly inside `run:` shell commands — always route through `env:` (`env: MY_VAR: ${{ expr }}` then `"$MY_VAR"` in the script). This rule applies only to `run:` steps; `${{ }}` in `if:` conditions and `with:` action inputs is fine. |
| zizmor `ref-version-mismatch` on a workflow your PR didn't touch | A SHA pin's comment names a moving tag (`# v7`) that upstream has re-pointed. Comment with the exact release the SHA is tagged as (`@<sha> # v7.0.1`); the `action-pin-full-version` pre-commit hook enforces this. |

## 10. Self-Review (before notifying user)
1. `uv run pytest test/ -m "not qualitative"` passes?
2. `ruff format` and `ruff check` clean?
3. New functions typed with concise docstrings?
4. Unit tests added for new functionality?
5. Avoided over-engineering?
6. If the diff adds `raise` statements to library code (`mellea/` but not `test/`), **or adds a class/function to `__all__`**, run the docstring quality gate before pushing:
   ```bash
   uv run python tooling/docs-autogen/build.py  # generate docs/docs/api first
   uv run python tooling/docs-autogen/audit_coverage.py --docs-dir docs/docs/api --quality --fail-on-quality --threshold 100
   ```
   Every new `raise` in a public function requires a matching `Raises:` entry, and every `Returns:` type annotation must match the function's actual return type. Adding a class to `__all__` promotes its method docstrings into quality-gate scope — the gate will start enforcing them. The `build-and-validate` CI job enforces both with `--fail-on-quality`.

## 11. Writing Tests

See **[test/README.md — Authoring guide](test/README.md#authoring-guide)** for the full authoring guide (naming, fixture discipline, mock discipline, assertion style).

- Place tests in `test/` mirroring source structure
- Name files `test_*.py` (required for pydocstyle)
- Use `gh_run` fixture for CI-aware tests (see `test/conftest.py`)
- Mark tests checking LLM output quality with `@pytest.mark.qualitative`
- If a test fails, fix the **code**, not the test (unless the test was wrong)
- **Static type checks** live in `test/typing/` as `check_*.py` files (not `test_*.py`, so pytest skips them). They use `typing.assert_type` inside function bodies to verify overload resolution and generic parameterization — e.g., that `session.aact(..., await_result=True)` narrows to `ComputedModelOutputThunk[str]`. Verification happens via `uv run mypy .`; the functions are never executed. Add a new `check_*.py` here when introducing or modifying `@overload` signatures or generic type parameters on public APIs.

## 12. Writing Docs

If you are modifying or creating pages under `docs/docs/`, follow the writing
conventions in [`docs/CONTRIBUTING_DOCS.md`](docs/CONTRIBUTING_DOCS.md).
Key rules that differ from typical Markdown habits:

- **No H1 in the body** — Docusaurus renders the frontmatter `title` automatically;
  a body `# Heading` produces a duplicate title in the published site
- **Cross-doc links: relative, with the extension** — use `../concepts/requirements-system.md`,
  not `../concepts/requirements-system`, and never a root-absolute `/concepts/requirements-system`.
  With the extension Docusaurus resolves against the **source file**, so a rename breaks the
  build loudly and the link also works when browsing on GitHub. Extensionless links are raw
  URL paths checked only against the route table, and root-absolute ones are version-blind —
  from a page in `docs/docs/` (the `next` version) they resolve to the *released* version's
  route. Numbered files keep their prefix in the link
  (`../tutorials/02-streaming-and-async.md`). The Docusaurus build does **not** reject
  extensionless links, so check with the `rg` sweep in
  [`docs/CONTRIBUTING_DOCS.md`](docs/CONTRIBUTING_DOCS.md) → Links.
- **Frontmatter required** — every page needs `title` and `description`; add
  `sidebar_label` if the title is long
- **markdownlint gate** — run `npx markdownlint-cli2 "docs/docs/**/*.md"` and fix
  all warnings before committing a doc page
- **Verified code only** — every code example must be checked against the current
  mellea source; mark forward-looking content with `> **Coming soon:**`
- **No visible TODOs** — if content is missing, open a GitHub issue instead

## 13. Feedback Loop

Found a bug, workaround, or pattern? Update the docs:

- **Issue/workaround?** → Add to [Section 9 (Common Issues)](#9-common-issues) in this file
- **Usage pattern?** → Add to [`docs/AGENTS_TEMPLATE.md`](docs/AGENTS_TEMPLATE.md)
- **New pitfall?** → Add warning near relevant section

## 14. Working with Adapter Functions

Adapter functions are specialized LoRA/aLoRA adapters that add task-specific capabilities (RAG evaluation, safety checks, calibration, etc.) to Granite models. Mellea handles adapter loading and input formatting automatically — you just call the right function.

### Using Adapter Functions in Mellea

**Prefer the high-level wrappers** in `mellea/stdlib/components/intrinsic/`. These handle adapter loading, context formatting, and output parsing for you:

| Module | Function | Description |
|--------|----------|-------------|
| `core` | `check_certainty(context, backend)` | Model certainty about its last response (0–1) |
| `core` | `requirement_check(context, backend, requirement)` | Whether text meets a requirement (0–1) |
| `core` | `find_context_attributions(response, documents, context, backend)` | Sentences that influenced the response |
| `rag` | `check_answerability(question, documents, context, backend)` | Whether documents can answer a question (0–1) |
| `rag` | `rewrite_question(question, context, backend)` | Rewrite question into a retrieval query |
| `rag` | `clarify_query(question, documents, context, backend)` | Generate clarification or return "CLEAR" |
| `rag` | `find_citations(response, documents, context, backend)` | Document sentences supporting the response |
| `rag` | `flag_hallucinated_content(response, documents, context, backend)` | Flag potentially hallucinated sentences |

```python
from mellea.backends.huggingface import LocalHFBackend
from mellea.stdlib.components import Message
from mellea.stdlib.components.intrinsic import core
from mellea.stdlib.context import ChatContext

backend = LocalHFBackend(model_id="ibm-granite/granite-4.1-3b")
context = (
    ChatContext()
    .add(Message("user", "What is the square root of 4?"))
    .add(Message("assistant", "The square root of 4 is 2."))
)
score = core.check_certainty(context, backend)
```

For lower-level control (custom adapters, model options), use `mfuncs.act()` with `Intrinsic` directly — see examples in `docs/examples/intrinsics/`.

### Weights binding shapes

`Adapter.weights` normalizes each deployment's activation mechanism behind three
shapes — a `WeightsBinding` lifecycle for weights you stage yourself,
`EmbeddedBinding.apply_activation` for weights already in the served model, or
`ServerMediatedBinding` for a model tag selected by the provider. The
post-activation shape each produces:

| Binding | Reality | Lifecycle verbs | Caller invokes | Normalized post-activation state |
|---------|---------|------------------|-----------------|-----------------------------------|
| `LocalFileBinding` | LocalFile/PEFT | `prepare` / `activate` / `deactivate` / `release` | `activate()` / `deactivate()`, via `adapter_scope` | Backend-internal PEFT adapter state toggled; the outgoing request is untouched |
| `EmbeddedBinding` | Embedded/Granite Switch | none — weights are already in the served model | `apply_activation(request, identity)` | `request.extra_body["chat_template_kwargs"]["adapter_name"]` set; `request.api_params["model"]` removed if present |
| `ServerMediatedBinding` | Ollama bundled adapter model | none for the current Ollama path | select the configured model tag during intrinsic generation | Ollama request's `model` is the bundled adapter tag; full lifecycle telemetry remains follow-up work |

### Project Resources

- **Canonical catalog**: `mellea/backends/adapters/catalog.py` — source of truth for adapter function names, HF repo IDs, and adapter types
- **Usage examples**: `docs/examples/intrinsics/` — working code for every adapter function
- **Helper functions**: `mellea/stdlib/components/intrinsic/rag.py` and `core.py`

### Adding New Adapter Functions

When adding support for a new adapter function (not just using an existing one), fetch its README from Hugging Face first. Each README contains the authoritative spec for input/output format, intended use, and examples.

**Writing examples?** The HF READMEs also document intended usage patterns and example inputs — useful reference when writing code in `docs/examples/intrinsics/`.

| Repo | Purpose | Adapter functions |
|------|---------|------------|
| [`ibm-granite/granitelib-rag-r1.0`](https://huggingface.co/ibm-granite/granitelib-rag-r1.0) | RAG pipeline | answerability, citations, hallucination_detection, query_rewrite, query_clarification |
| [`ibm-granite/granitelib-core-r1.0`](https://huggingface.co/ibm-granite/granitelib-core-r1.0) | Core capabilities | context-attribution, requirement-check, uncertainty |
| [`ibm-granite/granitelib-guardian-r1.0`](https://huggingface.co/ibm-granite/granitelib-guardian-r1.0) | Safety & compliance | guardian-core, policy-guardrails, factuality-detection, factuality-correction |

**README URLs** — RAG adapter functions (no model subfolder):
```
https://huggingface.co/ibm-granite/granitelib-rag-r1.0/blob/main/{intrinsic_name}/README.md
```

Core and Guardian adapter functions (include model subfolder):
```
https://huggingface.co/ibm-granite/granitelib-{core,guardian,rag}-r1.0/blob/main/{intrinsic_name}/granite-4.1-{3b,8b,30b}/{lora,alora}/README.md
```

More agent context in generative-computing/mellea

4 other files this repository gives its agents.