AGENTS.md

This file provides guidance to coding agents working with code in this repository.

Project Overview

sphinx-llm is a collection of Sphinx extensions for working with LLMs. It serves two purposes:

1. Enabling LLMs to consume documentation - Generates llms.txt, llms-full.txt, and per-page markdown files following the llms.txt standard
2. Leveraging LLMs to generate content - Uses LLMs to generate static content during builds (e.g., the docref directive for page summaries)

Development Commands

Prefer using Git worktrees for development.

Setup

# Install dependencies with uv (recommended)
uv sync --dev --extra gen

Testing

# Run all tests
uv run pytest src/sphinx_llm/tests/

# Run tests with coverage
uv run pytest src/sphinx_llm/tests/ --cov=sphinx_llm --cov-report=xml

# Run tests against a specific Sphinx version
uv run --with "sphinx>=7,<8" pytest src/sphinx_llm/tests/

Linting and Formatting

# Ensure pre-commit hooks are installed
uv run pre-commit install

# Run ruff linter with auto-fix
uv run pre-commit run ruff --all-files

# Run ruff formatter
uv run pre-commit run ruff-format --all-files

# Run all pre-commit hooks (includes ruff, prettier, markdownlint, codespell,
# license headers)
uv run pre-commit run --all-files

Building Documentation

# Build the primary site (Python >=3.10)
uv run --dev --group docs sphinx-autobuild docs/source docs/build/html

# Build docs once
uv run --group docs sphinx-build docs/source docs/build/html

Architecture

Core Extensions

sphinx_llm.txt (src/sphinx_llm/txt.py)

• Hooks into Sphinx's builder-inited and build-finished events
• Spawns a parallel subprocess running sphinx-build -b markdown to generate markdown files
• The MarkdownGenerator class orchestrates:
  1. Parallel markdown build (can be disabled via llms_txt_build_parallel config)
  2. Merging markdown output with HTML output (each page gets .html.md extension)
  3. Generating llms-full.txt (concatenated markdown)
  4. Generating llms.txt (sitemap with descriptions)
• Handles both html and dirhtml builders with different path structures
• For dirhtml builder, supports three suffix modes via llms_txt_suffix_mode:
  • "file-suffix": Only generates page/index.html.md files
  • "url-suffix": Only generates URL-style page.md files
  • "both" (default): Generates both formats for maximum compatibility

sphinx_llm.docref (src/sphinx_llm/docref.py)

• Parses directives into pending nodes and collects unique requests in the Sphinx environment
• Applies authored directive and html_meta overrides before consulting the generated-summary cache
• Generates missing summaries at env-updated through the shared OpenAI-compatible client and llms_txt_summary_* configuration without modifying source files
• Stores requests and effective state in the Sphinx environment, persists generated records in the shared versioned page-summary cache, and supports purge/merge
• Writes sphinx-llm-summaries.json with effective summaries and provenance
• Requires the gen dependencies and an explicitly configured model for cache misses

Test Structure

Tests live in src/sphinx_llm/tests/ (not a separate tests/ directory). Tests build src/sphinx_llm/tests/fixtures/integration_docs/ into temporary directories with different builders and parallel settings. Keep that corpus and its unthemed configuration independent of the primary site in docs/source/. The opt-in docs dependency group is for the primary site, not compatibility tests; its theme has a higher Python/Sphinx floor than the library.

Key test fixture: sphinx_build - parametrized fixture that tests both html and dirhtml builders with parallel and sequential markdown building.

When adding or changing tests, reuse or extend shared fixtures in src/sphinx_llm/tests/conftest.py before duplicating setup. Share project/build mechanics, provider stubs, and cleanup where the duplication removed outweighs the added indirection; keep scenario-specific configuration and assertions in each test. For documentation build coverage, prefer adding examples to the fixture docs in src/sphinx_llm/tests/fixtures/integration_docs/ and asserting against their built output over inlining documentation source within tests. Prefer lightweight fixtures for unit tests and real builds for integration tests.

Documentation Guidelines

Treat the configuration options table as the canonical documentation for each setting's name, purpose, type, default value, and basic behavior, including enable/disable behavior when applicable.

When adding or changing a configuration option:

• Update its entry in the configuration table.
• Keep the option's name, purpose, type, default value, and basic behavior in the table as their single source of truth.
• Add prose outside the table only when it provides distinct information that does not fit concisely in the table, such as a workflow, an interaction between multiple settings, migration guidance, or a substantial usage example.
• When additional prose is necessary, refer readers to the configuration table instead of restating the option's description or default value.

Before submitting a documentation change, compare all newly added prose against the configuration table and remove duplicated explanations.

Commit Requirements

All commits by contributors who are not employed by NVIDIA must be signed off using git commit -s (Developer Certificate of Origin).

Pre-commit hooks must be installed. They enforce:

• Ruff formatting and linting
• License header in all .py files (using LICENSE_HEADER file)
• Prettier for YAML
• Markdownlint for Markdown
• Codespell for spelling

Version Management

Uses hatch-vcs for version management from git tags. Version is generated at build time into src/sphinx_llm/_version.py.