AGENTS.md

This file provides guidance to AI coding assistants when working with code in this repository.

Repository Overview

This repository contains code for control-arena, an open source library for AI Control experiments built on top of Inspect AI. It is built by the UK AI Security Institute.

Code can be found in control_arena/. Docs can be found in docs/. Also see README.md and CONTRIBUTING.md for more information.

Build/Lint/Test Commands

• Run python code: uv run path/to/script.py
• Run all tests: uv run pytest or uv run pytest -m "not skip_ci and no long_running"
• Run a single test: uv run pytest tests/path/to/test_file.py::test_function_name -v
• Format code: uv run ruff format
• Lint code: uv run ruff check --fix
• Type check: uv run pyright

Code Style Guidelines

• Formatting: Follow Google style convention. Use ruff for formatting. Try to follow ruff styles directly when writing code. You can see what rules we have enabled in pyproject.toml.
• Imports: Use isort order (enforced by ruff). Imports must be at the top of the file, unless there is a very good reason otherwise.
• Types: Strict typing is required. All functions must have type annotations
• Naming: Use snake_case for variables, functions, methods; PascalCase for classes
• Docstrings: Google-style docstrings required for public APIs. These will be used for automatic documentation generation and so should be clear, easy to understand and complete.
• Documentation: This project uses Quarto for documentation. See docs/ for examples. When adding new public APIs we may need to update the docs, including the automated API reference section in docs/reference/. Lists within .qmd files must be separated by a blank line from anything above, otherwise they will not render properly.
• Spelling: This project is built by the UK AI Security Institute, so use UK English spelling where relevant.
• Error Handling: Use appropriate exception types; include context in error messages. Use if / raise rather than assert for runtime checks.
• Testing: Write tests with pytest; maintain high coverage. No need to mark async tests with pytest.mark.asyncio as this is configured automatically. Use pytest.mark.skip_ci for tests that should not run in CI (e.g. because they call LLMs), and pytest.mark.long_running for tests that take a long time to run.
• String Formatting: Don't wrap lines manually unless they should explicitly be separated by a line break. Instead, put strings (e.g. docstrings) on one line so that they can be wrapped automatically by whatever editor is being used.
• Async Code: This project supports both asyncio and trio backends via anyio (configured by the INSPECT_ASYNC_BACKEND environment variable). Do not use asyncio primitives directly. Instead:
  • Use anyio.sleep() instead of asyncio.sleep()
  • Use from control_arena._async_utils import gather instead of asyncio.gather() / asyncio.create_task()
  • Use from control_arena._async_utils import gather_with_exceptions instead of asyncio.gather(*coros, return_exceptions=True)
  • Use from control_arena._async_utils import as_completed_with_exceptions instead of asyncio.as_completed()
  • Use from control_arena._async_utils import run_sync instead of asyncio.run() for sync-to-async entry points
  • Use anyio.open_process() instead of asyncio.create_subprocess_exec() (and catch anyio.EndOfStream when reading from process streams)
  • Use anyio.to_thread.run_sync() instead of loop.run_in_executor()
  • Use anyio.fail_after() instead of asyncio.wait_for()
  • Use anyio.Semaphore() instead of asyncio.Semaphore()
  • See control_arena/_async_utils.py for available utilities

Git Conventions

• Branch naming: <username>/<type>/<description> in kebab-case (e.g. alice/feat/add-new-setting, bob/fix/monitor-timeout)
• Commit messages: Use Conventional Commits -- <type>: <description>. Common types: feat, fix, docs, refactor, chore, test. Use ! for breaking changes (e.g. feat!: Change sample ID format).

Respect existing code patterns when modifying files. Run linting before committing changes.