auto-round

# AGENTS.md

This file provides guidance to ANY AGENTS when working with code in this repository.

## Project

AutoRound — post-training quantization for LLMs/VLMs using sign-gradient descent. Publishes as `auto-round` (GPU/CPU) and `auto-round-hpu` (Intel Gaudi).

## Build & Install

```bash
# From source (GPU/CPU) — --no-build-isolation is required when PyTorch is already installed
pip install --no-build-isolation -e .

# HPU variant
BUILD_HPU_ONLY=1 pip install --no-build-isolation .
# or: python setup.py hpu install

# XPU variant — install Intel PyTorch first
pip install torch --index-url https://download.pytorch.org/whl/xpu
pip install --no-build-isolation .
```

## Testing

Tests are split into three tiers under `test/`: `unit/` (fast, runs in PR CI),
`integration/` (third-party frameworks, runs nightly), and `e2e/` (full models /
real inference engines, runs weekly). Each tier is further split by hardware
(`test_cpu/`, `test_cuda/`, `test_hpu/`, `test_xpu/`, `test_ark/`, `test_mlx/`).

```bash
# CPU unit tests (most common during development)
pytest test/unit/test_cpu/ -x -q

# Single test
pytest test/unit/test_cpu/ -k "test_name" -x -q

# Hardware-specific unit tests
pytest test/unit/test_cuda/
pytest test/unit/test_hpu/ --mode=lazy   # or --mode=compile
pytest test/unit/test_xpu/

# Slower suites (nightly / weekly)
pytest test/integration/test_cpu/
pytest test/e2e/test_cpu/
```

Test fixtures create tiny models (OPT-125M, Qwen-0.6B) at session scope — first run downloads them.

## Code Quality & Pre-commit

All syntax, formatting, licensing, spelling, and linting checks are enforced via `.pre-commit-config.yaml`.

### Execution Rules for Agents

1. **Always run pre-commit after editing files**:
   - For focused changes (preferred):
     ```bash
     pre-commit run --files <changed_file1> <changed_file2>
     ```
   - For a specific hook/tool (e.g., `ruff`, `black`, `codespell`):
     ```bash
     pre-commit run <hook_id> --files <changed_file>
     ```
   - For repository-wide checks (recommended for extensive changes or initial runs; fast and lightweight):
     ```bash
     pre-commit run --all-files
     ```
2. **Rerun until clean**: Tools like `ruff` and `codespell` automatically fix issues but will exit with non-zero on first modification (`--exit-non-zero-on-fix`). Keep changes and rerun the command until all checks pass.
3. **Manual fixes**: Only make manual changes if a check fails and cannot be auto-fixed (e.g., `bandit` security findings, `check-json`, or complex syntax errors).

### Automated Checks Summary

- **Auto-fixing hooks**: `black` (line length 120), `isort` (imports), `ruff` (lint fixes), `insert-license` (Apache 2.0 header), `codespell` (typos), `mixed-line-ending` (LF).
- **Check-only hooks**: `bandit` (security issues under `auto_round/`), `typos`, `check-yaml`/`check-json`, `markdown-link-check`.
- *Note*: `auto_round/export/export_to_gguf/conversion/` is intentionally excluded from most hooks.

## Commit & PR Conventions

- Conventional commits: `feat:`, `fix:`, `chore:`, `docs:`, `refactor:`, `test:`
- PRs target `main`, squash-merged
- **CN docs rule**: any change to a `.md` file must include a matching update to its `_CN` counterpart (e.g., `README.md` → `README_CN.md`)

## Key Environment Variables

- `BUILD_HPU_ONLY=1` — build HPU package variant
- `AR_USE_MODELSCOPE=1` — use ModelScope instead of HuggingFace for model downloads
- `FORCE_BF16=1` — force BF16 in tests (used in CI)

## Source Layout

- `auto_round/` — core library (AutoRound class, sign-SGD, exporters, eval, data types)
- `auto_round_extension/` — hardware backends (CUDA, HPU, IPEX/XPU, Triton, ARK, vLLM)
- `test/` — tests organized by tier then hardware: `unit/` (PR CI), `integration/` (nightly), `e2e/` (weekly), each with `test_cpu/`, `test_cuda/`, ...
- `examples/` — usage examples for different model types

## Gotchas

- `setup.py` forces `CC=CXX=g++` at import time
- Version is computed dynamically from git tags — untagged commits produce dev versions
- Some test dependencies (AutoAWQ, GPTQModel, llama-cpp) require manual git installs — see comments in `test/unit/test_cuda/requirements.txt`

More agent context in intel/auto-round

9 other files this repository gives its agents.