onyx-foss
AGENTS.md320 starsChanged 8 days agoMITView on GitHub
- Python deps live in a uv-managed virtualenv at .venv (repo root). If it doesn't exist yet, create it with uv sync --frozen, then source .venv/bin/activate. - Test secrets (API keys etc.) are resolved by backend/tests/utils/aws_secrets.py, in order: process env vars → the gitignored .vscode/.env (also used by the pytest commands in backend/AGENTS.md; create it by copying .vscode/env_template.txt) → AWS Secrets Manager (requires aws sso login). Tests declare what they need via @pytest.mark.secrets(TestSecret.X). If a key you need still can't…
- Reads credentials
Where files are headingNew AGENTS.md files have more rules about keeping secrets out of the code, and fewer about code style.
Of the AGENTS.md files we read, the 32,671 first seen in the last six months, against the older ones: the kind of rule that gained most share, and the kind that lost most. About files like this in general, not this one, and an estimate, since a kind is a pattern in the words.
In this repository10 other files it gives its agents.
- backend/AGENTS.mdAGENTS.md
- documentation/AGENTS.mdAGENTS.md
- mobile/AGENTS.mdAGENTS.md
- web/AGENTS.mdAGENTS.md
- docs-writingSkill
- feature-mapSkill
- merge-dependabot-prsSkill
- onyx-craft-devSkill
- onyx-dev-stackSkill
- playwright-e2e-testsSkill
The file
Show its rules and text
We split this AGENTS.md into the rules it gives an agent
29 rules. About 2 are common, 14 are distinctive, and 13 are very specific to this project.
≈2
Common · estimate
Many files like this have a rule of this kind.
≈14
Distinctive · estimate
This author's own choices. Worth a look.
13
Very specific
Tied to their project's names, files or tools. Probably skip.
- Common
- Distinctive
- Very specific
- could be automatic: a setting could enforce it
- Very specific: line 9
Test secrets (API keys etc.) are resolved by backend/tests/utils/aws_secrets.py, in order: process env vars → the gitignored .vscode/.env …
- Very specific: line 14
If using playwright to explore the frontend, log in with username admin_user@example.com and password TestPassword123!
- Very specific: line 24
When making calls to the backend, always go through the frontend.
- Very specific: line 25
If .agents-local.md exists at the repo root and its rules are not already in your context, read and follow it.
- Very specific: line 27
Before you start work, check that your branch does not need a rebase on origin/main.
- Very specific: line 35
Check for conflicts with git merge-tree --write-tree HEAD origin/main.
- Very specific: line 37
Check the age with git show -s --format=%cr $(git merge-base HEAD origin/main).
- Distinctive: line 40
If a merge or rebase is already in progress, finish it or abort it first.
- Distinctive: line 41
Never discard uncommitted changes to do this.
- Very specific: line 74
When you change something a customer sees or configures (UI, admin settings, connectors, environment variables, the public API, …
- Very specific: line 88
Look up a code path in .agents/feature-map/PATHS.md to find its component.
- Very specific: line 93
If your change makes a map document wrong, update it in the same pull request, as .agents/feature-map/README.md ("Keeping the map true") …
- Very specific: line 123
If pre-commit fails, fix the reported problems before you finish.
- Distinctive: line 139
Follow ASD-STE100 Simplified Technical English for technical text:
- Distinctive: line 141
Use approved words only. Each word has one meaning.
- Distinctive: line 142
Use one word for one idea.
- Distinctive: line 143
Write short sentences. Use 20 words or less for instructions.
- Distinctive: line 144
Use active voice. Write "Turn the switch", not "The switch must be turned".
- Distinctive: line 145
Write short paragraphs. Keep one topic in each paragraph.
- Distinctive: line 146
Keep code comments focused on information that is relevant long-term or for future readers.
- Very specific: line 156
When (1) writing integration tests or (2) doing live tests (e.g.
- Common: line 162
Never commit API keys or secrets to the repository
- Common: line 163
Use the encrypted credential storage for connector credentials
- Distinctive: line 164
Follow existing RBAC patterns for new features
- Very specific: line 168
When creating a plan in the plans directory (gitignored — create it if it doesn't exist), make sure to include at least these elements:
- Distinctive: line 184
Do NOT include these: _Timeline_, _Rollback plan_
- Distinctive: line 187
Keep it high level. You can reference certain files or functions though.
- Distinctive: line 189
Before writing your plan, make sure to do research.
- Distinctive: line 193
Before you add a helper, utility, pattern or convention, search the codebase for one that does the same job, and use it.
≈ “Common” is an estimate: at least 1 in 5 of the 41,581 AGENTS.md files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.
Files
PROJECT KNOWLEDGE BASE
This file provides guidance to AI agents when working with code in this repository.
KEY NOTES
- Python deps live in a
uv-managed virtualenv at.venv(repo root). If it doesn't exist yet, create it withuv sync --frozen, thensource .venv/bin/activate. - Test secrets (API keys etc.) are resolved by
backend/tests/utils/aws_secrets.py, in order: process env vars → the gitignored.vscode/.env(also used by the pytest commands inbackend/AGENTS.md; create it by copying.vscode/env_template.txt) → AWS Secrets Manager (requiresaws sso login). Tests declare what they need via@pytest.mark.secrets(TestSecret.X). If a key you need still can't be resolved, ask the user rather than skipping tests. - If using
playwrightto explore the frontend, log in with usernameadmin_user@example.comand passwordTestPassword123!(the admin user created by the playwright global setup — seeweb/tests/e2e/constants.ts). If it doesn't exist yet, register it via the signup page; the first user registered automatically becomes admin. The app can be accessed athttp://localhost:3000. - You should assume that all Onyx services are running. To verify, you can check the
backend/logdirectory to make sure we see logs coming out from the relevant service. - To connect to the Postgres database, use:
PGPASSWORD="${POSTGRES_PASSWORD:-password}" psql -h "${POSTGRES_HOST:-localhost}" -U postgres -c "<SQL>". This works on a host checkout and inside the devcontainer. If nopsqlclient is available, fall back todocker exec onyx-relational_db-1 psql -U postgres -c "<SQL>"(no-it— agent shells have no TTY). - When making calls to the backend, always go through the frontend. E.g. make a call to
http://localhost:3000/api/personanothttp://localhost:8080/api/persona - If
.agents-local.mdexists at the repo root and its rules are not already in your context, read and follow it. It carries developer-local agent guidance and is never committed. - Before you start work, check that your branch does not need a rebase on
origin/main. Rebase if the branch has merge conflicts with main. Rebase if the branch is stale: more than 3 days since it diverged from or last synced with main. Keeping a branch current prevents a much more involved conflict resolution later. Claude Code and Codex run this check at session start and on later prompts. The hook is.agents/hooks/ensure_latest_main.py. It fetches at most once an hour. It is silent unless a rebase is needed. Other agents: rungit fetch origin +refs/heads/main:refs/remotes/origin/main. Check for conflicts withgit merge-tree --write-tree HEAD origin/main. Exit status 1 means conflicts. Other non-zero codes are errors, not conflicts. Check the age withgit show -s --format=%cr $(git merge-base HEAD origin/main). To update, rungit rebase origin/main. Usegit merge origin/mainfor a shared branch. Onmain, rungit pull --ff-only origin main. If a merge or rebase is already in progress, finish it or abort it first. Never discard uncommitted changes to do this.
Project Overview
Onyx (formerly Danswer) is an open-source Gen-AI and Enterprise Search platform that connects to company documents, apps, and people. It features a modular architecture with both Community Edition (MIT licensed) and Enterprise Edition offerings.
Technology Stack
- Backend: Python 3.13, FastAPI, SQLAlchemy, Alembic, Celery
- Frontend: Next.js 16, React 19, TypeScript, Tailwind CSS
- Database: PostgreSQL with Redis caching
- Search: OpenSearch-backed keyword and vector document index
- Auth: OAuth2, SAML, multi-provider support
- AI/ML: LangChain, LiteLLM, multiple embedding models
Repository Layout & Sub-project Guides
Each sub-project has its own agents file with the standards for that area — read it before working there:
backend/— FastAPI app + Celery workers.onyx/is the Community Edition core,ee/mirrors its layout for Enterprise features,alembic/holds migrations,tests/the test suites. Standards (Celery, migrations, testing, error handling, LLM tracing):backend/AGENTS.md.web/— Next.js frontend. Standards (also coverdesktop/, the Tauri shell):web/AGENTS.md.mobile/— React Native + Expo app. Standards:mobile/AGENTS.md. Mobile differs from web on several points (no DOM, NativeWind, expo-router), so do not assume the web rules apply there.documentation/— customer docs (Mintlify, docs.onyx.app). Standards:documentation/AGENTS.md. Production docs follow the latest stable release, notmain.
Explore the tree with ls rather than relying on docs for the full package list.
Docs
When you change something a customer sees or configures (UI, admin settings, connectors, environment variables, the
public API, deployment), update the matching page in documentation/ in the same PR. Recapture screenshots of UI you
changed: follow .agents/skills/docs-writing/SKILL.md and use the local app and admin login above. Skip docs for
internal refactors, tests, CI, and multi-tenant cloud operations, which customers cannot use.
Feature Map
.agents/feature-map/ maps the product: every surface, what it does, the code behind
it, and how the parts connect. Use it before you change a product surface, when you
review a diff or PR, and when you need to know what a feature does or what a change
can break. The feature-map skill (.agents/skills/feature-map/SKILL.md) gives the
procedure.
- For new work, start at the Common tasks table in
.agents/feature-map/INDEX.md. - Look up a code path in
.agents/feature-map/PATHS.mdto find its component. - The step-by-step review procedure is
.agents/feature-map/VERIFYING.md. - Onyx overloads words such as Persona, Tool, turn, and SearchDoc. Read
.agents/feature-map/GLOSSARY.mdbefore trusting a name.
If your change makes a map document wrong, update it in the same pull request, as
.agents/feature-map/README.md ("Keeping the map true") describes.
Skills
Shared agent skills live in .agents/skills/. .claude/skills, .cursor/skills and
.codex/skills are symlinks to it, so add or edit skills only in .agents/skills/.
Code Quality
# Install and run pre-commit hooks
pre-commit install
pre-commit run --all-files
# Faster: run only on the files you touched
pre-commit run --files <path> [<path> ...]Claude Code and Codex also run pre-commit on the files you changed in a turn when
you end it (.agents/hooks/pre_commit_on_stop.py, wired as UserPromptSubmit and
Stop hooks in .claude/settings.json and .codex/hooks.json). Files that were
already dirty when the turn started are skipped unless you change them. Codex runs
the hooks only after you trust the project .codex/ layer. Committed changes come from
local commit, amend, cherry-pick, revert, and conflict-resolution entries in the
worktree HEAD reflog. For a merge commit, only files that differ from Git's automatic
merge result are checked. For a rebase, the check includes every file in each commit
resolved with git rebase --continue, and skips clean picks. The --apply rebase
backend logs resolved conflicts as picks, so the hook skips them. Checkout, pull, clean
merge, and clean rebase updates do not add files to the check.
If pre-commit fails, fix the reported problems before you finish. When the repo has the feature map
(.agents/feature-map/stale_docs.py), the same hook blocks the stop once with an
editing brief: the sections and keys of the feature-map documents that cite code
the turn changed, and changed files that PATHS.md does not cover. Ownership
through PATHS.md alone does not block. Answer it
with "Editing judgment" in .agents/feature-map/README.md. No update is a normal
answer; edit only what the change made wrong or what a future agent needs.
NOTE: Always make sure everything is strictly typed (both in Python and Typescript).
NOTE: Keep code comments brief and focused on information that stays relevant long-term.
Writing
These rules apply to all prose you write: docs, commit messages, PR descriptions, reports, and replies.
Follow ASD-STE100 Simplified Technical English for technical text:
- Use approved words only. Each word has one meaning.
- Use one word for one idea. Do not use two words for the same thing.
- Write short sentences. Use 20 words or less for instructions.
- Use active voice. Write "Turn the switch", not "The switch must be turned".
- Write short paragraphs. Keep one topic in each paragraph.
- Keep code comments focused on information that is relevant long-term or for future readers.
Testing
There are 4 main types of tests: unit, external dependency unit, integration, and playwright e2e
(web/tests/e2e). Commands and guidance for all four live in backend/AGENTS.md; shared fixtures
and deeper detail in backend/tests/README.md. Prefer integration tests over the other types.
Logs
When (1) writing integration tests or (2) doing live tests (e.g. curl / playwright) you can get access
to logs via the backend/log/<service_name>_debug.log file. All Onyx services (api_server, web_server, celery_X)
will be tailing their logs to this file.
Security Considerations
- Never commit API keys or secrets to the repository
- Use the encrypted credential storage for connector credentials
- Follow existing RBAC patterns for new features
Creating a Plan
When creating a plan in the plans directory (gitignored — create it if it doesn't exist), make sure to
include at least these elements:
Issues to Address What the change is meant to do.
Important Notes Things you come across in your research that are important to the implementation.
Implementation strategy How you are going to make the changes happen. High level approach.
Tests What unit (use rarely), external dependency unit, integration, and playwright tests you plan to write to verify the correct behavior. Don't overtest. Usually, a given change only needs one type of test.
Do NOT include these: Timeline, Rollback plan
This is a minimal list - feel free to include more. Do NOT write code as part of your plan. Keep it high level. You can reference certain files or functions though.
Before writing your plan, make sure to do research. Explore the relevant sections in the codebase.
Reuse Existing Standards
Before you add a helper, utility, pattern or convention, search the codebase for one that does the same job, and use it. For example, use the existing rate limiting, caching, URL validation, router and settings patterns. Do not invent a parallel mechanism. If the existing one does not do enough, extend it so other callers get the change too. A new pattern needs a reason that the existing ones cannot meet.
Best Practices
In addition to the other content in this file, best practices for contributing
to the codebase can be found in the "Engineering Best Practices" section of
CONTRIBUTING.md. Understand its contents and follow them.
Does it work?
Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.
No reviews yet
People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.
- It worked
- Partly
- It didn't
Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.
Your trail
Files you open will collect here, so you can find them again.
Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.
Now do this for your own.
Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.

