phoenix-evals
Skill12k starsChanged 51 days agoLicence not identifiedView on GitHub
Build, run, and validate evaluators for AI/LLM applications with Phoenix, in Python or TypeScript (including Vercel AI SDK apps) — code-based checks and LLM-as-a-judge, run over exported traces or dataframes or as Phoenix experiments, gate them in CI, and validate judges against human-labeled data.
Trending
Less talk lately.about the whole repository
Dec 2025Jun 2026Nov 2026
Who's talking about Arize-ai/phoenix
- A Hacker News story“Show HN: Phoenix OSS – Applying LLM Spans, Traces, and Evals for AI Insights”Hacker News · Oct 4, 2023 · 23 points
- Arize-ai/openinference“[feature request] auto-instrumentation for n8n”GitHub · Aug 6, 2025 · 12 reactions
- open-telemetry/semantic-conventions-genai“ATIF trajectory format used by Harbor”GitHub · Jun 23, 2026 · 6 reactions
- @arize-phoenix.bsky.social“🎉 5000 Stars and Counting... 🎉 We're celebrating Phoenix reaching 5000 stars on GitHub! This milestone …”Bluesky · Mar 19, 2025 · 5 likes
- @anthonypowell.me“Checkout github.com/Arize-ai/pho... Open source, with growing TS support Disclaimer: I work here”Bluesky · Nov 8, 2024 · 5 likes
The biggest 5 of 62 mentions linking to the repository that our crawl found. Each line is their own headline.
See all 62
Talked about on GitHub, Bluesky, the web, Hacker News and Stack Overflow (62 times).
On GitHub · 50
- Aug 6, 2025[feature request] auto-instrumentation for n8n — Arize-ai/openinference12 reactions
- Jun 23, 2026ATIF trajectory format used by Harbor — open-telemetry/semantic-conventions-genai6 reactions · 1 comment
- May 20, 2025HTTP multipart subscriptions are not terminated with closing boundary when error is present in payload — strawberry-graphql/strawberry3 reactions · 2 comments
- Oct 1, 2024[feature request] Add an easy way to get the root span from a child span — Arize-ai/openinference3 reactions · 1 comment
- May 13, 2025[bug] MCP instrumentation and context propagation is not working — Arize-ai/openinference2 reactions · 1 comment
45 more on GitHub
- Dec 26, 2024Add Arize OSS Phoenix integration — langgenius/dify2 reactions · 9 comments
- Oct 11, 2024[BUG][litellm] litellm instrumentation may fail depending on how the package is imported — Arize-ai/openinference2 reactions · 3 comments
- Aug 27, 2026chore(deps): bump the llm group across 1 directory with 8 updates — langgenius/dify1 reaction · 1 comment
- Aug 4, 2026chore(deps): bump the llm group across 1 directory with 9 updates — langgenius/dify1 reaction · 1 comment
- Jul 30, 2026recommendations for bashkit snapshot deltas / patches — everruns/bashkit1 reaction · 5 comments
- Jul 20, 2026chore(deps): bump the llm group across 1 directory with 6 updates — langgenius/dify1 reaction · 1 comment
- Jul 20, 2026chore(deps): bump the llm group across 1 directory with 7 updates — langgenius/dify1 reaction · 1 comment
- Jul 13, 2026chore(deps): bump the llm group in /api with 6 updates — langgenius/dify1 reaction · 1 comment
- Jul 13, 2026chore(deps): bump the llm group in /api with 7 updates — langgenius/dify1 reaction · 1 comment
- Jul 6, 2026build(deps): bump the llm group across 1 directory with 6 updates — langgenius/dify1 reaction · 2 comments
- Jul 3, 2026Add engineering resources: Phoenix migration, LLM gateway explainer, coding-agent tracing — langfuse/langfuse-docs1 reaction · 3 comments
- Jun 13, 2026Initial implementation of Arize OpenInference — maximhq/bifrost1 reaction · 10 comments
- Jun 9, 2026feat(langchain-instrumentation): Add GraphInterrupttoIGNORED_EXCEPTION_PATTERNS — Arize-ai/openinference1 reaction · 2 comments
- Jun 8, 2026chore(deps): bump the llm group across 1 directory with 5 updates — langgenius/dify1 reaction · 2 comments
- Jun 4, 2026build(deps): bump the llm group across 1 directory with 7 updates — langgenius/dify1 reaction · 1 comment
- Jun 1, 2026chore(deps): bump the llm group across 1 directory with 6 updates — langgenius/dify1 reaction · 1 comment
- Jun 1, 2026chore(deps): bump the llm group across 1 directory with 6 updates — langgenius/dify1 reaction · 2 comments
- May 25, 2026chore(deps): bump the llm group across 1 directory with 5 updates — langgenius/dify1 reaction · 2 comments
- Jan 20, 2026[bug] retriever for vector store search missing retrieved documents — Arize-ai/openinference1 reaction · 2 comments
- Nov 5, 2025How to trace on goggle adk artifact using openinference-google-adk — Arize-ai/openinference1 reaction · 2 comments
- Sep 26, 2025Add support for users to provide feedback on specific messages — jmatthiesen/csharp-ai-buddy-site1 reaction
- Jul 29, 2025[instrumentation] add the ability to get the first_span_id from the capture context manager — Arize-ai/openinference1 reaction
- May 6, 2025[ERROR]: openinference.instrumentation.langchain._tracer:Failed to get attribute — Arize-ai/openinference1 reaction · 5 comments
- Feb 19, 2025[bug] llama-index python instrumentation sends tool parameters as invalid JSON — Arize-ai/openinference1 reaction · 7 comments
- Jan 16, 2025[llama-index] serialization error and missing token counts with llamaindex and gemini — Arize-ai/openinference1 reaction · 4 comments
- Dec 4, 2024capture url path in openinference openai instrumentor — Arize-ai/openinference1 reaction · 1 comment
- Mar 26, 2024Support langchain tags — Arize-ai/openinference1 reaction · 1 comment
- Feb 9, 2024Instrument OpenTelemetry — breadboard-ai/breadboard1 reaction
- Nov 4, 2023[Feature Request]: Callback Events for Refine output_cls pydantic invocations — run-llama/llama_index1 reaction · 6 comments
- Oct 26, 2026deps(deps): update arize-phoenix requirement from <5,>=4.0 to >=20.25.0,<21 — sahiltalwar88/job-hunter0 reactions · 1 comment
- Oct 23, 2026chore(deps): update arize-phoenix requirement from >=20.3.0 to >=20.23.0 in /agent — edycutjong/suture0 reactions · 1 comment
- Oct 23, 2026chore(deps): bump the python-dependencies group across 1 directory with 108 updates — timjardenross/TJRHQ0 reactions
- Oct 21, 2026Import adapters for Langfuse, Arize Phoenix and LangSmith trace exports — soumendrak/widescope0 reactions
- Oct 21, 2026Positioning: lead README/landing with the user job and companion framing — soumendrak/widescope0 reactions
- Oct 13, 2026Bump the llm group across 1 directory with 6 updates — context-aware-benchmark2/dify0 reactions
- Oct 5, 2026Bump the llm group across 1 directory with 5 updates — akshit-mahajan26/dify-render-template-mrj0l4sh0 reactions
- Oct 5, 2026deps: update arize-phoenix-otel requirement from >=0.16.1 to >=0.17.2 — pjcau/agent-orchestrator0 reactions · 1 comment
- Oct 5, 2026Bump the python-dependencies group across 1 directory with 10 updates — GuanyiLi-Craig/employee-right-agent0 reactions
- Aug 25, 2026chore(deps-dev): update arize-phoenix requirement from <18,>=17 to >=20.3.0,<21 — thehipsterciso/hc-assay0 reactions
- Aug 24, 2026chore(deps): bump the all-dependencies group in /python/agents/travel-concierge with 17 updates — sanjaymk908/truclaw-adk-samples0 reactions · 1 comment
- Aug 18, 2026Update arize-phoenix requirement from <16.0,>=4.0 to >=4.0,<21.0 in /packages/agent-sre — igordjuric404/ophanix-platform0 reactions
- Aug 16, 2026chore(deps): bump the pip-root group across 1 directory with 3 updates — francisco-perez-sorrosal/praxion0 reactions · 5 comments
- Aug 16, 2026chore(deps): bump the pip-eval group across 1 directory with 2 updates — francisco-perez-sorrosal/praxion0 reactions · 5 comments
- Aug 12, 2026chore(deps): bump the llm group across 1 directory with 5 updates — 18300676767/rule-engine0 reactions
- Aug 9, 2026build(deps): update arize-phoenix-otel requirement from >=0.6.0 to >=0.17.0 — mrinmoyece/cortex0 reactions
On Bluesky · 6
- Mar 19, 2025🎉 5000 Stars and Counting... 🎉 We're celebrating Phoenix reaching 5000 stars on GitHub! This milestone underscores the growing demand for… — @arize-phoenix.bsky.social5 likes and reposts · 1 reply
- Nov 8, 2024Checkout github.com/Arize-ai/pho... Open source, with growing TS support Disclaimer: I work here — @anthonypowell.me5 likes and reposts
- Mar 27, 2026Guess how many 🌟 Phoenix has on GitHub? Yes, this meme has been all over our internal slack today. github.com/Arize-ai/ph... — @arize-phoenix.bsky.social2 likes and reposts
- Dec 5, 2024Prompt Playground is out in @arize.bsky.social Phoenix 6.0 📊Test and compare prompts, tool definitions, output schemas, and models,… — @mikeldking.bsky.social2 likes and reposts
- May 4, 2026That’s where context propagation in Phoenix comes in. And yes — it’s fully open source. GitHub:… — @arize-phoenix.bsky.social1 likes and repost · 1 reply
1 more on Bluesky
- May 29, 2025🐳 @arize.bsky.social OSS Prompt Playground @arize-phoenix.bsky.social gets Deepseek support! Now you can compare outputs of all the top… — @mikeldking.bsky.social1 likes and repost
On the web · 3
- May 5, 2026[agents] tracing project skills #13044 - Arize-ai/phoenix — google.com
- Feb 22, 2025When I manually call log_evaluations, no data shown in ... — google.com
- Apr 15, 2024[ENHANCEMENT] Support for Multi-Modal RAG Tracing ... — google.com
On Hacker News · 2
- Oct 4, 2023Show HN: Phoenix OSS – Applying LLM Spans, Traces, and Evals for AI Insights23 points · 3 comments
- May 22, 2023ML Observability in a Notebook2 points
On Stack Overflow · 1
- Jun 9, 2025Why would simple value change in WHERE filter could slow query performance by 5000x in postgresql0 votes · 2 answers
Other versions2 projects use a version of this.
Word for word · 0
None found.
Edited · 2
- github/awesome-copilot★ 41k91% same
- ReaperOAK/ForgeOS★ 093% same
Wrote their own version · 0
None found.
How matches are found. Word for word: the exact same text. Edited: shares passages with this file (“99% same” means 99% of its five-word phrases also appear here; 10% or more counts). Their own version: similar meaning, or, weakest, the same name. We can’t tell which project had it first. Projects are counted among the 200 most-starred candidates, and mentions are links to the repository we found, up to 50 per site, so both counts are a floor.
In this repository43 other files it gives its agents.
- AGENTS.mdAGENTS.md
- .agents/skills/vercel-react-best-practices/AGENTS.mdAGENTS.md
- .cursor/rules/client-migration.mdcCursor rule
- .cursor/rules/evals-migration.mdcCursor rule
- .cursor/rules/frontend/RULE.mdCursor rule
- .cursor/rules/python/RULE.mdCursor rule
- .cursor/rules/typescript-packages/RULE.mdCursor rule
- create-taskSkill
- daytonaSkill
- gh-stackSkill
- harbor-execSkill
- issue-triageSkill
- mintlifySkill
- phoenix-cloud-triageSkill
- phoenix-designSkill
- phoenix-docs-gap-auditSkill
- phoenix-docsSkill
- phoenix-evals-new-metricSkill
- phoenix-frontendSkill
- phoenix-githubSkill
- phoenix-integration-snippetsSkill
- phoenix-llms-txtSkill
- phoenix-playwright-testsSkill
- phoenix-pr-screenshotSkill
- phoenix-pxi-playwrightSkill
- phoenix-railway-triageSkill
- phoenix-release-notesSkill
- phoenix-release-pleaseSkill
- phoenix-rest-apiSkill
- phoenix-serverSkill
- phoenix-skills-auditSkill
- phoenix-sqleanSkill
- phoenix-storybookSkill
- phoenix-typescript-package-docsSkill
- phoenix-typescriptSkill
- rewardkitSkill
- test-integrationSkill
- typescript-tooling-migrationSkill
- vercel-react-best-practicesSkill
- phoenix-cliSkill
- phoenix-error-analysisSkill
- phoenix-harborSkill
- phoenix-tracingSkill
Does it work?
Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.
Reviews can't be read right now.
Your trail
Files you open will collect here, so you can find them again.
Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.
Now do this for your own.
Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.