Langfuse

数据来源与分析方式

• 本地日志方式不依赖 Langfuse API、凭据或网络，不执行下文远端凭据预检。
• 用户只说“分析日志”而未明确来源时，先根据已提供的路径或 trace 信息定位；仍无法确定再询问，不默认查询远端。
• 两种来源不能默认互相替代。跨来源对照需核实 session、请求标识和时间范围；未采集或未返回的字段标记为未检查。
• 原始日志可能包含凭据和私密内容，不直接展示 headers、完整请求体或原始响应；先使用摘要，内容下钻前确认已脱敏。

1. Langfuse API via CLI

Use langfuse-cli to interact with the full Langfuse REST API. Run via bunx (auto-loads .env):

bunx langfuse-cli api --help                              # 列出所有 resources
bunx langfuse-cli api <resource> --help                     # List actions for a resource
bunx langfuse-cli api <resource> <action> --help            # Show args for an action
bunx langfuse-cli api <resource> <action> [options]         # Execute

Credentials

bunx automatically loads .env. Ensure it contains:

LANGFUSE_PUBLIC_KEY=pk-lf-...
LANGFUSE_SECRET_KEY=sk-lf-...
LANGFUSE_HOST=https://cloud.langfuse.com  # Required

If credentials are missing, ask the user to add them to .env. Do not ask to paste keys in chat.

CLI Preflight and Query Integrity

Before attributing missing or malformed data to application behavior:

1. Confirm LANGFUSE_HOST or LANGFUSE_BASE_URL is set, credentials are present, and the selected host returns JSON rather than an HTML fallback. Never print credentials or authorization headers.
2. Discover the installed CLI schema with bunx langfuse-cli api --help and resource/action --help; resource names vary by CLI version, so do not assume observations-v2s or another historical alias exists.
3. For list endpoints, inspect pagination metadata and fetch every required page. The public API page limit is 100; a single page is not proof of completeness.
4. Record the requested field projection. If input/output fields were not requested or returned by the selected endpoint, report them as not inspected, not null or missing.
5. Stop business-level diagnosis on host/auth/schema failure. A 401, unsupported resource, HTML response, or truncated page set is a query precondition failure, not evidence about the trace producer.

CLI Tips

• Use --json for machine-readable output
• Use --curl to preview HTTP request without executing
• Discover resources/actions with --help; do not hard-code version-specific v2 aliases
• Prefer the bundled scripts for traces and observations because they implement the current public endpoints and pagination

2. Data Retrieval Tools (脚本工具集)

本节 Langfuse 查询脚本的时间与元数据过滤选项如下；本地网关日志脚本的参数见 本地网关日志分析，不要混用两套参数：

2a. trace-search — 灵活搜索/过滤/导出

bun .claude/skills/langfuse/scripts/trace-search.ts [选项]

# 示例
bun .claude/skills/langfuse/scripts/trace-search.ts --days 7 --tag production                # 最近 7 天带 production tag 的 trace
bun .claude/skills/langfuse/scripts/trace-search.ts --session sess_abc --csv > session.csv   # 导出 session 为 CSV
bun .claude/skills/langfuse/scripts/trace-search.ts --model claude-sonnet --status error     # 查询特定模型的错误 trace
bun .claude/skills/langfuse/scripts/trace-search.ts --from 2026-07-01T00:00:00Z --summary   # 只看汇总统计
bun .claude/skills/langfuse/scripts/trace-search.ts --days 30 --json > report.json           # 导出 JSON
bun .claude/skills/langfuse/scripts/trace-search.ts --user user_123 --limit 100              # 按用户过滤
bun .claude/skills/langfuse/scripts/trace-search.ts --order latency.desc --limit 10          # 按延迟排序，找最慢的

Output modes: table (default), --csv, --json, --summary (aggregate only), --full (detailed fields).

2b. analyze — 成本/质量综合分析

bun .claude/skills/langfuse/scripts/analyze.ts [N]              # Overview + trace table + flags
bun .claude/skills/langfuse/scripts/analyze.ts --tools [N]      # Tool call analysis
bun .claude/skills/langfuse/scripts/analyze.ts --growth [N]     # Context growth trend
bun .claude/skills/langfuse/scripts/analyze.ts --report [N]     # Full report (all 7 sections)
bun .claude/skills/langfuse/scripts/analyze.ts --trace-id <id>  # Single trace detail

# 支持时间/元数据过滤
bun .claude/skills/langfuse/scripts/analyze.ts 20 --days 7 --user user_123 --report         # 某用户最近 7 天的完整报告

2c. session-analyze — Session 完整分析

bun .claude/skills/langfuse/scripts/session-analyze.ts --session <id> [选项]

# 选项
--limit <N>    最多拉取 trace 数（默认 100）
--detail       显示每个 trace 的逐轮 token 流
--csv          导出 CSV（每个 LLM 调用一行）

# 输出内容
# - Session 总体指标（traces, tokens, cost, time span）
# - Trace 时间线表格
# - 累积 token 增长趋势
# - 工具使用频率统计
# - 异常检测

2d. daily-report — 日报/周报

bun .claude/skills/langfuse/scripts/daily-report.ts [选项]

bun .claude/skills/langfuse/scripts/daily-report.ts                           # 今天的日报
bun .claude/skills/langfuse/scripts/daily-report.ts --days 7                  # 最近 7 天周报
bun .claude/skills/langfuse/scripts/daily-report.ts --days 30 --tag prod      # 按 tag 过滤的月报
bun .claude/skills/langfuse/scripts/daily-report.ts --model claude-sonnet     # 按模型过滤
bun .claude/skills/langfuse/scripts/daily-report.ts --detail                  # 显示所有 trace 详情

# 输出内容
# - Key Metrics（traces, sessions, errors, tokens, cost）
# - By Model 分布
# - Top Users（按输入 token）
# - Top Traces（按输入 token）
# - 异常 trace 列表

2e. 单 trace 深度分析

# Token 流 + 缓存异常
bun .claude/skills/langfuse/scripts/trace-tokens.ts <traceId>
bun .claude/skills/langfuse/scripts/trace-tokens.ts --index 1 --days 7        # 用 --index 从过滤结果中选 trace

# 消息组成 + diff
bun .claude/skills/langfuse/scripts/trace-messages.ts <traceId> [--detail]
bun .claude/skills/langfuse/scripts/trace-messages.ts --index 3 --user user_123

# System prompt 段落拆解
bun .claude/skills/langfuse/scripts/prompt-breakdown.ts <traceId>
bun .claude/skills/langfuse/scripts/prompt-breakdown.ts --index 1 --days 7

# Trace 汇总列表
bun .claude/skills/langfuse/scripts/traces-list.ts [N] [过滤选项]

2f. trace-tree — observation parent/orphan 审计

bun .claude/skills/langfuse/scripts/trace-tree.ts <traceId>

This command fetches all observation pages, prints a metadata-only tree, and exits non-zero when it finds duplicate IDs, missing parent observations, or cycles. A parent equal to the trace ID is a valid root attachment. Use it whenever the diagnosis concerns subagent ownership, generation/tool/batch nesting, or orphan observations; do not infer parent integrity from a flat list.

Production Verification Gate

A unit/mock pass proves only local construction. After changing instrumentation or parent assignment:

1. Restart the actual producer process and record the new process/session provenance without exposing secrets.
2. Generate a new real trace after restart; do not reuse pre-fix data as acceptance evidence.
3. Run trace-tree.ts on that trace and inspect expected generation/tool/batch ownership.
4. Report code tests and production trace verification separately. If restart, credentials, or a live trace is unavailable, mark production verification blocked rather than complete.

3. Query Recipes（常见数据获取场景）

按时间查询

按用户/会话查询

成本排查

质量排查

模型对比

调试 Prompt

4. Data Retrieval Patterns（按目的选择工具）

日常监控 → daily-report.ts

快速了解系统状态：今天/本周有多少 trace、花了多少钱、有没有异常。每天跑一次即可。

深入问题诊断 → analyze.ts --report

当发现异常（成本飙升、缓存降低、用户反馈质量差）时，对最近 N 条 trace 做全维度扫描。

精准搜索 → trace-search.ts

当你已经知道要找什么（某用户、某 session、某时间段、某模型），直接筛选。支持导出 CSV/JSON 做进一步分析。

单条追踪 → trace-tokens.ts + trace-messages.ts + prompt-breakdown.ts

定位到具体 trace 后，这三件套分别看 token 流、消息变化、prompt 结构，逐轮定位问题。

Session 回溯 → session-analyze.ts

需要完整还原用户的一次会话时使用，看 trace 时间线、token 累积、工具使用演变。

Prompt 工程 → prompt-breakdown.ts + CLI get prompt

先看现有 request 中 system prompt 的段落分布（哪些段落最大），然后用 CLI 管理 Langfuse prompt：

bunx langfuse-cli api prompts list
bunx langfuse-cli api prompts get --name <name>
bunx langfuse-cli api prompts create --name <name> --type chat --prompt '[...]'

5. Cost Analysis（详细版）

Report Sections

Red Flags

Optimization Checklist

After analysis, evaluate:

1. System Prompt Weight — >40% of context → trim; largest section → shorten or lazy-load; stale CLAUDE.md TRAPs → archive
2. Context Accumulation — tool results retained across turns?; micro-compact threshold right?; redundant reads?
3. Agent Loop Efficiency — redundant tool calls?; sequential reads → batch?; broad exploration → targeted search?
4. Task Decomposition — complex task → focused sub-tasks?; sub-agents to reduce context pressure?

Reflection Output Format

## Cost Reflection

### Metrics
- Traces analyzed: N
- Total input: X tokens (Y% cache hit)
- Total output: Z tokens
- Avg LLM calls per trace: M

### Findings
1. [Pattern with specific trace example]
2. [Another pattern]

### Recommendations
1. [Actionable optimization] — estimated savings: ~X tokens/trace
2. [Another recommendation]

6. Langfuse Documentation

6a. Documentation Index (llms.txt)

curl -s https://langfuse.com/llms.txt

Returns structured list of every doc page. Use to discover the right page, then fetch it.

6b. Fetch Pages as Markdown

Append .md to any doc path:

curl -s "https://langfuse.com/docs/observability/overview.md"

6c. Search Documentation

curl -s "https://langfuse.com/api/search-docs?query=How+do+I+trace+LangGraph+agents"

Returns matching documents with URLs, titles, and excerpts. Also indexes GitHub Issues/Discussions.

Workflow

1. Start with llms.txt to orient
2. Fetch specific pages when identified
3. Fall back to search when topic is unclear

7. 上下文 Diff 诊断（对比两次 LLM 调用的完整输入）

当不同 trace/session 的 input tokens 存在无法解释的差异时，下载完整 input 做 diff 是最直接的定位手段。

步骤

1. 找到差异 trace 的 generation observation ID

# 列出 session 的所有 trace
bunx langfuse-cli api traces list --session-id <session_id> --json | jq '.body.data[].id'

# 列出 trace 下所有 GENERATION observation
curl -s -u "$LANGFUSE_PUBLIC_KEY:$LANGFUSE_SECRET_KEY" \
  "$LANGFUSE_HOST/api/public/observations?traceId=<trace_id>&limit=100" \
  | jq '[.data[] | select(.type == "GENERATION") | {id, inputTokens: .usageDetails.input}]'

2. 下载完整 input 并保存

curl -s -u "$LANGFUSE_PUBLIC_KEY:$LANGFUSE_SECRET_KEY" \
  "$LANGFUSE_HOST/api/public/observations/<obs_id>" \
  | jq '.input' > /tmp/input_a.json

curl -s -u "$LANGFUSE_PUBLIC_KEY:$LANGFUSE_SECRET_KEY" \
  "$LANGFUSE_HOST/api/public/observations/<obs_id>" \
  | jq '.input' > /tmp/input_b.json

3. Diff

diff /tmp/input_a.json /tmp/input_b.json

典型场景

注意

• .input 是完整请求体（包含 messages、tools、model 等字段），diff 能精确定位任何差异
• 如果只需要比 system prompt：jq '.input.messages[0].content' -r
• 如果只需要比 tools：jq '.input.tools'
• Generation observation 的 usageDetails 包含 cache_read_input_tokens 和 cache_creation_input_tokens，是缓存诊断的关键数据

Use Case References

• instrumenting an application: references/instrumentation.md
• migrating prompts: references/prompt-migration.md
• user feedback as scores: references/user-feedback.md
• CLI tips: references/cli.md
• SDK upgrade: references/sdk-upgrade.md
• judge calibration: references/judge-calibration.md
• error analysis: references/error-analysis.md
• skill feedback: references/skill-feedback.md