agentleFS
apacheapache/doris-skills/skills/doris-debug/query/SKILL.md

doris-debug-query

Skill50 starsChanged 13 days agoApache-2.0View on GitHub

Use for Apache Doris slow/hanging/timeout queries. Covers FE planner (Nereids timeout), BE Profile bottlenecks, and Exchange WaitForData / brpc E1008 on port 8060. Session vars: enable_profile, query_timeout, nereids_timeout_second.

The file

We split this skill into the rules it gives an agent

2 rules. About 0 are common, 1 is distinctive, and 1 is very specific to this project.

  • ≈0

    Common · estimate

    Many files like this have a rule of this kind.

  • ≈1

    Distinctive · estimate

    This author's own choices. Worth a look.

  • 1

    Very specific

    Tied to their project's names, files or tools. Probably skip.

  • Common
  • Distinctive
  • Very specific
  1. Distinctive:

    When Profile shows high active ExecTime (not Wait\*), route to doris-profile-reader for deep Profile analysis:

    line 39
  2. Very specific:

    Do not trust ICMP alone. Check priority_networks.

    line 81

≈ “Common” is an estimate: at least 1 in 5 of the 372,429 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.

Keep what fits and drop the rest. Your version, ready for your agents.
Files
SKILL.md

Query Troubleshooting

Causes

IDCauseEvidence
AClient timeout too smallquery_timeout vs Profile wall
BPlannerNereids cost too much time / nereids_timeout_second
CBE compute (scan/join/spill)Profile active ExecTime
DExchange / brpcWaitForData ≈ timeout; failed to send brpc when exchange; [E1008] on :8060

10 min

SET enable_profile = true;
-- reproduce
./scripts/doris-debug log-grep fe/log be/log --query-id "$QID" --pack planner
./scripts/doris-debug log-grep be/log --query-id "$QID" --pack exchange
./scripts/doris-debug audit-topk fe/log/fe.audit.log -k 20 --min-ms 5000

Planner mitigation (session only): SET disable_join_reorder=true; for fat ETL joins; fix stats with ANALYZE.

Cause C — BE compute (scan/join/spill)

When Profile shows high active ExecTime (not Wait*), route to doris-profile-reader for deep Profile analysis:

Skill routing: If doris-profile-reader is available, invoke it with the query's Profile to identify which operator (scan/join/agg/spill) dominates. Key indicators to pass:

  • ActiveTime per operator
  • RowsRead / ScanBytes for scan operators
  • SpillDataSize if disk spill occurred
  • JoinType + JoinRows for join operators

Without profile-reader, check directly:

# Get the Profile text
curl -s "http://$FE:8030/api/query_profile?query_id=$QID"

Key Profile counters for Cause C:

CounterMeaningThreshold
ScannerGetBlockTimeTime reading from storage> 50% of ExecTime → IO bound
SpillDataSizeData spilled to disk> 0 → memory pressure, check enable_spill
JoinProbeTimeHash join probe wall time> 30% → check join order / type
RowsRead / ScanBytesScan volume> 1B rows → check partition pruning

Profile interpretation: prefer active time over Wait*.

Cause D — Exchange / brpc

Code facts (apache/doris):

  1. Error text in be/src/exec/operator/exchange_sink_buffer.h.
  2. GET /api/reset_rpc_channel/{endpoints} clears brpc_internal_client_cache only (reset_rpc_channel_action.cpp).
  3. enable_brpc_connection_check (default false) → FragmentMgr::_check_brpc_available hand_shake; consecutive failures erase internal cache.
curl -s "http://$BE:8040/api/reset_rpc_channel/all"
# be.conf — restart for static DEFINE_Bool
enable_brpc_connection_check = true
brpc_connection_check_timeout_ms = 10000

Do not trust ICMP alone. Check priority_networks.

Source

  • SessionVariable.java — QUERY_TIMEOUT, NEREIDS_TIMEOUT_SECOND
  • Profile WaitForData — exchange source operator
  • exchange_sink_buffer.h, reset_rpc_channel_action.cpp, fragment_mgr.cpp

Download now (ZIP)

Does it work?

Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.

No reviews yet

People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.

  • It worked
  • Partly
  • It didn't
What did you use it for? e.g. Reviewing pull requests in a Rails app

Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.

Your trail

Files you open will collect here, so you can find them again.

Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.

Now do this for your own.

Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.