agentleFS
acupof-aiacupof-ai/arle/.claude/skills/arle-upstream-runtime-scan/SKILL.md

arle-upstream-runtime-scan

Skill19 starsChanged 54 days agoMITView on GitHub

Use this skill when ckl asks to ground an ARLE serving/runtime, model-path, benchmark, capacity, Qwen3.5/DeepSeek, scheduler, paged_kv, MLX, or OPD-teacher decision in upstream SGLang/vLLM/TensorRT-LLM evidence before changing local code. It distills BBuf AI-Infra Auto Driven SKILLS into an ARLE-specific source-survey workflow without symlinking or vendoring those repositories.

The file

We split this skill into the rules it gives an agent

12 rules. About 0 are common, 5 are distinctive, and 7 are very specific to this project.

  • ≈0

    Common · estimate

    Many files like this have a rule of this kind.

  • ≈5

    Distinctive · estimate

    This author's own choices. Worth a look.

  • 7

    Very specific

    Tied to their project's names, files or tools. Probably skip.

  • Common
  • Distinctive
  • Very specific
  1. Very specific:

    Use it to decide what upstream evidence to read before an ARLE runtime change, then license or kill the local decision with ARLE logs, …

    line 10
  2. Distinctive:

    Do not use for:

    line 27
  3. Very specific:

    Start from ARLE truth surfaces: docs/index.md, relevant AGENTS.md, and current code.

    line 38
  4. Very specific:

    Treat upstream source survey as hypothesis-grade.

    line 40
  5. Very specific:

    If the lane is kernel-local, stop and switch to kernel-optimization.

    line 63
  6. Very specific:

    If already cloned, use git -C <repo> fetch --depth 1 and record git rev-parse HEAD.

    line 74
  7. Distinctive:

    Use model PR history when the local task touches model-specific runtime behavior or an external framework is already faster.

    line 94
  8. Very specific:

    Do not copy code snippets from upstream PR cards into ARLE.

    line 115
  9. Very specific:

    Use the BBuf benchmark material as a checklist, then run ARLE's own bench protocol.

    line 121
  10. Distinctive:

    Do not crown a framework winner until each requested framework has had its main serving knobs tuned or explicitly fixed by the experiment …

    line 135
  11. Distinctive:

    Keep prefill, decode, and mixed continuous-batch windows separate.

    line 169
  12. Distinctive:

    If this scan produces a code/docs change, record the source scan and the kept/killed verdicts in the PR description.

    line 195

≈ “Common” is an estimate: at least 1 in 5 of the 372,429 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.

Keep what fits and drop the rest. Your version, ready for your agents.
Files
SKILL.md

arle-upstream-runtime-scan

This is an ARLE source-survey skill, not a replacement for local measurement. Use it to decide what upstream evidence to read before an ARLE runtime change, then license or kill the local decision with ARLE logs, benches, traces, and tests.

Scope

Use for:

  • Comparing ARLE against SGLang, vLLM, TensorRT-LLM, or MLX-style serving.
  • Choosing a Qwen3.5, Qwen3-Next, DeepSeek V3/V4, MoE, attention, cache, sampler, loader, or quantization precedent before local implementation.
  • Explaining capacity, KV-cache budget, memory fraction, max tokens, continuous batching, prefix-cache, or queue-growth behavior.
  • Reading external profiler traces where prefill/decode stage separation matters.
  • Designing eval serving baselines that need a fair external reference.

Do not use for:

  • Local kernel retuning. Use kernel-optimization; BBuf KernelWiki and KernelPilot are merged there.
  • Pure SGLang PR review, production incident response, or architecture-diagram lookup. Those BBuf skills are not ARLE-specific enough.
  • Any claim that can be answered by local ARLE code, docs, or measurement without upstream context.

Non-Negotiables

  • Start from ARLE truth surfaces: docs/index.md, relevant AGENTS.md, and current code.
  • Treat upstream source survey as hypothesis-grade. It may suggest a design or risk; it does not prove ARLE behavior.
  • Fetch external repos into /tmp for inspection when needed. Do not clone, symlink, vendor, or copy BBuf repos into .claude/skills/.
  • Summarize and cite source URLs. Do not paste long source passages or port copied text into this repo.
  • Preserve ARLE benchmark protocol: matched workload, version/commit capture, internal counters when available, and wall-clock framing as ground truth.

Workflow

1. Classify The Question

Pick exactly one primary lane before reading upstream material:

LaneUse whenARLE anchor
Fair serving benchmarkckl asks "ARLE vs SGLang/vLLM" or wants a best external baselinedocs/bench-and-trace-spec.md, scripts/bench_throughput.py
Capacity / OOM / KV budgetmemory pool, max tokens, KV dtype, mem fraction, request capacitydocs/support-matrix.md, server logs, /v1/stats, nvidia-smi
Model PR historyQwen3.5, DeepSeek V4, Qwen3-Next, MoE, loader, sampler, cache pathcrates/infer-cuda/src/, crates/infer-metal/src/, crates/*-spec/
Trace triageexternal torch profiler or ARLE nsys/ncu points at a stagescripts/profile_*
Eval baselineserving correctness or capability numbers need external validationscripts/eval_harness/, scripts/needle_gate.py

If the lane is kernel-local, stop and switch to kernel-optimization.

2. Fetch Only What You Need

For a fresh upstream scan:

mkdir -p /tmp/arle-bbuf-scan
git clone --depth 1 https://github.com/BBuf/AI-Infra-Auto-Driven-SKILLS /tmp/arle-bbuf-scan/AI-Infra-Auto-Driven-SKILLS

If already cloned, use git -C <repo> fetch --depth 1 and record git rev-parse HEAD. Read only matching skill files or model-history pages.

Relevant source areas:

  • model-pr-optimization-history/SKILL.md
  • model-pr-optimization-history/{sglang,vllm}/{qwen35,qwen3-core,qwen3-next,deepseek-v4}/
  • skills/llm-serving-auto-benchmark/SKILL.md
  • skills/llm-serving-capacity-planner/SKILL.md
  • skills/llm-torch-profiler-analysis/SKILL.md
  • skills/model-compute-simulation/SKILL.md
  • skills/llm-pipeline-analysis/SKILL.md

Kill these by default for ARLE: sglang-humanize-review, sglang-prod-incident-triage, sglang-sota-humanize-loop, model-architecture-diagram. They are SGLang/plugin/operator UX skills unless ckl explicitly asks for that external task.

3. Model PR History Pass

Use model PR history when the local task touches model-specific runtime behavior or an external framework is already faster.

Useful queries from the BBuf checkout:

python3 model-pr-optimization-history/scripts/query.py --list
python3 model-pr-optimization-history/scripts/query.py --framework sglang --model qwen35 --paths-only
python3 model-pr-optimization-history/scripts/query.py --framework vllm --model deepseek-v4 "fused norm router" --limit 5
python3 model-pr-optimization-history/scripts/query.py --framework vllm "qwen3.5 gated delta net cache sampler" --limit 8

Extract only:

  • upstream PR number, URL, framework, status, and model family;
  • touched source files and symbols;
  • optimization type: fusion, overlap, quantization, attention/cache, sampler, loader, graph capture, or scheduler;
  • validation lane and regression risk;
  • exact ARLE file or module that might be influenced.

Do not copy code snippets from upstream PR cards into ARLE. If a code-level idea matters, inspect the real upstream source/PR under its license and record the URL in the PR description.

4. Fair Benchmark Pass

Use the BBuf benchmark material as a checklist, then run ARLE's own bench protocol.

Required matched-control fields:

  • model and checkpoint path;
  • tokenizer;
  • prompt/output distribution;
  • concurrency or request-rate profile;
  • max sequence length, batched-token budget, num slots, KV dtype, prefix cache, speculative decode, and graph/capture settings;
  • framework versions or commit hashes;
  • launch command, benchmark command, raw output table, and failed candidates.

Do not crown a framework winner until each requested framework has had its main serving knobs tuned or explicitly fixed by the experiment design. Do not search memory fractions by default unless capacity is the lane.

For ARLE, use scripts/bench_throughput.py and the report contract in docs/bench-and-trace-spec.md. External framework tools are references, not the ARLE truth surface.

5. Capacity / KV Budget Pass

Map external capacity concepts into ARLE knobs before proposing changes:

External conceptARLE question
weight load memorymodel residency and quantized loader behavior
KV pool memorymax_seq_len, num_slots, KV dtype, paged KV block bytes
graph/capture memorygraph key count, captured shapes, warmup budget
framework overheadscheduler/runtime buffers and allocator headroom
token capacityrequest shape x slot budget, not just aggregate tokens

Useful evidence:

  • startup log before/after weight load and KV allocation;
  • /v1/stats snapshots during steady state;
  • nvidia-smi per-rank memory;
  • model config.json for layers, heads, KV heads, hidden size, MoE/MLA/CSA fields;
  • ARLE bench envelope logs.

Flag claims about KV replication, SWA/CSA/HCA compression, or prefix cache hit rate as hypotheses until local logs or code paths confirm them.

6. Trace / Profiler Pass

Keep prefill, decode, and mixed continuous-batch windows separate. A hot decode kernel does not explain TTFT unless wall-clock framing says it does.

Output three small tables when trace evidence exists:

TablePurpose
Kernel tabletop kernels by stage and wall-clock share
Overlap tableCPU/GPU gaps, launch density, copies, syncs
Fuse tablesource-backed candidates, not fuzzy matches

For ARLE traces, prefer nsys/ncu wrappers and internal counters over torch-profiler. Use torch-profiler guidance only for external SGLang/vLLM/ TensorRT-LLM traces or for comparing stage naming.

7. Output Contract

Return or write:

  • upstream sources read, with repo URL and commit hash;
  • kept/killed upstream candidates and why;
  • ARLE modules affected or explicitly out of scope;
  • hypotheses generated;
  • local evidence still required to license or kill;
  • exact next command or patch boundary.

If this scan produces a code/docs change, record the source scan and the kept/killed verdicts in the PR description.

References

Download now (ZIP)

Does it work?

Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.

No reviews yet

People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.

  • It worked
  • Partly
  • It didn't
What did you use it for? e.g. Reviewing pull requests in a Rails app

Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.

Your trail

Files you open will collect here, so you can find them again.

Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.

Now do this for your own.

Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.