arle-upstream-runtime-scan
Skill19 starsChanged 54 days agoMITView on GitHub
Use this skill when ckl asks to ground an ARLE serving/runtime, model-path, benchmark, capacity, Qwen3.5/DeepSeek, scheduler, paged_kv, MLX, or OPD-teacher decision in upstream SGLang/vLLM/TensorRT-LLM evidence before changing local code. It distills BBuf AI-Infra Auto Driven SKILLS into an ARLE-specific source-survey workflow without symlinking or vendoring those repositories.
In this repository6 other files it gives its agents.
- AGENTS.mdAGENTS.md
- CLAUDE.mdCLAUDE.md
- find-simplificationsSkill
- kernel-optimizationSkill
- tmux-agent-controlSkill
- understand-until-simpleSkill
The file
We split this skill into the rules it gives an agent
12 rules. About 0 are common, 5 are distinctive, and 7 are very specific to this project.
≈0
Common · estimate
Many files like this have a rule of this kind.
≈5
Distinctive · estimate
This author's own choices. Worth a look.
7
Very specific
Tied to their project's names, files or tools. Probably skip.
- Common
- Distinctive
- Very specific
- Very specific: line 10
Use it to decide what upstream evidence to read before an ARLE runtime change, then license or kill the local decision with ARLE logs, …
- Distinctive: line 27
Do not use for:
- Very specific: line 38
Start from ARLE truth surfaces: docs/index.md, relevant AGENTS.md, and current code.
- Very specific: line 40
Treat upstream source survey as hypothesis-grade.
- Very specific: line 63
If the lane is kernel-local, stop and switch to kernel-optimization.
- Very specific: line 74
If already cloned, use git -C <repo> fetch --depth 1 and record git rev-parse HEAD.
- Distinctive: line 94
Use model PR history when the local task touches model-specific runtime behavior or an external framework is already faster.
- Very specific: line 115
Do not copy code snippets from upstream PR cards into ARLE.
- Very specific: line 121
Use the BBuf benchmark material as a checklist, then run ARLE's own bench protocol.
- Distinctive: line 135
Do not crown a framework winner until each requested framework has had its main serving knobs tuned or explicitly fixed by the experiment …
- Distinctive: line 169
Keep prefill, decode, and mixed continuous-batch windows separate.
- Distinctive: line 195
If this scan produces a code/docs change, record the source scan and the kept/killed verdicts in the PR description.
≈ “Common” is an estimate: at least 1 in 5 of the 372,429 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.
Files
arle-upstream-runtime-scan
This is an ARLE source-survey skill, not a replacement for local measurement. Use it to decide what upstream evidence to read before an ARLE runtime change, then license or kill the local decision with ARLE logs, benches, traces, and tests.
Scope
Use for:
- Comparing ARLE against SGLang, vLLM, TensorRT-LLM, or MLX-style serving.
- Choosing a Qwen3.5, Qwen3-Next, DeepSeek V3/V4, MoE, attention, cache, sampler, loader, or quantization precedent before local implementation.
- Explaining capacity, KV-cache budget, memory fraction, max tokens, continuous batching, prefix-cache, or queue-growth behavior.
- Reading external profiler traces where prefill/decode stage separation matters.
- Designing eval serving baselines that need a fair external reference.
Do not use for:
- Local kernel retuning. Use
kernel-optimization; BBuf KernelWiki and KernelPilot are merged there. - Pure SGLang PR review, production incident response, or architecture-diagram lookup. Those BBuf skills are not ARLE-specific enough.
- Any claim that can be answered by local ARLE code, docs, or measurement without upstream context.
Non-Negotiables
- Start from ARLE truth surfaces:
docs/index.md, relevantAGENTS.md, and current code. - Treat upstream source survey as hypothesis-grade. It may suggest a design or risk; it does not prove ARLE behavior.
- Fetch external repos into
/tmpfor inspection when needed. Do not clone, symlink, vendor, or copy BBuf repos into.claude/skills/. - Summarize and cite source URLs. Do not paste long source passages or port copied text into this repo.
- Preserve ARLE benchmark protocol: matched workload, version/commit capture, internal counters when available, and wall-clock framing as ground truth.
Workflow
1. Classify The Question
Pick exactly one primary lane before reading upstream material:
| Lane | Use when | ARLE anchor |
| Fair serving benchmark | ckl asks "ARLE vs SGLang/vLLM" or wants a best external baseline | docs/bench-and-trace-spec.md, scripts/bench_throughput.py |
| Capacity / OOM / KV budget | memory pool, max tokens, KV dtype, mem fraction, request capacity | docs/support-matrix.md, server logs, /v1/stats, nvidia-smi |
| Model PR history | Qwen3.5, DeepSeek V4, Qwen3-Next, MoE, loader, sampler, cache path | crates/infer-cuda/src/, crates/infer-metal/src/, crates/*-spec/ |
| Trace triage | external torch profiler or ARLE nsys/ncu points at a stage | scripts/profile_* |
| Eval baseline | serving correctness or capability numbers need external validation | scripts/eval_harness/, scripts/needle_gate.py |
If the lane is kernel-local, stop and switch to kernel-optimization.
2. Fetch Only What You Need
For a fresh upstream scan:
mkdir -p /tmp/arle-bbuf-scan
git clone --depth 1 https://github.com/BBuf/AI-Infra-Auto-Driven-SKILLS /tmp/arle-bbuf-scan/AI-Infra-Auto-Driven-SKILLSIf already cloned, use git -C <repo> fetch --depth 1 and record
git rev-parse HEAD. Read only matching skill files or model-history pages.
Relevant source areas:
model-pr-optimization-history/SKILL.mdmodel-pr-optimization-history/{sglang,vllm}/{qwen35,qwen3-core,qwen3-next,deepseek-v4}/skills/llm-serving-auto-benchmark/SKILL.mdskills/llm-serving-capacity-planner/SKILL.mdskills/llm-torch-profiler-analysis/SKILL.mdskills/model-compute-simulation/SKILL.mdskills/llm-pipeline-analysis/SKILL.md
Kill these by default for ARLE: sglang-humanize-review,
sglang-prod-incident-triage, sglang-sota-humanize-loop,
model-architecture-diagram. They are SGLang/plugin/operator UX skills unless
ckl explicitly asks for that external task.
3. Model PR History Pass
Use model PR history when the local task touches model-specific runtime behavior or an external framework is already faster.
Useful queries from the BBuf checkout:
python3 model-pr-optimization-history/scripts/query.py --list
python3 model-pr-optimization-history/scripts/query.py --framework sglang --model qwen35 --paths-only
python3 model-pr-optimization-history/scripts/query.py --framework vllm --model deepseek-v4 "fused norm router" --limit 5
python3 model-pr-optimization-history/scripts/query.py --framework vllm "qwen3.5 gated delta net cache sampler" --limit 8Extract only:
- upstream PR number, URL, framework, status, and model family;
- touched source files and symbols;
- optimization type: fusion, overlap, quantization, attention/cache, sampler, loader, graph capture, or scheduler;
- validation lane and regression risk;
- exact ARLE file or module that might be influenced.
Do not copy code snippets from upstream PR cards into ARLE. If a code-level idea matters, inspect the real upstream source/PR under its license and record the URL in the PR description.
4. Fair Benchmark Pass
Use the BBuf benchmark material as a checklist, then run ARLE's own bench protocol.
Required matched-control fields:
- model and checkpoint path;
- tokenizer;
- prompt/output distribution;
- concurrency or request-rate profile;
- max sequence length, batched-token budget, num slots, KV dtype, prefix cache, speculative decode, and graph/capture settings;
- framework versions or commit hashes;
- launch command, benchmark command, raw output table, and failed candidates.
Do not crown a framework winner until each requested framework has had its main serving knobs tuned or explicitly fixed by the experiment design. Do not search memory fractions by default unless capacity is the lane.
For ARLE, use scripts/bench_throughput.py and the report contract in
docs/bench-and-trace-spec.md. External framework tools are references, not the ARLE
truth surface.
5. Capacity / KV Budget Pass
Map external capacity concepts into ARLE knobs before proposing changes:
| External concept | ARLE question |
| weight load memory | model residency and quantized loader behavior |
| KV pool memory | max_seq_len, num_slots, KV dtype, paged KV block bytes |
| graph/capture memory | graph key count, captured shapes, warmup budget |
| framework overhead | scheduler/runtime buffers and allocator headroom |
| token capacity | request shape x slot budget, not just aggregate tokens |
Useful evidence:
- startup log before/after weight load and KV allocation;
/v1/statssnapshots during steady state;nvidia-smiper-rank memory;- model
config.jsonfor layers, heads, KV heads, hidden size, MoE/MLA/CSA fields; - ARLE bench envelope logs.
Flag claims about KV replication, SWA/CSA/HCA compression, or prefix cache hit rate as hypotheses until local logs or code paths confirm them.
6. Trace / Profiler Pass
Keep prefill, decode, and mixed continuous-batch windows separate. A hot decode kernel does not explain TTFT unless wall-clock framing says it does.
Output three small tables when trace evidence exists:
| Table | Purpose |
| Kernel table | top kernels by stage and wall-clock share |
| Overlap table | CPU/GPU gaps, launch density, copies, syncs |
| Fuse table | source-backed candidates, not fuzzy matches |
For ARLE traces, prefer nsys/ncu wrappers and internal counters over torch-profiler. Use torch-profiler guidance only for external SGLang/vLLM/ TensorRT-LLM traces or for comparing stage naming.
7. Output Contract
Return or write:
- upstream sources read, with repo URL and commit hash;
- kept/killed upstream candidates and why;
- ARLE modules affected or explicitly out of scope;
- hypotheses generated;
- local evidence still required to license or kill;
- exact next command or patch boundary.
If this scan produces a code/docs change, record the source scan and the kept/killed verdicts in the PR description.
References
- BBuf AI-Infra-Auto-Driven-SKILLS: https://github.com/BBuf/AI-Infra-Auto-Driven-SKILLS
- Model PR history: https://github.com/BBuf/AI-Infra-Auto-Driven-SKILLS/tree/main/model-pr-optimization-history
- LLM serving benchmark skill: https://github.com/BBuf/AI-Infra-Auto-Driven-SKILLS/tree/main/skills/llm-serving-auto-benchmark
- LLM capacity planner skill: https://github.com/BBuf/AI-Infra-Auto-Driven-SKILLS/tree/main/skills/llm-serving-capacity-planner
- LLM torch profiler skill: https://github.com/BBuf/AI-Infra-Auto-Driven-SKILLS/tree/main/skills/llm-torch-profiler-analysis
- Model compute simulation skill: https://github.com/BBuf/AI-Infra-Auto-Driven-SKILLS/tree/main/skills/model-compute-simulation
Does it work?
Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.
No reviews yet
People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.
- It worked
- Partly
- It didn't
Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.
Your trail
Files you open will collect here, so you can find them again.
Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.
Now do this for your own.
Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.

