flamegraph
Skill1.1k starsChanged 6 days agoMITView on GitHub
Parse and analyze async-profiler flamegraph HTML files.
Trending
Less talk lately.about the whole repository
Who's talking about hail-is/hail
- ipython/ipython“In ipython 8.18 (but not ipython 8.16), the current event loop is overwritten in each cell”GitHub · Jan 17, 2024 · 7 reactions
- @stephenturner.us“The Scalable Variant Call Representation: Enabling Genetic Analysis Beyond One Million Genomes …”Bluesky · Jan 12, 2024 · 7 likes
- has2k1/plotnine“Request: re-license under MIT?”GitHub · Dec 6, 2021 · 3 reactions
- Stack Overflow“Get relative path from inside a scala library (while developing that library)”Stack Overflow · Nov 27, 2018 · 3 votes
- d12frosted/homebrew-emacs-plus“Fix linking how-to in formula caveats”GitHub · Dec 21, 2023 · 2 reactions
The biggest 5 of 54 mentions linking to the repository that our crawl found. Each line is their own headline.
See all 54
Talked about on GitHub, Stack Overflow, Hacker News and Bluesky (54 times).
On GitHub · 50
- Jan 17, 2024In ipython 8.18 (but not ipython 8.16), the current event loop is overwritten in each cell — ipython/ipython7 reactions · 2 comments
- Dec 6, 2021Request: re-license under MIT? — has2k1/plotnine3 reactions · 1 comment
- Dec 21, 2023Fix linking how-to in formula caveats — d12frosted/homebrew-emacs-plus2 reactions
- Dec 14, 2023Incorrect lint in 0.1.8? RUF006 Store a reference to the return value of `asyncio.create_task` — astral-sh/ruff2 reactions · 9 comments
- Nov 4, 2026New hail-is/hail version available: 0.2.140 — TheBoutrosLab/docker-Hail1 reaction
45 more on GitHub
- Aug 20, 2026New hail-is/hail version available: 0.2.139 — TheBoutrosLab/docker-Hail1 reaction
- Feb 7, 2021Hail developer set-up — populationgenomics/team-docs1 reaction
- Aug 4, 2026Bump the pip group across 1 directory with 5 updates — broadinstitute/variant-curation-portal0 reactions
- Jun 24, 2026Update hadoop to hailtop after hail's hadoop deprecation — broadinstitute/gnomad_methods0 reactions · 2 comments
- Jun 16, 2026[skill-suggestion] hail: scalable genomic analysis and GWAS engine (Broad Institute) — stanfish06/skillquarium0 reactions
- Sep 11, 2025[explore] Variant store: Hail — tidywf/nemo0 reactions
- Jul 25, 2025Add LEN to compute_stats_per_ref_site after hail updates — broadinstitute/gnomad_methods0 reactions
- Mar 14, 2025Temporarily revert username validation, as new rule disallows dashes — populationgenomics/hail0 reactions
- Jan 10, 2025Refine combiner usage — populationgenomics/production-pipelines0 reactions · 1 comment
- Dec 16, 2024fix: remove obsolete Hortonworks repositories — h2oai/h2o-30 reactions · 6 comments
- Dec 15, 2024Cherry-pick upstream fix: Set StreamReadConstraints maxStringLength as early as possible — populationgenomics/hail0 reactions
- Sep 20, 2024Bump the pip group with 4 updates — broadinstitute/variant-curation-portal0 reactions · 2 comments
- Jul 15, 2024chore(deps): bump the pip group across 2 directories with 15 updates — broadinstitute/gnomad-browser0 reactions · 4 comments
- Jul 5, 2024chore(deps): bump the pip group across 2 directories with 14 updates — broadinstitute/gnomad-browser0 reactions · 1 comment
- Jun 17, 2024chore(deps): bump the pip group across 3 directories with 14 updates — broadinstitute/gnomad-browser0 reactions · 1 comment
- Jun 11, 2024chore(deps): bump the pip group across 3 directories with 15 updates — broadinstitute/gnomad-browser0 reactions · 2 comments
- Jun 11, 2024chore(deps): bump the pip group across 3 directories with 14 updates — broadinstitute/gnomad-browser0 reactions · 1 comment
- Jun 10, 2024chore(deps): bump the pip group across 3 directories with 13 updates — broadinstitute/gnomad-browser0 reactions · 1 comment
- Jun 6, 2024chore(deps): bump the pip group in /data-pipeline with 12 updates — broadinstitute/gnomad-browser0 reactions · 1 comment
- May 14, 2024Fix syntax — broadinstitute/seqr-loading-pipelines0 reactions
- May 1, 2024Local_to_global: local alleles not strictly increasing: [0,2,1] — iris-garden/test-process0 reactions
- May 1, 2024Where to find data for practicing using Hail — iris-garden/test-process0 reactions
- Apr 30, 2024De novo confidence queries — iris-garden/test-process0 reactions
- Apr 30, 2024DNANexus: hail installation on custom spark cluster — iris-garden/test-process0 reactions
- Apr 30, 2024Hail architecture or design philosophy — iris-garden/test-process0 reactions
- Apr 30, 2024Assertion Error during entries().show() — iris-garden/test-process0 reactions
- Apr 30, 2024Dataproc connect should support --project command line option — iris-garden/test-process0 reactions
- Apr 30, 2024CollectionExpression join — iris-garden/test-process0 reactions
- Apr 30, 2024Use agg.linreg for GWAS — iris-garden/test-process0 reactions
- Apr 30, 2024Limit memory usage — iris-garden/test-process0 reactions
- Apr 30, 2024ArrayIndexOutOfBoundsException error in hail 0.2.40 — iris-garden/test-process0 reactions
- Apr 30, 2024Import_bgen 16 bits error — iris-garden/test-process0 reactions
- Apr 30, 2024VCF4.1, is.hail.utils.HailException: array index out of bounds — iris-garden/test-process0 reactions
- Apr 30, 2024How to get the ration transition-transversion on all dataset? — iris-garden/test-process0 reactions
- Apr 30, 2024GQ set to NA when PL is NA after split_multi_hts — iris-garden/test-process0 reactions
- Apr 30, 2024Download for tutorial notebooks — iris-garden/test-process0 reactions
- Apr 30, 2024Typo in impute_sex() description — iris-garden/test-process0 reactions
- Apr 30, 2024Hail V01 to V02 missing genotypes? — iris-garden/test-process0 reactions
- Apr 30, 2024Import_table on gs with wildcard — iris-garden/test-process0 reactions
- Apr 30, 2024Hail 0.2.10 patch notes — iris-garden/test-process0 reactions
- Apr 30, 2024Interval.parse in v0.2 — iris-garden/test-process0 reactions
- Apr 30, 2024Hail 0.2.6 patch notes — iris-garden/test-process0 reactions
- Apr 30, 2024Hail 0.2.5 patch notes — iris-garden/test-process0 reactions
- Apr 30, 2024Chained logistic regression — iris-garden/test-process0 reactions
- Apr 30, 2024[Breaking change] partition keys removed from python interface — iris-garden/test-process0 reactions
On Stack Overflow · 2
On Hacker News · 1
- Aug 28, 2020Hail.is: open-source library for exploring genomic data1 point
On Bluesky · 1
- Jan 12, 2024The Scalable Variant Call Representation: Enabling Genetic Analysis Beyond One Million Genomes www.biorxiv.org/content/10.1... 🧬🖥️… — @stephenturner.us7 likes and reposts
In this repository5 other files it gives its agents.
- hail/AGENTS.mdAGENTS.md
- hail/CLAUDE.mdCLAUDE.md
- hail-batch-devSkill
- hail-batchSkill
- release-changelogSkill
The file
We split this skill into the rules it gives an agent
6 rules. About 0 are common, 3 are distinctive, and 3 are very specific to this project.
≈0
Common · estimate
Many files like this have a rule of this kind.
≈3
Distinctive · estimate
This author's own choices. Worth a look.
3
Very specific
Tied to their project's names, files or tools. Probably skip.
- Common
- Distinctive
- Very specific
- Very specific: line 104
Run the comparison: python3 ...
- Distinctive: line 105
Check the sample ratio — is there an overall slowdown?
- Distinctive: line 106
Check the "self-time EXCESS vs uniform scaling" section first: if no function exceeds uniform scaling, stop blaming the code — verify the …
- Distinctive: line 107
Look at self-time changes to find where new CPU work is happening
- Very specific: line 109
Check for structural changes: new split_ToArray (materialization regression), new GCS write paths, new decode steps
- Very specific: line 111
Use --callers to trace call paths to any new hotspot: --callers 'HotFunction' --depth 15
≈ “Common” is an estimate: at least 1 in 5 of the 372,429 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.
Files
Flamegraph Analysis
Parse async-profiler flamegraph HTML files to extract structured profiling data. These files encode stack traces in a compact JavaScript format (constant pool + frame calls) that browsers render as interactive flamegraphs. This skill extracts that data into tabular or JSON form for analysis.
Parsing tool
scripts/parse_flamegraph.py (relative to this skill directory) is a standalone Python 3 script with no dependencies beyond the standard library.
Single file — summarize a profile
python3 <skill-dir>/scripts/parse_flamegraph.py profile.html
python3 <skill-dir>/scripts/parse_flamegraph.py profile.html --top 30
python3 <skill-dir>/scripts/parse_flamegraph.py profile.html --filter 'is/hail'Two files — compare profiles for regressions
python3 <skill-dir>/scripts/parse_flamegraph.py before.html after.html
python3 <skill-dir>/scripts/parse_flamegraph.py before.html after.html --filter 'is/hail|__C|split_|DECODE'The comparison shows:
- Total sample counts and their ratio
- Functions with the largest increase/decrease in inclusive %
- Self-time changes (where CPU time is actually spent, not just call-tree ancestry)
- Self-time excess vs uniform scaling (see below — the key regression-vs-noise discriminator)
- Functions unique to each profile
Single file — trace call paths to a hot function
python3 <skill-dir>/scripts/parse_flamegraph.py profile.html --callers 'ArrayBuilder'
python3 <skill-dir>/scripts/parse_flamegraph.py profile.html --callers 'DECODE' --top 10
python3 <skill-dir>/scripts/parse_flamegraph.py profile.html --callers 'Region.*allocate' --depth 12Shows the full call stacks leading to every frame matching the regex, grouped by unique stack and sorted by sample count. Recursive cycles (e.g., mutual recursion between applyTable and lower$2) are automatically collapsed with a repetition count (x29). When the matched function is itself recursive, only the topmost occurrence per stack is traced, so each sample is counted once.
Flags
| Flag | Effect |
--top N / -n N | Show top N entries per section (default 20) |
--filter REGEX / -f REGEX | Only show functions matching the regex (summary/comparison modes) |
--json / -j | Output as JSON for programmatic use |
--callers REGEX / -c REGEX | Trace call paths to functions matching the regex (single file only) |
--depth N / -d N | Max stack depth to show with --callers (default 20, 0 = unlimited) |
JSON mode
Use --json when you need to do further computation on the results (e.g., feeding into another script or building a report). The output is a single JSON object with inclusive_top and self_time_top arrays (single file), by_increase/by_decrease/by_self_excess arrays (comparison), or stacks array (--callers). In --callers --json mode, recursive cycles appear as {"cycle": [...], "count": N} objects within the frames array.
How to interpret the output
Inclusive time counts every sample where a function appears anywhere in the call stack, counted once per sample: recursive functions are attributed at their topmost occurrence, so percentages are always ≤ 100%. (Naively summing frame widths would multiply a recursive walk's time by its recursion depth — a tree traversal 40 frames deep would misleadingly report 40× its true cost.)
Self time counts only samples where the function is at the top of the stack (the CPU was executing that function, not one of its callees). This is where the CPU is actually spending cycles. Regressions in self-time point to the actual bottleneck; regressions in inclusive-time point to the call subtree containing the bottleneck.
Run-specific names are normalized so the same code aligns across profiles: JVM lambda classes (Foo$$Lambda$733/101464973 → Foo$$Lambda$*), reflection accessors (GeneratedMethodAccessor120 → GeneratedMethodAccessor*), and Hail's generated classes/methods (__C197... → __C*..., __m1206split_Block → __m*split_Block). Compare Hail generated code by the method suffix (split_StreamFor, DECODE_r_struct_of_..., btree_get, ...).
When comparing two profiles:
- The sample ratio tells you the overall slowdown/speedup — but check whether the two captures are comparable (same capture window, same number of requests, same machine) before attributing it to code.
- The "self-time EXCESS vs uniform scaling" section is the key discriminator: it ranks functions by how much their self-time deviates from
samples_a × ratio. A localized code regression shows a large positive excess in specific functions; if every excess is near zero, the profiles have the same shape and the ratio comes from something global — a longer capture window, more requests captured, or a slower/contended CPU — not a code change. - Large increases in self-time % pinpoint where new work is being done.
- Functions present in only one profile reveal structural changes to the code path (renamed functions, new call wrappers, refactored APIs). One recurring false positive:
NativeMethodAccessorImpl.invokein one profile vsGeneratedMethodAccessor*.invokein the other is JVM reflection warmup (the JVM swaps in a generated accessor after ~16 calls), not a code change. - For JVM driver profiles, break down by thread group first: JIT compiler threads (
CompileBroker::compiler_thread_loop) and GC workers (GangWorker::loop) often dominate total CPU. A regression confined to worker threads is a code change; uniform growth across JIT + GC + workers points at the environment.
Analyzing Hail profiles specifically
Hail's JIT-compiled query code appears as __C<N>collect_distributed_array_... with generated method names:
| Pattern | Meaning |
split_StreamFor | The main iteration loop over partitioned data |
split_ToArray | Materializing a stream into an array — an extra one signals a lowering regression |
DECODE_* / INPLACE_DECODE_* | Deserializing encoded data from Region memory |
btree_get | B-tree index lookup (aggregation key lookup) |
compWithKey / ord_compare | Key comparison during aggregation |
arrayref_bounds_check | Array bounds checking in generated code |
position | Stream position tracking |
Key Hail runtime functions to watch:
| Function | What it does |
Region$.loadByte / loadBit / loadInt | Low-level memory reads from Region memory |
Region$.storeByte / setBit | Memory writes |
RegionMemory.allocate / free | Region memory management |
LEB128InputBuffer.readByte / readInt | Variable-length integer decoding |
BlockingInputBuffer.ensure / readByte | Buffered I/O from encoded data |
LZ4.decompress / LZ4InputBlockBuffer.readBlock | Block decompression |
GoogleStorageFS.* | GCS I/O — check for unexpected reads/writes |
AnyRefArrayBuilder / LongArrayBuilder | Array construction during aggregation |
Workflow for regression analysis
- Run the comparison:
python3 ... before.html after.html --top 30 - Check the sample ratio — is there an overall slowdown?
- Check the "self-time EXCESS vs uniform scaling" section first: if no function exceeds uniform scaling, stop blaming the code — verify the captures are comparable (window length, request count, machine/CPU contention) before digging further
- Look at self-time changes to find where new CPU work is happening
- Filter to Hail-specific code:
--filter 'hail|__C|split_|DECODE|Region|Memory|LZ4|btree' - Check for structural changes: new
split_ToArray(materialization regression), new GCS write paths, new decode steps - Cross-reference with the "functions only in X" sections to find renamed/refactored code paths
- Use
--callersto trace call paths to any new hotspot:--callers 'HotFunction' --depth 15 - Summarize findings: what changed in the generated code, what changed in the runtime, what's the likely root cause
Does it work?
Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.
No reviews yet
People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.
- It worked
- Partly
- It didn't
Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.
Your trail
Files you open will collect here, so you can find them again.
Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.
Now do this for your own.
Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.

