hail-batch
Skill1.1k starsChanged 6 days agoMITView on GitHub
Submit jobs, debug failures, and write Hail Batch submission scripts as an end user
Trending
Less talk lately.about the whole repository
Who's talking about hail-is/hail
- ipython/ipython“In ipython 8.18 (but not ipython 8.16), the current event loop is overwritten in each cell”GitHub · Jan 17, 2024 · 7 reactions
- @stephenturner.us“The Scalable Variant Call Representation: Enabling Genetic Analysis Beyond One Million Genomes …”Bluesky · Jan 12, 2024 · 7 likes
- has2k1/plotnine“Request: re-license under MIT?”GitHub · Dec 6, 2021 · 3 reactions
- Stack Overflow“Get relative path from inside a scala library (while developing that library)”Stack Overflow · Nov 27, 2018 · 3 votes
- d12frosted/homebrew-emacs-plus“Fix linking how-to in formula caveats”GitHub · Dec 21, 2023 · 2 reactions
The biggest 5 of 54 mentions linking to the repository that our crawl found. Each line is their own headline.
See all 54
Talked about on GitHub, Stack Overflow, Hacker News and Bluesky (54 times).
On GitHub · 50
- Jan 17, 2024In ipython 8.18 (but not ipython 8.16), the current event loop is overwritten in each cell — ipython/ipython7 reactions · 2 comments
- Dec 6, 2021Request: re-license under MIT? — has2k1/plotnine3 reactions · 1 comment
- Dec 21, 2023Fix linking how-to in formula caveats — d12frosted/homebrew-emacs-plus2 reactions
- Dec 14, 2023Incorrect lint in 0.1.8? RUF006 Store a reference to the return value of `asyncio.create_task` — astral-sh/ruff2 reactions · 9 comments
- Nov 4, 2026New hail-is/hail version available: 0.2.140 — TheBoutrosLab/docker-Hail1 reaction
45 more on GitHub
- Aug 20, 2026New hail-is/hail version available: 0.2.139 — TheBoutrosLab/docker-Hail1 reaction
- Feb 7, 2021Hail developer set-up — populationgenomics/team-docs1 reaction
- Aug 4, 2026Bump the pip group across 1 directory with 5 updates — broadinstitute/variant-curation-portal0 reactions
- Jun 24, 2026Update hadoop to hailtop after hail's hadoop deprecation — broadinstitute/gnomad_methods0 reactions · 2 comments
- Jun 16, 2026[skill-suggestion] hail: scalable genomic analysis and GWAS engine (Broad Institute) — stanfish06/skillquarium0 reactions
- Sep 11, 2025[explore] Variant store: Hail — tidywf/nemo0 reactions
- Jul 25, 2025Add LEN to compute_stats_per_ref_site after hail updates — broadinstitute/gnomad_methods0 reactions
- Mar 14, 2025Temporarily revert username validation, as new rule disallows dashes — populationgenomics/hail0 reactions
- Jan 10, 2025Refine combiner usage — populationgenomics/production-pipelines0 reactions · 1 comment
- Dec 16, 2024fix: remove obsolete Hortonworks repositories — h2oai/h2o-30 reactions · 6 comments
- Dec 15, 2024Cherry-pick upstream fix: Set StreamReadConstraints maxStringLength as early as possible — populationgenomics/hail0 reactions
- Sep 20, 2024Bump the pip group with 4 updates — broadinstitute/variant-curation-portal0 reactions · 2 comments
- Jul 15, 2024chore(deps): bump the pip group across 2 directories with 15 updates — broadinstitute/gnomad-browser0 reactions · 4 comments
- Jul 5, 2024chore(deps): bump the pip group across 2 directories with 14 updates — broadinstitute/gnomad-browser0 reactions · 1 comment
- Jun 17, 2024chore(deps): bump the pip group across 3 directories with 14 updates — broadinstitute/gnomad-browser0 reactions · 1 comment
- Jun 11, 2024chore(deps): bump the pip group across 3 directories with 15 updates — broadinstitute/gnomad-browser0 reactions · 2 comments
- Jun 11, 2024chore(deps): bump the pip group across 3 directories with 14 updates — broadinstitute/gnomad-browser0 reactions · 1 comment
- Jun 10, 2024chore(deps): bump the pip group across 3 directories with 13 updates — broadinstitute/gnomad-browser0 reactions · 1 comment
- Jun 6, 2024chore(deps): bump the pip group in /data-pipeline with 12 updates — broadinstitute/gnomad-browser0 reactions · 1 comment
- May 14, 2024Fix syntax — broadinstitute/seqr-loading-pipelines0 reactions
- May 1, 2024Local_to_global: local alleles not strictly increasing: [0,2,1] — iris-garden/test-process0 reactions
- May 1, 2024Where to find data for practicing using Hail — iris-garden/test-process0 reactions
- Apr 30, 2024De novo confidence queries — iris-garden/test-process0 reactions
- Apr 30, 2024DNANexus: hail installation on custom spark cluster — iris-garden/test-process0 reactions
- Apr 30, 2024Hail architecture or design philosophy — iris-garden/test-process0 reactions
- Apr 30, 2024Assertion Error during entries().show() — iris-garden/test-process0 reactions
- Apr 30, 2024Dataproc connect should support --project command line option — iris-garden/test-process0 reactions
- Apr 30, 2024CollectionExpression join — iris-garden/test-process0 reactions
- Apr 30, 2024Use agg.linreg for GWAS — iris-garden/test-process0 reactions
- Apr 30, 2024Limit memory usage — iris-garden/test-process0 reactions
- Apr 30, 2024ArrayIndexOutOfBoundsException error in hail 0.2.40 — iris-garden/test-process0 reactions
- Apr 30, 2024Import_bgen 16 bits error — iris-garden/test-process0 reactions
- Apr 30, 2024VCF4.1, is.hail.utils.HailException: array index out of bounds — iris-garden/test-process0 reactions
- Apr 30, 2024How to get the ration transition-transversion on all dataset? — iris-garden/test-process0 reactions
- Apr 30, 2024GQ set to NA when PL is NA after split_multi_hts — iris-garden/test-process0 reactions
- Apr 30, 2024Download for tutorial notebooks — iris-garden/test-process0 reactions
- Apr 30, 2024Typo in impute_sex() description — iris-garden/test-process0 reactions
- Apr 30, 2024Hail V01 to V02 missing genotypes? — iris-garden/test-process0 reactions
- Apr 30, 2024Import_table on gs with wildcard — iris-garden/test-process0 reactions
- Apr 30, 2024Hail 0.2.10 patch notes — iris-garden/test-process0 reactions
- Apr 30, 2024Interval.parse in v0.2 — iris-garden/test-process0 reactions
- Apr 30, 2024Hail 0.2.6 patch notes — iris-garden/test-process0 reactions
- Apr 30, 2024Hail 0.2.5 patch notes — iris-garden/test-process0 reactions
- Apr 30, 2024Chained logistic regression — iris-garden/test-process0 reactions
- Apr 30, 2024[Breaking change] partition keys removed from python interface — iris-garden/test-process0 reactions
On Stack Overflow · 2
On Hacker News · 1
- Aug 28, 2020Hail.is: open-source library for exploring genomic data1 point
On Bluesky · 1
- Jan 12, 2024The Scalable Variant Call Representation: Enabling Genetic Analysis Beyond One Million Genomes www.biorxiv.org/content/10.1... 🧬🖥️… — @stephenturner.us7 likes and reposts
In this repository5 other files it gives its agents.
- hail/AGENTS.mdAGENTS.md
- hail/CLAUDE.mdCLAUDE.md
- flamegraphSkill
- hail-batch-devSkill
- release-changelogSkill
The file
We split this skill into the rules it gives an agent
7 rules. About 0 are common, 1 is distinctive, and 6 are very specific to this project.
≈0
Common · estimate
Many files like this have a rule of this kind.
≈1
Distinctive · estimate
This author's own choices. Worth a look.
6
Very specific
Tied to their project's names, files or tools. Probably skip.
- Common
- Distinctive
- Very specific
- Very specific: line 9
If hailctl isn't found, it may be in a virtual environment — look for one and activate it.
- Very specific: line 11
Never suppress stderr: when running hailctl or other Hail tools — important connection info and warnings appear there.
- Very specific: line 51
Always specify --container — the bare command dumps all containers as escaped YAML and is hard to read.
- Very specific: line 110
Use the hailgenetics/hail:<version> image.
- Distinctive: line 178
When given a batch ID:
- Very specific: line 179
Run hailctl batch get BATCH_ID — report state, n_jobs, n_succeeded, n_failed, cost
- Very specific: line 180
Run hailctl batch jobs BATCH_ID --state bad to find failed/errored jobs
≈ “Common” is an estimate: at least 1 in 5 of the 372,429 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.
Files
You are helping a user work with Hail Batch — a job orchestration service for running analysis pipelines on cloud compute.
Running hailctl commands
If hailctl isn't found, it may be in a virtual environment — look for one and activate it. Once activated, it stays active for the session.
Never suppress stderr when running hailctl or other Hail tools — important connection info and warnings appear there. Do not use 2>&1 or 2>/dev/null.
Authentication
Users need two things on GCP:
# 1. GCP Application Default Credentials (for cloud storage access)
gcloud auth application-default login
# 2. Hail service auth (reads ~/.hail/tokens.json)
hailctl auth loginBoth are needed. If a user gets auth errors, check which one is missing.
Investigating batches: use the CLI
For inspection tasks (checking status, reading logs, diagnosing failures), always prefer hailctl batch CLI commands — they're simpler and don't require writing Python.
List recent batches
hailctl batch list # last 50 batches
hailctl batch list --limit 20 --query 'state=failure'
hailctl batch list -o json # machine-readableGet batch status
hailctl batch get BATCH_ID # full details (yaml)
hailctl batch get BATCH_ID -o jsonInspect a specific job
hailctl batch job BATCH_ID JOB_ID # status + spec (yaml)Get logs
Always specify --container — the bare command dumps all containers as escaped YAML and is hard to read. If a job has been retried, use --attempt to get the right one (defaults to most recent).
hailctl batch attempts BATCH_ID JOB_ID # list attempts + IDs (only if needed)
hailctl batch log BATCH_ID JOB_ID --container main # user code (start here)
hailctl batch log BATCH_ID JOB_ID --container main --attempt ID # specific attempt
hailctl batch log BATCH_ID JOB_ID --container input # file copies in
hailctl batch log BATCH_ID JOB_ID --container output # file copies outWait for a batch to complete
hailctl batch wait BATCH_ID
hailctl batch wait BATCH_ID -o jsonCancel or delete
hailctl batch cancel BATCH_ID
hailctl batch delete BATCH_IDBilling
hailctl batch billing list
hailctl batch billing get BILLING_PROJECTWriting batch submission scripts: use the Python API
For writing scripts that submit batches programmatically, use hailtop.batch:
import hailtop.batch as hb
b = hb.Batch(
name='my-analysis',
backend=hb.ServiceBackend(
billing_project='my-project',
remote_tmpdir='gs://my-bucket/tmp',
),
)
j = b.new_job(name='step1')
j.image('hailgenetics/hail:0.2.x')
j.memory('4Gi')
j.cpu(2)
j.command('python3 my_script.py')
b.run()For quick one-off jobs, hailctl batch submit works too:
hailctl batch submit --image hailgenetics/hail:0.2.x --memory 4Gi python3 my_script.pyHail Query within a Batch job
Use the hailgenetics/hail:<version> image. Write outputs to cloud storage and read them back:
# In the batch job script (runs on the worker):
import hail as hl
hl.init()
mt = hl.read_matrix_table('gs://bucket/input.mt')
results = mt.some_analysis()
results.write('gs://bucket/output.ht')
# Back in the parent script after b.run():
results = hl.read_table('gs://bucket/output.ht')Direct API access with hailctl curl
For endpoints not covered by hailctl batch subcommands, use hailctl curl — it handles auth automatically:
hailctl curl default batch /PATHTo discover all available endpoints, fetch the OpenAPI spec:
hailctl curl default batch /openapi.yamlListing jobs within a batch
hailctl batch jobs BATCH_ID # first 50 jobs
hailctl batch jobs BATCH_ID --state bad # failed or errored jobs
hailctl batch jobs BATCH_ID --state bad -o json # machine-readable
hailctl batch jobs BATCH_ID --exit-code 137 # OOM kills
hailctl batch jobs BATCH_ID --name my-step # by job name
hailctl batch jobs BATCH_ID --limit 0 # all jobs (no limit)
hailctl batch jobs BATCH_ID --limit 50 --last-job-id 50 # pagination: next page after job 50Valid --state values: pending, ready, creating, running, live (ready+creating+running), cancelled, error, failed, bad (error+failed), success, done (all terminal states).
Flags can be combined: --state bad --exit-code 1 returns jobs that are bad AND have exit code 1.
Job states
| State | Meaning |
Pending | Waiting on dependencies |
Ready | Dependencies met, queued for a worker |
Creating | Worker being provisioned |
Running | Executing on a worker |
Success | Exited 0 |
Failed | User code exited non-zero |
Error | Infrastructure/platform failure (not user code) |
Cancelled | Cancelled before completion |
Common failure patterns
| Symptom | Likely cause | Action |
Failed, exit code 1 | Script error | hailctl batch log BATCH JOB --container main |
Failed, exit code 137 | OOM kill | Increase j.memory() in the submission script |
Error state | Infrastructure failure | Check --container input or --container output logs; may be transient |
Jobs stuck in Pending | Dependency failure or quota | Check upstream jobs |
| High cost, slow progress | Insufficient parallelism | Increase job fan-out in the script |
How to investigate a specific batch
When given a batch ID:
- Run
hailctl batch get BATCH_ID— report state, n_jobs, n_succeeded, n_failed, cost - Run
hailctl batch jobs BATCH_ID --state badto find failed/errored jobs - For each bad job (up to ~5), run
hailctl batch log BATCH_ID JOB_ID --container main - Identify the root cause and suggest next steps
Does it work?
Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.
No reviews yet
People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.
- It worked
- Partly
- It didn't
Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.
Your trail
Files you open will collect here, so you can find them again.
Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.
Now do this for your own.
Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.

