deeplabcut
Skill39 starsChanged 9 months agoMITView on GitHub
Toolbox for markerless animal pose estimation with DeepLabCut. Covers single/multi-animal tracking, SuperAnimal pretrained models, 2D/3D pose estimation, keypoint labeling GUI, model training/evaluation, video analysis, and behavioral quantification. Use when the user needs animal pose estimation, behavior tracking, keypoint detection in videos, or mentions DeepLabCut/DLC/SuperAnimal.
- Installs packages
In this repository49 other files it gives its agents.
- act-r-model-builderSkill
- alternative-uses-task-designerSkill
- bayesian-cognitive-model-builderSkill
- brain-connectivity-modelerSkill
- calcium-imaging-analysis-guideSkill
- cognitive-paradigm-designSkill
- cogsci-power-analysisSkill
- cogsci-statisticsSkill
- cogsci-visualizationSkill
- contribute-skillSkill
- contribute-skills-via-prSkill
- creativity-self-efficacy-mediationSkill
- divergent-thinking-scoringSkill
- drift-diffusion-modelSkill
- eeg-paradigm-designerSkill
- eeg-preprocessing-pipeline-guideSkill
- erp-analysisSkill
- evidence-accumulation-selectorSkill
- fmri-glm-analysis-guideSkill
- fmri-preprocessing-pipeline-guideSkill
- fmriprepSkill
- fmri-task-design-guideSkill
- infant-looking-time-designerSkill
- lesion-symptom-mapping-guideSkill
- markdown-report-writingSkill
- mne-python-guideSkill
- netneurotools-guideSkill
- neural-decoding-analysisSkill
- neural-population-analysis-guideSkill
- neuroimaging-power-guideSkill
- neuroimaging-sample-size-calculatorSkill
- neuropsych-battery-selectorSkill
- optogenetics-protocol-designerSkill
- paper-to-skillSkill
- parameter-recovery-checkerSkill
- pycortex-guideSkill
- reading-time-analysisSkill
- repo-to-skillSkill
- research-literacySkill
- self-paced-reading-designerSkill
- sentence-stimulus-normingSkill
- share-caseSkill
- share-usageSkill
- signal-detection-analysisSkill
- spikeinterface-skillSkill
- spiking-network-model-builderSkill
- tom-task-selectorSkill
- verify-skillSkill
- visual-search-array-generatorSkill
The file
We split this skill into the rules it gives an agent
2 rules. About 0 are common, 1 is distinctive, and 1 is very specific to this project.
≈0
Common · estimate
Many files like this have a rule of this kind.
≈1
Distinctive · estimate
This author's own choices. Worth a look.
1
Very specific
Tied to their project's names, files or tools. Probably skip.
- Common
- Distinctive
- Very specific
- Distinctive: line 149
Wrong PyTorch version: Always install PyTorch BEFORE deeplabcut.
- Very specific: line 152
Video codec issues: If create_labeled_video fails, try converting to .mp4 (H.264) or .avi first.
≈ “Common” is an estimate: at least 1 in 5 of the 364,980 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.
Files
DeepLabCut — Markerless Animal Pose Estimation
Purpose
DeepLabCut is a Python toolbox for state-of-the-art markerless pose estimation of animals. It uses deep learning to track body parts from videos without physical markers. The library is animal-agnostic, supports both single and multi-animal scenarios, and includes the SuperAnimal family of pretrained models for out-of-the-box inference.
When to Use This Skill
Activate when the user:
- Wants to track animal body parts from video
- Asks about pose estimation, keypoint detection, or behavioral tracking
- Mentions DeepLabCut, DLC, SuperAnimal, or markerless tracking
- Needs to analyze animal movement/kinematics
- Asks about multi-animal tracking or 3D pose reconstruction
- Wants to use pretrained animal pose models
Quick Decision Tree
What does the user need?
├── No labeled data, just want to track animals → SuperAnimal (video_inference_superanimal)
├── Single animal, have labeled data → Standard single-animal pipeline
├── Multiple animals interacting → maDLC (multi-animal pipeline)
├── 3D pose reconstruction → 3D pipeline (calibrate_cameras + triangulate)
└── Just post-process results → filterpredictions, analyzeskeletonReference Files (Progressive Disclosure)
| Topic | File | When to Read |
| Standard Pipeline | references/standard-pipeline.md | Full workflow: create project → train → analyze |
| SuperAnimal & ModelZoo | references/modelzoo.md | Pretrained models, zero-shot inference |
| Multi-Animal (maDLC) | references/maDLC.md | Tracking multiple interacting animals |
| 3D Pose Estimation | references/3d-pose.md | Triangulation from multiple camera views |
| Video & Data Utilities | references/utilities.md | Video cropping, format conversion, data export |
Installation
# Minimal (headless, no GUI):
pip install deeplabcut
# With GUI (label_frames, refine_labels, SkeletonBuilder):
pip install "deeplabcut[gui]"
# PyTorch must be installed separately:
pip install torch torchvision
# Or for GPU (check pytorch.org for your CUDA version):
conda install pytorch cudatoolkit=11.3 -c pytorchVerify: python -c "import deeplabcut; print(deeplabcut.__version__)"
Standard Pipeline Overview
import deeplabcut as dlc
# 1. Create project
config_path = dlc.create_new_project(
"ProjectName", "ExperimenterName", ["/path/to/video.mp4"],
working_directory="/path/to/projects"
)
# 2. Extract frames for labeling
dlc.extract_frames(config_path, mode="automatic", algo="kmeans", crop=True)
# 3. USER labels frames manually in GUI
# dlc.label_frames(config_path) # launches the labeling GUI
# 4. Create training dataset from labeled frames
dlc.create_training_dataset(config_path, net_type="resnet_50")
# 5. Train the network
dlc.train_network(config_path, maxiters=100000, saveiters=5000)
# 6. Evaluate
dlc.evaluate_network(config_path, plotting=True)
# 7. Analyze videos (predict poses)
dlc.analyze_videos(config_path, ["/path/to/video.mp4"], videotype=".mp4")
# 8. Create labeled videos (overlay predictions)
dlc.create_labeled_video(config_path, ["/path/to/video.mp4"])
# 9. Export results to CSV
dlc.analyze_videos_converth5_to_csv("/path/to/videoDLC_resnet50_ProjectNameJul9")Key API Reference
Project Management
| Function | Description |
create_new_project(project, experimenter, videos, working_directory) | Start a new single-animal project |
create_new_project_3d(project, experimenter, num_cameras, working_directory) | Start a new 3D project |
create_pretrained_project(path, task, videos, SUPERANIMAL_NAME, model_name, detector_name) | Create project from SuperAnimal pretrained model |
add_new_videos(config_path, videos) | Add videos to existing project |
Training & Evaluation
| Function | Description |
create_training_dataset(config_path, net_type, augmenter_type) | Prepare training data; net_type: resnet_50, resnet_101, mobilenet_v2_1.0, efficientnet-b0 |
train_network(config_path, maxiters, saveiters) | Train; key params: maxiters=100000, saveiters=5000 |
evaluate_network(config_path, plotting=True) | Evaluate on test set, produce metrics |
Video Analysis
| Function | Description |
analyze_videos(config_path, videos, videotype, save_as_csv) | Predict poses for all frames in videos |
create_labeled_video(config_path, videos, videotype, filtered) | Overlay predicted keypoints on video |
video_inference_superanimal(videos, superanimal_name, ...) | Zero-shot inference with pretrained SuperAnimal models |
Post-Processing
| Function | Description |
filterpredictions(config_path, video, ...) | Smooth predictions (ARIMA, median filtering) |
analyzeskeleton(config_path, video, ...) | Compute bone lengths, joint angles from predictions |
plot_trajectories(config_path, video, ...) | Plot body part trajectories over time |
SuperAnimal Pretrained Models
Available models for zero-shot inference:
| Model | Species | Body Parts |
superanimal_topviewmouse | Top-view mouse (various strains) | 21 keypoints |
superanimal_quadruped | Quadrupeds (dog, horse, sheep, etc.) | 39 keypoints |
superanimal_face | Primate/human faces | 54 keypoints |
superanimal_full | Full body animals (topview mouse + quadruped) | Combined |
Common Pitfalls
- Wrong PyTorch version: Always install PyTorch BEFORE deeplabcut. Check pytorch.org for the correct CUDA version.
- GPU out of memory: Reduce batch size (default 8 → 4 or 2) in
pose_cfg.yaml. - Missing GUI dependencies:
label_framesrequirespip install "deeplabcut[gui]". On headless servers, use X11 forwarding or label locally. - Video codec issues: If
create_labeled_videofails, try converting to.mp4(H.264) or.avifirst. - maDLC detector: Multi-animal mode requires a detection model (
fasterrcnn_resnet50_fpn_v2is the default). Single-animal can work with heatmap regression alone. - SuperAnimal scale: Adjust
scale_listparameter for SuperAnimal — try[200, 300, 400]first; smaller animals may need higher values.
Does it work?
Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.
No reviews yet
People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.
- It worked
- Partly
- It didn't
Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.
Your trail
Files you open will collect here, so you can find them again.
Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.
Now do this for your own.
Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.

