segment-anything-model
Skill1 starsChanged 58 days agoMITView on GitHub
Foundation model for image segmentation with zero-shot transfer. Use when you need to segment any object in images using points, boxes, or masks as prompts, or automatically generate all object masks in an image.
- Installs packages
8 other projects use a version of this, 5 of them edited.
Adopted by
Most recently taken up by
- Ivan-propio/claude-skillsword for word, first seen Nov 16, 2026
- OpenLAIR/dr-clawword for word, first seen Nov 8, 2026
- ashleysbrain/skillsedited, first seen Nov 8, 2026
- delorenj/skillsedited, first seen Nov 7, 2026
- pedroiff0/awesome-skillsedited, first seen Sep 29, 2026
- Samin12/skillsedited, first seen Aug 14, 2026
Used with
Projects with it also use popular-web-designs, heartmula and 10 more.
Other files people use alongside this one.
Files found in the same public projects as a version of this one, more often than they turn up anywhere else. Each project counts once, large collections count for less, and this file's own repository is left out.
Tommy-ywSkillpopular-web-designs
54 production-quality design systems extracted from real websites. Load a template to generate HTML/CSS that matches the visual identity of sites like Stripe, Linear, Vercel, Notion, Airbnb, and more. Each template includes colors, typography, components, layout rules, and ready-to-use CSS values.
Also in 5 projects that use this
math-incSkillheartmula
Set up and run HeartMuLa, the open-source music generation model family (Suno-like). Generates full songs from lyrics + tags with multilingual support.
Also in 3 projects that use this
mae616Skillp5js
p5.js(クリエイティブコーディング)のProcessing由来「コードでスケッチ」思想を軸に、setup/drawループ・座標系・インタラクション・パフォーマンス最適化を整理する。p5.js、Processing、クリエイティブコーディング、ジェネラティブアート、インタラクティブ表現、ビジュアルプログラミング、Canvas描画、パーティクル、フラクタル、ノイズ表現に関する相談で必ず使うこと。ユーザーが「コードでアート」「動くビジュアル」「インタラクティブな描画」と言った場合にも適用する。
Also in 4 projects that use this
AsharibAliSkillpretext
DOM-free multiline text measurement and layout using @chenglou/pretext. Use when the user needs text height calculation, virtualized list row sizing, canvas/SVG text rendering, rich-text inline flow, or any scenario involving text layout without DOM reflow.
Also in 3 projects that use this
yusei531642Skillclaude-design
Vibe-Editor の UI を「Claude.ai + Claude Code 公式風」にリファインする際に参照するデザインガイド。色 (warm off-white / terra cotta #d97757)、タイポ (Claude 返答は serif・ユーザーは sans)、余白、記号 (⏺・⎿・スピナー)、アニメ (150ms / cubic-bezier(.4,0,.2,1))、メッセージングトーンの具体値を提供する。UI 刷新・新規コンポーネント作成・テーマ追加・エージェント発話の装飾など、見た目やトーンの判断が必要な作業で必ず参照。
Also in 4 projects that use this
math-incSkillfindmy
Track Apple devices and AirTags via FindMy.app on macOS using AppleScript and screen capture.
Also in 3 projects that use this
- +1 more from math-inc
Other versions8 projects use a version of this.
Word for word · 3
- OpenLAIR/dr-claw★ 1.2k100% same
- Clay-HHK/claude-skills★ 1100% same
- Ivan-propio/claude-skills★ 0100% same
Edited · 5
- delorenj/skills★ 1797% same
- Samin12/skills★ 897% same
- ashleysbrain/skills★ 197% same
- pedroiff0/awesome-skills★ 197% same
- wm2313/my-skills★ 097% same
Wrote their own version · 0
None found.
How matches are found. Word for word: the exact same text. Edited: shares passages with this file (“99% same” means 99% of its five-word phrases also appear here; 10% or more counts). Their own version: similar meaning, or, weakest, the same name. We can’t tell which project had it first. Projects are counted among the 200 most-starred candidates, and mentions are links to the repository we found, up to 50 per site, so both counts are a floor.
In this repository97 other files it gives its agents.
- autoresearchSkill
- academic-plottingSkill
- huggingface-accelerateSkill
- evolving-ai-agentsSkill
- audiocraft-audio-generationSkill
- autogpt-agentsSkill
- awq-quantizationSkill
- axolotlSkill
- evaluating-code-modelsSkill
- quantizing-models-bitsandbytesSkill
- blip-2-vision-languageSkill
- brainstorming-research-ideasSkill
- chromaSkill
- clipSkill
- ara-compilerSkill
- constitutional-aiSkill
- evaluating-cosmos-policySkill
- creative-thinking-for-researchSkill
- crewai-multi-agentSkill
- deepspeedSkill
- dspySkill
- faissSkill
- optimizing-attention-flashSkill
- gguf-quantizationSkill
- gptqSkill
- grpo-rl-trainingSkill
- guidanceSkill
- hqq-quantizationSkill
- huggingface-tokenizersSkill
- instructorSkill
- knowledge-distillationSkill
- lambda-labs-gpu-cloudSkill
- langchainSkill
- langsmith-observabilitySkill
- implementing-llms-litgptSkill
- llama-cppSkill
- llama-factorySkill
- llamaguardSkill
- llamaindexSkill
- llavaSkill
- evaluating-llms-harnessSkill
- long-contextSkill
- mamba-architectureSkill
- training-llms-megatronSkill
- miles-rl-trainingSkill
- mlflowSkill
- ml-paper-writingSkill
- ml-training-recipesSkill
- modal-serverless-gpuSkill
- model-mergingSkill
- model-pruningSkill
- moe-trainingSkill
- nanogptSkill
- nemo-curatorSkill
- nemo-evaluator-sdkSkill
- nemo-guardrailsSkill
- nnsight-remote-interpretabilitySkill
- fine-tuning-serving-openpiSkill
- openrlhf-trainingSkill
- fine-tuning-openvla-oftSkill
- The first 60 shown.
The file
Files
Segment Anything Model (SAM)
Comprehensive guide to using Meta AI's Segment Anything Model for zero-shot image segmentation.
When to use SAM
Use SAM when:
- Need to segment any object in images without task-specific training
- Building interactive annotation tools with point/box prompts
- Generating training data for other vision models
- Need zero-shot transfer to new image domains
- Building object detection/segmentation pipelines
- Processing medical, satellite, or domain-specific images
Key features:
- Zero-shot segmentation: Works on any image domain without fine-tuning
- Flexible prompts: Points, bounding boxes, or previous masks
- Automatic segmentation: Generate all object masks automatically
- High quality: Trained on 1.1 billion masks from 11 million images
- Multiple model sizes: ViT-B (fastest), ViT-L, ViT-H (most accurate)
- ONNX export: Deploy in browsers and edge devices
Use alternatives instead:
- YOLO/Detectron2: For real-time object detection with classes
- Mask2Former: For semantic/panoptic segmentation with categories
- GroundingDINO + SAM: For text-prompted segmentation
- SAM 2: For video segmentation tasks
Quick start
Installation
# From GitHub
pip install git+https://github.com/facebookresearch/segment-anything.git
# Optional dependencies
pip install opencv-python pycocotools matplotlib
# Or use HuggingFace transformers
pip install transformersDownload checkpoints
# ViT-H (largest, most accurate) - 2.4GB
wget https://dl.fbaipublicfiles.com/segment_anything/sam_vit_h_4b8939.pth
# ViT-L (medium) - 1.2GB
wget https://dl.fbaipublicfiles.com/segment_anything/sam_vit_l_0b3195.pth
# ViT-B (smallest, fastest) - 375MB
wget https://dl.fbaipublicfiles.com/segment_anything/sam_vit_b_01ec64.pthBasic usage with SamPredictor
import numpy as np
from segment_anything import sam_model_registry, SamPredictor
# Load model
sam = sam_model_registry["vit_h"](checkpoint="sam_vit_h_4b8939.pth")
sam.to(device="cuda")
# Create predictor
predictor = SamPredictor(sam)
# Set image (computes embeddings once)
image = cv2.imread("image.jpg")
image = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)
predictor.set_image(image)
# Predict with point prompts
input_point = np.array([[500, 375]]) # (x, y) coordinates
input_label = np.array([1]) # 1 = foreground, 0 = background
masks, scores, logits = predictor.predict(
point_coords=input_point,
point_labels=input_label,
multimask_output=True # Returns 3 mask options
)
# Select best mask
best_mask = masks[np.argmax(scores)]HuggingFace Transformers
import torch
from PIL import Image
from transformers import SamModel, SamProcessor
# Load model and processor
model = SamModel.from_pretrained("facebook/sam-vit-huge")
processor = SamProcessor.from_pretrained("facebook/sam-vit-huge")
model.to("cuda")
# Process image with point prompt
image = Image.open("image.jpg")
input_points = [[[450, 600]]] # Batch of points
inputs = processor(image, input_points=input_points, return_tensors="pt")
inputs = {k: v.to("cuda") for k, v in inputs.items()}
# Generate masks
with torch.no_grad():
outputs = model(**inputs)
# Post-process masks to original size
masks = processor.image_processor.post_process_masks(
outputs.pred_masks.cpu(),
inputs["original_sizes"].cpu(),
inputs["reshaped_input_sizes"].cpu()
)Core concepts
Model architecture
SAM Architecture:
┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐
│ Image Encoder │────▶│ Prompt Encoder │────▶│ Mask Decoder │
│ (ViT) │ │ (Points/Boxes) │ │ (Transformer) │
└─────────────────┘ └─────────────────┘ └─────────────────┘
│ │ │
Image Embeddings Prompt Embeddings Masks + IoU
(computed once) (per prompt) predictionsModel variants
| Model | Checkpoint | Size | Speed | Accuracy |
| ViT-H | vit_h | 2.4 GB | Slowest | Best |
| ViT-L | vit_l | 1.2 GB | Medium | Good |
| ViT-B | vit_b | 375 MB | Fastest | Good |
Prompt types
| Prompt | Description | Use Case |
| Point (foreground) | Click on object | Single object selection |
| Point (background) | Click outside object | Exclude regions |
| Bounding box | Rectangle around object | Larger objects |
| Previous mask | Low-res mask input | Iterative refinement |
Interactive segmentation
Point prompts
# Single foreground point
input_point = np.array([[500, 375]])
input_label = np.array([1])
masks, scores, logits = predictor.predict(
point_coords=input_point,
point_labels=input_label,
multimask_output=True
)
# Multiple points (foreground + background)
input_points = np.array([[500, 375], [600, 400], [450, 300]])
input_labels = np.array([1, 1, 0]) # 2 foreground, 1 background
masks, scores, logits = predictor.predict(
point_coords=input_points,
point_labels=input_labels,
multimask_output=False # Single mask when prompts are clear
)Box prompts
# Bounding box [x1, y1, x2, y2]
input_box = np.array([425, 600, 700, 875])
masks, scores, logits = predictor.predict(
box=input_box,
multimask_output=False
)Combined prompts
# Box + points for precise control
masks, scores, logits = predictor.predict(
point_coords=np.array([[500, 375]]),
point_labels=np.array([1]),
box=np.array([400, 300, 700, 600]),
multimask_output=False
)Iterative refinement
# Initial prediction
masks, scores, logits = predictor.predict(
point_coords=np.array([[500, 375]]),
point_labels=np.array([1]),
multimask_output=True
)
# Refine with additional point using previous mask
masks, scores, logits = predictor.predict(
point_coords=np.array([[500, 375], [550, 400]]),
point_labels=np.array([1, 0]), # Add background point
mask_input=logits[np.argmax(scores)][None, :, :], # Use best mask
multimask_output=False
)Automatic mask generation
Basic automatic segmentation
from segment_anything import SamAutomaticMaskGenerator
# Create generator
mask_generator = SamAutomaticMaskGenerator(sam)
# Generate all masks
masks = mask_generator.generate(image)
# Each mask contains:
# - segmentation: binary mask
# - bbox: [x, y, w, h]
# - area: pixel count
# - predicted_iou: quality score
# - stability_score: robustness score
# - point_coords: generating pointCustomized generation
mask_generator = SamAutomaticMaskGenerator(
model=sam,
points_per_side=32, # Grid density (more = more masks)
pred_iou_thresh=0.88, # Quality threshold
stability_score_thresh=0.95, # Stability threshold
crop_n_layers=1, # Multi-scale crops
crop_n_points_downscale_factor=2,
min_mask_region_area=100, # Remove tiny masks
)
masks = mask_generator.generate(image)Filtering masks
# Sort by area (largest first)
masks = sorted(masks, key=lambda x: x['area'], reverse=True)
# Filter by predicted IoU
high_quality = [m for m in masks if m['predicted_iou'] > 0.9]
# Filter by stability score
stable_masks = [m for m in masks if m['stability_score'] > 0.95]Batched inference
Multiple images
# Process multiple images efficiently
images = [cv2.imread(f"image_{i}.jpg") for i in range(10)]
all_masks = []
for image in images:
predictor.set_image(image)
masks, _, _ = predictor.predict(
point_coords=np.array([[500, 375]]),
point_labels=np.array([1]),
multimask_output=True
)
all_masks.append(masks)Multiple prompts per image
# Process multiple prompts efficiently (one image encoding)
predictor.set_image(image)
# Batch of point prompts
points = [
np.array([[100, 100]]),
np.array([[200, 200]]),
np.array([[300, 300]])
]
all_masks = []
for point in points:
masks, scores, _ = predictor.predict(
point_coords=point,
point_labels=np.array([1]),
multimask_output=True
)
all_masks.append(masks[np.argmax(scores)])ONNX deployment
Export model
python scripts/export_onnx_model.py \
--checkpoint sam_vit_h_4b8939.pth \
--model-type vit_h \
--output sam_onnx.onnx \
--return-single-maskUse ONNX model
import onnxruntime
# Load ONNX model
ort_session = onnxruntime.InferenceSession("sam_onnx.onnx")
# Run inference (image embeddings computed separately)
masks = ort_session.run(
None,
{
"image_embeddings": image_embeddings,
"point_coords": point_coords,
"point_labels": point_labels,
"mask_input": np.zeros((1, 1, 256, 256), dtype=np.float32),
"has_mask_input": np.array([0], dtype=np.float32),
"orig_im_size": np.array([h, w], dtype=np.float32)
}
)Common workflows
Workflow 1: Annotation tool
import cv2
# Load model
predictor = SamPredictor(sam)
predictor.set_image(image)
def on_click(event, x, y, flags, param):
if event == cv2.EVENT_LBUTTONDOWN:
# Foreground point
masks, scores, _ = predictor.predict(
point_coords=np.array([[x, y]]),
point_labels=np.array([1]),
multimask_output=True
)
# Display best mask
display_mask(masks[np.argmax(scores)])Workflow 2: Object extraction
def extract_object(image, point):
"""Extract object at point with transparent background."""
predictor.set_image(image)
masks, scores, _ = predictor.predict(
point_coords=np.array([point]),
point_labels=np.array([1]),
multimask_output=True
)
best_mask = masks[np.argmax(scores)]
# Create RGBA output
rgba = np.zeros((image.shape[0], image.shape[1], 4), dtype=np.uint8)
rgba[:, :, :3] = image
rgba[:, :, 3] = best_mask * 255
return rgbaWorkflow 3: Medical image segmentation
# Process medical images (grayscale to RGB)
medical_image = cv2.imread("scan.png", cv2.IMREAD_GRAYSCALE)
rgb_image = cv2.cvtColor(medical_image, cv2.COLOR_GRAY2RGB)
predictor.set_image(rgb_image)
# Segment region of interest
masks, scores, _ = predictor.predict(
box=np.array([x1, y1, x2, y2]), # ROI bounding box
multimask_output=True
)Output format
Mask data structure
# SamAutomaticMaskGenerator output
{
"segmentation": np.ndarray, # H×W binary mask
"bbox": [x, y, w, h], # Bounding box
"area": int, # Pixel count
"predicted_iou": float, # 0-1 quality score
"stability_score": float, # 0-1 robustness score
"crop_box": [x, y, w, h], # Generation crop region
"point_coords": [[x, y]], # Input point
}COCO RLE format
from pycocotools import mask as mask_utils
# Encode mask to RLE
rle = mask_utils.encode(np.asfortranarray(mask.astype(np.uint8)))
rle["counts"] = rle["counts"].decode("utf-8")
# Decode RLE to mask
decoded_mask = mask_utils.decode(rle)Performance optimization
GPU memory
# Use smaller model for limited VRAM
sam = sam_model_registry["vit_b"](checkpoint="sam_vit_b_01ec64.pth")
# Process images in batches
# Clear CUDA cache between large batches
torch.cuda.empty_cache()Speed optimization
# Use half precision
sam = sam.half()
# Reduce points for automatic generation
mask_generator = SamAutomaticMaskGenerator(
model=sam,
points_per_side=16, # Default is 32
)
# Use ONNX for deployment
# Export with --return-single-mask for faster inferenceCommon issues
| Issue | Solution |
| Out of memory | Use ViT-B model, reduce image size |
| Slow inference | Use ViT-B, reduce points_per_side |
| Poor mask quality | Try different prompts, use box + points |
| Edge artifacts | Use stability_score filtering |
| Small objects missed | Increase points_per_side |
References
- Advanced Usage - Batching, fine-tuning, integration
- Troubleshooting - Common issues and solutions
Resources
Does it work?
Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.
No reviews yet
People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.
- It worked
- Partly
- It didn't
Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.
Your trail
Files you open will collect here, so you can find them again.
Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.
Now do this for your own.
Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.

