ai-multimodal
Skill0 starsChanged 24 days agoMITView on GitHub
Process and generate multimedia content using Google Gemini API. Capabilities include analyze audio files (transcription with timestamps, summarization, speech understanding, music/sound analysis up to 9.5 hours), understand images (captioning, object detection, OCR, visual Q&A, segmentation), process videos (scene detection, Q&A, temporal analysis, YouTube URLs, up to 6 hours), extract from documents (PDF tables, forms, charts, diagrams, multi-page), generate images (text-to-image, editing, composition, refinement). Use when working with audio/video files, analyzing images or screenshots, processing PDF documents, extracting structured data from media, creating images from text prompts, or implementing multimodal AI features. Supports multiple models (Gemini 2.5/2.0) with context windows up to 2M tokens.
- Reads credentials
- Installs packages
14 other projects use a version of this, 6 of them edited.
Adopted by
Most recently taken up by
- david33137/claude-skillstheir own version, first seen Nov 16, 2026
- workflownicojunk/claude-skillsedited, first seen Oct 20, 2026
- davidanthonyn/claude-skillsedited, first seen Oct 20, 2026
- Dinh2026/claudekit-skillsword for word, first seen Oct 15, 2026
- captain-corgi/marketplace-websiteedited, first seen Oct 11, 2026
- typv/nest-turbo-startertheir own version, first seen Oct 8, 2026
Used with
Projects with it also use Problem-Solving Techniques, markdown-novel-viewer and 10 more.
Other files people use alongside this one.
Files found in the same public projects as a version of this one, more often than they turn up anywhere else. Each project counts once, large collections count for less, and this file's own repository is left out.
truongtv22SkillProblem-Solving Techniques
Apply systematic problem-solving techniques for complexity spirals (simplification cascades), innovation blocks (collision-zone thinking), recurring patterns (meta-pattern recognition), assumption constraints (inversion exercise), scale uncertainty (scale game), and dispatch when stuck. Techniques derived from Microsoft Amplifier project patterns adapted for immediate application.
Also in 6 projects that use this
OpenAnalystIncSkillmarkdown-novel-viewer
Also in 4 projects that use this
OpenAnalystIncSkillads-management
Activate for paid advertising campaigns on Google Ads, Meta Ads, LinkedIn Ads, TikTok Ads. Includes ad copywriting, audience targeting, budget optimization, A/B testing, and ROAS tracking. Used by ads-specialist and campaign-manager agents.
Also in 3 projects that use this
truongtv22Skillmobile-development
Build modern mobile applications with React Native, Flutter, Swift/SwiftUI, and Kotlin/Jetpack Compose. Covers mobile-first design principles, performance optimization (battery, memory, network), offline-first architecture, platform-specific guidelines (iOS HIG, Material Design), testing strategies, security best practices, accessibility, app store deployment, and mobile development mindset. Use when building mobile apps, implementing mobile UX patterns, optimizing for mobile constraints, or making native vs cross-platform decisions.
Also in 5 projects that use this
sumuligeSkillseo-optimization
SEO best practices for Git, documentation, and web projects
Also in 3 projects that use this
claude-office-skillsSkillemail-marketing
Email marketing automation - campaign creation, sequence building, A/B testing, deliverability optimization, and analytics
Also in 3 projects that use this
- +6 more from OpenAnalystInc
Other versions14 projects use a version of this.
Word for word · 5
- mrgoonie/claudekit-skills★ 2.2k100% same
- vneseyoungster/ChocoVine★ 27100% same
- VoDaiLocz/kilo-kit-mcp★ 27100% same
- Dinh2026/claudekit-skills★ 0100% same
- Linhv14/claude-skill★ 0100% same
Edited · 6
- duongthinh03/portfolio★ 8535% same
- OpenAnalystInc/10x-Team★ 811% same
- truongtv22/theone-training-skills★ 1100% same
- captain-corgi/marketplace-website★ 014% same
- davidanthonyn/claude-skills★ 097% same
- workflownicojunk/claude-skills★ 095% same
Wrote their own version · 3
- typv/nest-turbo-starter★ 224same name
- david33137/claude-skills★ 0same name
- docaohieu2808/claude-skills★ 0same name
How matches are found. Word for word: the exact same text. Edited: shares passages with this file (“99% same” means 99% of its five-word phrases also appear here; 10% or more counts). Their own version: similar meaning, or, weakest, the same name. We can’t tell which project had it first. Projects are counted among the 200 most-starred candidates, and mentions are links to the repository we found, up to 50 per site, so both counts are a floor.
In this repository104 other files it gives its agents.
- .claude/skills/claude-code/llms.txtllms.txt
- aestheticSkill
- angular-architectSkill
- api-designerSkill
- architecture-designerSkill
- atlassian-mcpSkill
- backend-developmentSkill
- banner-designSkill
- better-authSkill
- brandSkill
- bunnySkill
- chaos-engineerSkill
- chrome-devtoolsSkill
- claude-codeSkill
- cli-developerSkill
- cloud-architectSkill
- code-documenterSkill
- code-reviewerSkill
- code-reviewSkill
- context-engineeringSkill
- cpp-proSkill
- csharp-developerSkill
- database-optimizerSkill
- databasesSkill
- debuggingSkill
- debugging-wizardSkill
- designSkill
- design-systemSkill
- devops-engineerSkill
- devopsSkill
- django-expertSkill
- django-storages-s3Skill
- docs-seekerSkill
- dotnet-core-expertSkill
- embedded-systemsSkill
- fastapi-expertSkill
- feature-forgeSkill
- fine-tuning-expertSkill
- flutter-expertSkill
- frontend-designSkill
- frontend-dev-guidelinesSkill
- fullstack-guardianSkill
- game-developerSkill
- golang-proSkill
- google-adk-pythonSkill
- graphql-architectSkill
- java-architectSkill
- javascript-proSkill
- kotlin-specialistSkill
- kubernetes-specialistSkill
- laravel-specialistSkill
- legacy-modernizerSkill
- mcp-builderSkill
- mcp-developerSkill
- mcp-managementSkill
- media-processingSkill
- mermaidjs-v11Skill
- microservices-architectSkill
- ml-pipelineSkill
- monitoring-expertSkill
- The first 60 shown.
The file
We split this skill into the rules it gives an agent
5 rules. 0 are common, 3 are distinctive, and 2 are very specific to this project.
0
Common
Most of the 9 edited versions we read keep these.
3
Distinctive
This author's own choices. Worth a look.
2
Very specific
Tied to their project's names, files or tools. Probably skip.
- Common
- Distinctive
- Very specific
- Very specific: line 287
Use gemini-2.5-flash for most tasks (best price/performance)
- Distinctive: line 288
Use File API for files >20MB or repeated queries
- Distinctive: line 291
Use lower FPS for static content
- Distinctive: line 324
All scripts support unified API key detection and error handling:
- Very specific: line 347
Run any script with --help for detailed usage.
“Common” is measured: kept by at least 6 in 10 of the 9 edited versions of this file we could read. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.
Files
AI Multimodal Processing Skill
Process audio, images, videos, documents, and generate images using Google Gemini's multimodal API. Unified interface for all multimedia content understanding and generation.
Core Capabilities
Audio Processing
- Transcription with timestamps (up to 9.5 hours)
- Audio summarization and analysis
- Speech understanding and speaker identification
- Music and environmental sound analysis
- Text-to-speech generation with controllable voice
Image Understanding
- Image captioning and description
- Object detection with bounding boxes (2.0+)
- Pixel-level segmentation (2.5+)
- Visual question answering
- Multi-image comparison (up to 3,600 images)
- OCR and text extraction
Video Analysis
- Scene detection and summarization
- Video Q&A with temporal understanding
- Transcription with visual descriptions
- YouTube URL support
- Long video processing (up to 6 hours)
- Frame-level analysis
Document Extraction
- Native PDF vision processing (up to 1,000 pages)
- Table and form extraction
- Chart and diagram analysis
- Multi-page document understanding
- Structured data output (JSON schema)
- Format conversion (PDF to HTML/JSON)
Image Generation
- Text-to-image generation
- Image editing and modification
- Multi-image composition (up to 3 images)
- Iterative refinement
- Multiple aspect ratios (1:1, 16:9, 9:16, 4:3, 3:4)
- Controllable style and quality
Capability Matrix
| Task | Audio | Image | Video | Document | Generation |
| Transcription | ✓ | - | ✓ | - | - |
| Summarization | ✓ | ✓ | ✓ | ✓ | - |
| Q&A | ✓ | ✓ | ✓ | ✓ | - |
| Object Detection | - | ✓ | ✓ | - | - |
| Text Extraction | - | ✓ | - | ✓ | - |
| Structured Output | ✓ | ✓ | ✓ | ✓ | - |
| Creation | TTS | - | - | - | ✓ |
| Timestamps | ✓ | - | ✓ | - | - |
| Segmentation | - | ✓ | - | - | - |
Model Selection Guide
Gemini 2.5 Series (Recommended)
- gemini-2.5-pro: Highest quality, all features, 1M-2M context
- gemini-2.5-flash: Best balance, all features, 1M-2M context
- gemini-2.5-flash-lite: Lightweight, segmentation support
- gemini-2.5-flash-image: Image generation only
Gemini 2.0 Series
- gemini-2.0-flash: Fast processing, object detection
- gemini-2.0-flash-lite: Lightweight option
Feature Requirements
- Segmentation: Requires 2.5+ models
- Object Detection: Requires 2.0+ models
- Multi-video: Requires 2.5+ models
- Image Generation: Requires flash-image model
Context Windows
- 2M tokens: ~6 hours video (low-res) or ~2 hours (default)
- 1M tokens: ~3 hours video (low-res) or ~1 hour (default)
- Audio: 32 tokens/second (1 min = 1,920 tokens)
- PDF: 258 tokens/page (fixed)
- Image: 258-1,548 tokens based on size
Quick Start
Prerequisites
API Key Setup: Supports both Google AI Studio and Vertex AI.
The skill checks for GEMINI_API_KEY in this order:
- Process environment:
export GEMINI_API_KEY="your-key" - Project root:
.env .claude/.env.claude/skills/.env.claude/skills/ai-multimodal/.env
Get API key: https://aistudio.google.com/apikey
For Vertex AI:
export GEMINI_USE_VERTEX=true
export VERTEX_PROJECT_ID=your-gcp-project-id
export VERTEX_LOCATION=us-central1 # OptionalInstall SDK:
pip install google-genai python-dotenv pillowCommon Patterns
Transcribe Audio:
python scripts/gemini_batch_process.py \
--files audio.mp3 \
--task transcribe \
--model gemini-2.5-flashAnalyze Image:
python scripts/gemini_batch_process.py \
--files image.jpg \
--task analyze \
--prompt "Describe this image" \
--output docs/assets/<output-name>.md \
--model gemini-2.5-flashProcess Video:
python scripts/gemini_batch_process.py \
--files video.mp4 \
--task analyze \
--prompt "Summarize key points with timestamps" \
--output docs/assets/<output-name>.md \
--model gemini-2.5-flashExtract from PDF:
python scripts/gemini_batch_process.py \
--files document.pdf \
--task extract \
--prompt "Extract table data as JSON" \
--output docs/assets/<output-name>.md \
--format jsonGenerate Image:
python scripts/gemini_batch_process.py \
--task generate \
--prompt "A futuristic city at sunset" \
--output docs/assets/<output-file-name> \
--model gemini-2.5-flash-image \
--aspect-ratio 16:9Optimize Media:
# Prepare large video for processing
python scripts/media_optimizer.py \
--input large-video.mp4 \
--output docs/assets/<output-file-name> \
--target-size 100MB
# Batch optimize multiple files
python scripts/media_optimizer.py \
--input-dir ./videos \
--output-dir docs/assets/optimized \
--quality 85Convert Documents to Markdown:
# Convert to PDF
python scripts/document_converter.py \
--input document.docx \
--output docs/assets/document.md
# Extract pages
python scripts/document_converter.py \
--input large.pdf \
--output docs/assets/chapter1.md \
--pages 1-20Supported Formats
Audio
- WAV, MP3, AAC, FLAC, OGG Vorbis, AIFF
- Max 9.5 hours per request
- Auto-downsampled to 16 Kbps mono
Images
- PNG, JPEG, WEBP, HEIC, HEIF
- Max 3,600 images per request
- Resolution: ≤384px = 258 tokens, larger = tiled
Video
- MP4, MPEG, MOV, AVI, FLV, MPG, WebM, WMV, 3GPP
- Max 6 hours (low-res) or 2 hours (default)
- YouTube URLs supported (public only)
Documents
- PDF only for vision processing
- Max 1,000 pages
- TXT, HTML, Markdown supported (text-only)
Size Limits
- Inline: <20MB total request
- File API: 2GB per file, 20GB project quota
- Retention: 48 hours auto-delete
Reference Navigation
For detailed implementation guidance, see:
Audio Processing
references/audio-processing.md- Transcription, analysis, TTS- Timestamp handling and segment analysis
- Multi-speaker identification
- Non-speech audio analysis
- Text-to-speech generation
Image Understanding
references/vision-understanding.md- Captioning, detection, OCR- Object detection and localization
- Pixel-level segmentation
- Visual question answering
- Multi-image comparison
Video Analysis
references/video-analysis.md- Scene detection, temporal understanding- YouTube URL processing
- Timestamp-based queries
- Video clipping and FPS control
- Long video optimization
Document Extraction
references/document-extraction.md- PDF processing, structured output- Table and form extraction
- Chart and diagram analysis
- JSON schema validation
- Multi-page handling
Image Generation
references/image-generation.md- Text-to-image, editing- Prompt engineering strategies
- Image editing and composition
- Aspect ratio selection
- Safety settings
Cost Optimization
Token Costs
Input Pricing:
- Gemini 2.5 Flash: $1.00/1M input, $0.10/1M output
- Gemini 2.5 Pro: $3.00/1M input, $12.00/1M output
- Gemini 1.5 Flash: $0.70/1M input, $0.175/1M output
Token Rates:
- Audio: 32 tokens/second (1 min = 1,920 tokens)
- Video: ~300 tokens/second (default) or ~100 (low-res)
- PDF: 258 tokens/page (fixed)
- Image: 258-1,548 tokens based on size
TTS Pricing:
- Flash TTS: $10/1M tokens
- Pro TTS: $20/1M tokens
Best Practices
- Use
gemini-2.5-flashfor most tasks (best price/performance) - Use File API for files >20MB or repeated queries
- Optimize media before upload (see
media_optimizer.py) - Process specific segments instead of full videos
- Use lower FPS for static content
- Implement context caching for repeated queries
- Batch process multiple files in parallel
Rate Limits
Free Tier:
- 10-15 RPM (requests per minute)
- 1M-4M TPM (tokens per minute)
- 1,500 RPD (requests per day)
YouTube Limits:
- Free tier: 8 hours/day
- Paid tier: No length limits
- Public videos only
Storage Limits:
- 20GB per project
- 2GB per file
- 48-hour retention
Error Handling
Common errors and solutions:
- 400: Invalid format/size - validate before upload
- 401: Invalid API key - check configuration
- 403: Permission denied - verify API key restrictions
- 404: File not found - ensure file uploaded and active
- 429: Rate limit exceeded - implement exponential backoff
- 500: Server error - retry with backoff
Scripts Overview
All scripts support unified API key detection and error handling:
gemini_batch_process.py: Batch process multiple media files
- Supports all modalities (audio, image, video, PDF)
- Progress tracking and error recovery
- Output formats: JSON, Markdown, CSV
- Rate limiting and retry logic
- Dry-run mode
media_optimizer.py: Prepare media for Gemini API
- Compress videos/audio for size limits
- Resize images appropriately
- Split long videos into chunks
- Format conversion
- Quality vs size optimization
document_converter.py: Convert documents to PDF
- Convert DOCX, XLSX, PPTX to PDF
- Extract page ranges
- Optimize PDFs for Gemini
- Extract images from PDFs
- Batch conversion support
Run any script with --help for detailed usage.
Resources
Does it work?
Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.
No reviews yet
People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.
- It worked
- Partly
- It didn't
Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.
Your trail
Files you open will collect here, so you can find them again.
Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.
Now do this for your own.
Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.

