domain-audio-speech
Skill8 starsChanged 4 months agoMITView on GitHub
Use for audio and speech tasks: speech recognition (speech to text, ASR, transcription), audio or sound classification and tagging, keyword spotting, speaker identification or verification, sound event detection, and music or audio analysis. Picks the best model per sub-task and the right features, splits, and metrics. Triggers on 'speech to text', 'transcribe', 'ASR', 'audio classification', 'sound', 'keyword spotting', 'speaker recognition', 'voice', 'audio dataset'.
In this repository22 other files it gives its agents.
- accuracy-improvement-loopSkill
- data-rigor-and-leakageSkill
- dataset-profilingSkill
- deployment-explainabilitySkill
- domain-3dSkill
- domain-anomaly-detectionSkill
- domain-computer-visionSkill
- domain-generativeSkill
- domain-graphSkill
- domain-medical-imagingSkill
- domain-multimodalSkill
- domain-nlp-llmSkill
- domain-recommenderSkill
- domain-reinforcement-learningSkill
- domain-tabularSkill
- domain-time-seriesSkill
- domain-videoSkill
- literature-reviewSkill
- ml-research-methodologySkill
- notebook-deliverySkill
- rigorous-evaluationSkill
- training-optimizationSkill
The file
Files
Audio and Speech - Method Selection
Represent audio as a log-mel spectrogram for most classifiers, or feed the raw waveform to self-supervised speech models. Resample to the model's rate (16 kHz is common).
Decision table
| Sub-task | Recommended | Notes |
| Speech recognition (ASR) | Whisper (large-v3, or turbo for speed); wav2vec2 or HuBERT with CTC | Whisper is strong and multilingual zero-shot. Metric is WER, never accuracy. |
| Audio or sound classification and tagging | AST or PaSST (spectrogram transformers, AudioSet-pretrained); PANNs (CNN) | log-mel input. Multi-label tagging uses mAP; single-label uses accuracy or macro F1. |
| Keyword spotting | small CNN or MatchboxNet on log-mel | on-device. Track accuracy and false-alarm rate. |
| Speaker verification or ID | ECAPA-TDNN or x-vector embeddings + cosine similarity | metric-learning. Metrics are EER and minDCF. Split by speaker. |
| Sound event detection | CRNN or PaSST with framewise output | event-based F1, PSDS. |
Cross-cutting practice
- Augment with SpecAugment, time and pitch shift, and noise or room-impulse mixing. Normalize levels.
- Leakage: split by speaker, recording, or session, not by clip. The same speaker in train and test inflates results.
- Metrics: WER (and CER) for ASR; mAP for multi-label tagging; EER for verification. Do not report plain accuracy for ASR.
- Explainability: saliency or Grad-CAM on the spectrogram for classifiers; attention or confidence and alignment for ASR.
- Improve accuracy: use
accuracy-improvement-loop. Evaluate withrigorous-evaluationusing the metric that fits the sub-task.
Does it work?
Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.
No reviews yet
People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.
- It worked
- Partly
- It didn't
Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.
Your trail
Files you open will collect here, so you can find them again.
Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.
Now do this for your own.
Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.