audio-to-midi
Skill21 starsChanged 7 months agoMITView on GitHub
Convert MP3/WAV/FLAC audio files to MIDI (.mid) and MusicXML (.musicxml) with full music analysis. Two engines: Basic Pitch (general-purpose polyphonic) and Piano Model (high-accuracy piano, 96.7% F1). Optional Demucs stem separation. Use --engine piano for piano/keyboard music for best results. Trigger when the user wants to: transcribe audio to MIDI, convert music to sheet music/notes, extract notes from audio, get MusicXML from a recording, analyze tempo/key/chords of a song, separate stems (vocals/drums/bass) and transcribe each, or any audio-to-notation task. Also triggers for requests like "convert this MP3 to MIDI", "get the notes from this song", "what key is this song in", "transcribe this audio", "separate and transcribe stems".
In this repository22 other files it gives its agents.
- 2d-pixel-assetSkill
- anthropic-skilljar-extractorSkill
- auto-researchSkill
- gemini-image-genSkill
- gemini-web-media-analysisSkill
- google-ai-studio-ttsSkill
- hermes-tweetSkill
- link-downloadSkill
- llama-cppSkill
- meta-muse-video-analysisSkill
- midi-synthSkill
- opencodeSkill
- seedanceSkill
- tech-product-landingSkill
- unreal-best-practicesSkill
- unreal-blueprint-codegenSkill
- unreal-gasSkill
- unreal-pcg-pythonSkill
- unreal-thirdpartySkill
- video-downloadSkill
- video-fetch-and-summarizeSkill
- video-summarizerSkill
The file
We split this skill into the rules it gives an agent
6 rules. About 0 are common, 1 is distinctive, and 5 are very specific to this project.
≈0
Common · estimate
Many files like this have a rule of this kind.
≈1
Distinctive · estimate
This author's own choices. Worth a look.
5
Very specific
Tied to their project's names, files or tools. Probably skip.
- Common
- Distinctive
- Very specific
- Very specific: line 26
Stems?: Add --stems to separate vocals/drums/bass/other with Demucs first
- Very specific: line 27
Output dir?: Use -o path or default to input file's directory
- Very specific: line 28
Skip analysis?: Add --no-analysis if only MIDI/MusicXML needed
- Very specific: line 29
Skip MusicXML?: Add --no-musicxml if only MIDI needed
- Distinctive: line 30
Run the transcription script
- Very specific: line 110
Important: Use py -3.12 (not python) to run the script.
≈ “Common” is an estimate: at least 1 in 5 of the 350,495 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.
Files
Audio-to-MIDI Transcription
Convert audio files to MIDI and MusicXML with full music analysis (tempo, key, chords, dynamics, instruments).
Tool
Script: scripts/transcribe.py -- wraps Basic Pitch, Demucs, librosa, and music21. Auto-installs dependencies.
Workflow
- Identify the input audio file (MP3, WAV, FLAC, OGG, M4A)
- Determine options:
- Stems? Add
--stemsto separate vocals/drums/bass/other with Demucs first - Output dir? Use
-o pathor default to input file's directory - Skip analysis? Add
--no-analysisif only MIDI/MusicXML needed - Skip MusicXML? Add
--no-musicxmlif only MIDI needed
- Stems? Add
- Run the transcription script
- Report output files and analysis summary to user
Engines
| Engine | Flag | Best for | Accuracy |
| Basic Pitch | --engine basic-pitch (default) | Mixed/polyphonic music | Good |
| Piano Model | --engine piano | Piano/keyboard music | 96.7% F1 |
For piano or keyboard music, always use --engine piano — it captures sustain/pedal, has far fewer ghost notes, and produces much more accurate MIDI.
Usage
# Piano music (recommended for piano/keyboard)
py -3.12 scripts/transcribe.py "piano.mp3" --engine piano
# General music (default engine: Basic Pitch)
py -3.12 scripts/transcribe.py "song.mp3"
# With stem separation (Demucs): each stem gets its own MIDI + MusicXML
py -3.12 scripts/transcribe.py "song.wav" --stems
# Custom output directory
py -3.12 scripts/transcribe.py "song.mp3" -o ./output --engine piano
# Tuning Basic Pitch sensitivity
py -3.12 scripts/transcribe.py "song.mp3" --onset-threshold 0.6 --frame-threshold 0.4
# MIDI only (skip MusicXML)
py -3.12 scripts/transcribe.py "song.mp3" --no-musicxml
# MIDI + MusicXML without analysis
py -3.12 scripts/transcribe.py "song.mp3" --no-analysisOutput Files
For input song.mp3:
song.mid-- MIDI file (for DAWs, notation software)song.musicxml-- MusicXML (for MuseScore, Finale, Sibelius, Dorico)song_analysis.json-- Full analysis (tempo, key, chords, dynamics, spectral)
With --stems, each stem produces its own MIDI + MusicXML:
vocals.mid,vocals.musicxmldrums.mid,drums.musicxmlbass.mid,bass.musicxmlother.mid,other.musicxml
Analysis Output
The _analysis.json contains:
- tempo_bpm: Detected BPM
- key: Detected key and mode (e.g. "A minor")
- key_confidence: 0-1 confidence score
- chords: Time-stamped chord progression
- unique_chords: Deduplicated chord list
- dynamics: Mean/max/min dB, dynamic range
- spectral: Centroid, bandwidth, rolloff, ZCR
- instrument_hints: Detected instrument categories
Tuning Parameters
| Flag | Default | Effect |
--onset-threshold | 0.5 | Higher = fewer ghost notes, may miss quiet notes |
--frame-threshold | 0.3 | Higher = stricter note detection |
--min-note-length | 58 | Minimum note duration in ms |
For clean recordings (piano, guitar): defaults work well.
For complex mixes: use --stems for best results.
For percussive music: lower onset threshold to 0.3-0.4.
Dependencies
Auto-installed on first run: basic-pitch, librosa, music21, pretty_midi, numpy, onnxruntime.
With --stems: also installs demucs (includes PyTorch).
Requires: Python 3.12 (py -3.12), ffmpeg (for MP3 decoding).
Important: Use py -3.12 (not python) to run the script. Python 3.14 has compatibility issues with ML packages. The script auto-selects the ONNX backend for Basic Pitch (most compatible).
Supported Input Formats
MP3, WAV, FLAC, OGG, M4A, AAC, WMA
Does it work?
Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.
No reviews yet
People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.
- It worked
- Partly
- It didn't
Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.
Your trail
Files you open will collect here, so you can find them again.
Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.
Now do this for your own.
Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.