voice-references
Skill231 starsChanged 32 days agoMITView on GitHub
Make the reference clips a zero-shot voice cloner needs for each character (Fish Audio Drama 3 or S2 Pro, MiniMax, ElevenLabs-style cloning) from recordings the user has the rights to. Split into lines, pick clean 6-11 s runs, transcribe them, drop screams and noise, build a ~25 s clone clip plus single reference clips and mood clips (angry, drunk), check them, and upload them to the voice server. Use when adding a character's voice, when a cloned voice sounds off, thin or noisy, or when the user asks how to clone a voice for an NPC.
In this repository25 other files it gives its agents.
- AGENTS.mdAGENTS.md
- CLAUDE.mdCLAUDE.md
- fish-voice-serverSkill
- make-npcs-talkSkill
- npc-actionsSkill
- npc-dialogueSkill
- npc-showcase-clipsSkill
- voice-inputSkill
- voice-likeness-evalSkill
- voice-referencesSkill
- fish-voice-serverSkill
- make-npcs-talkSkill
- npc-actionsSkill
- npc-dialogueSkill
- npc-showcase-clipsSkill
- voice-inputSkill
- voice-likeness-evalSkill
- voice-referencesSkill
- fish-voice-serverSkill
- make-npcs-talkSkill
- npc-actionsSkill
- npc-dialogueSkill
- npc-showcase-clipsSkill
- voice-inputSkill
- voice-likeness-evalSkill
The file
We split this skill into the rules it gives an agent
3 rules. About 0 are common, 1 is distinctive, and 2 are very specific to this project.
≈0
Common · estimate
Many files like this have a rule of this kind.
≈1
Distinctive · estimate
This author's own choices. Worth a look.
2
Very specific
Tied to their project's names, files or tools. Probably skip.
- Common
- Distinctive
- Very specific
- Very specific: line 21
If music is unavoidable, isolate vocals first (Demucs, as server/finetune/prepare_dataset.py does).
- Very specific: line 36
Write ref_0..2 (the best three) and clone.wav: the best candidates joined up to ~25 s.
- Distinctive: line 56
Moods: keep 2 clips per mood. If an angry line sounds like a different person, the mood clips are too far from the neutral ones (another …
≈ “Common” is an estimate: at least 1 in 5 of the 364,980 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.
Files
Reference voices
A zero-shot cloner copies whatever is in the reference clip: the voice, but also the room, the music, the mic and the mood. Most "the clone sounds wrong" problems are reference problems.
Rights first
Clone only voices the user has the rights to: their own, an actor who agreed, a licensed or generated voice.
A game's characters are voiced by real people. For a private single-player experiment the user may decide to
use the game's own audio; it still never goes into a repo, a release or a public video without the rights.
npctalk check fails on audio files and .lab transcripts.
Good source audio
- One speaker, no music or effects under the voice (game cutscenes usually have both).
- Normal speaking level for the neutral voice; shouting goes in an
angrymood folder, slurring indrunk. - A minute or more of lines with pauses between them. More lines means more candidates to pick from.
- If music is unavoidable, isolate vocals first (Demucs, as
server/finetune/prepare_dataset.pydoes).
Make the clips
npctalk voice make --name Narrator --audio narrator.wav # neutral voice
npctalk voice make --name Narrator --audio narrator_angry.wav --mood angry # lines tagged [angry]
npctalk voice upload voices/Narrator # -> the server's /voices/NarratorWhat voice make does (the same steps built the GTA V example's 13 voices):
- Split into lines at pauses of 0.3 s (20 ms frames louder than -42 dBFS are speech).
- Join runs of up to 4 neighbouring lines into 6-11 s candidates; score them for normal level (about -20 dBFS, -15 for angry), even loudness, no clipping, few joins, ~9 s; keep the best 12 that don't overlap.
- Transcribe each with fal speech-to-text (
FAL_KEY). - Drop candidates with fewer than 6 words or a speaking rate outside 1.4-4.8 words/s (screams, grunts, music, another language).
- Write
ref_0..2(the best three) andclone.wav: the best candidates joined up to ~25 s. Every clip has a.labtranscript next to it; the cloner needs the exact words.
voices/Narrator/clone.wav + clone.lab what the server clones from by default ("Narrator/clone")
voices/Narrator/ref_0..2.wav + .lab single clips, for engines that want a short one
voices/Narrator/angry/ref_0..1.wav + .lab used when a line starts with [angry]
voices_work/Narrator/cand_*.wav + .lab every candidate, to listen to (never commit)Which clip to clone from (measured)
On Drama 3, the ~30 s clone.wav gave 0.680 voice match on Lamar's test lines; each of his single 6-10 s clips
gave 0.610-0.630. Passing all four clips as references at once was no better than clone.wav alone. S2 Pro
preferred a single clean clip (0.707 vs 0.680), so test both if you switch models (skill
voice-likeness-eval).
Check them
- Listen to
clone.wav. Anything in it (a laugh, a car, a second voice) shows up in every line. - Read the
.lab. A wrong transcript makes the cloner mumble or skip words; fix the text by hand. - Say a test line:
npctalk say "Testing, one two." --voice Narrator/clone --out t.wav. - Moods: keep 2 clips per mood. If an angry line sounds like a different person, the mood clips are too far from the neutral ones (another recording session or mic): drop them.
Upload and use
The server reads /voices in its Modal volume when it starts, so restart it after an upload
(modal app stop npcvoice-drama). A game refers to a voice by folder and clip name (Narrator/clone); mood
folders are picked by the line's leading tag. Until a restart, a game can send the clip by URL with its
transcript (reference_audio_url + reference_text), at ~9 s extra the first time.
Does it work?
Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.
No reviews yet
People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.
- It worked
- Partly
- It didn't
Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.
Your trail
Files you open will collect here, so you can find them again.
Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.
Now do this for your own.
Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.

