xybrid-init
Skill524 starsChanged 3 days agoApache-2.0View on GitHub
Generate model metadata for an ML model so it works with xybrid.
In this repository4 other files it gives its agents.
- AGENTS.mdAGENTS.md
- CLAUDE.mdCLAUDE.md
- test-modelSkill
- xybrid-connectSkill
The file
We split this skill into the rules it gives an agent
12 rules. About 0 are common, 5 are distinctive, and 7 are very specific to this project.
≈0
Common · estimate
Many files like this have a rule of this kind.
≈5
Distinctive · estimate
This author's own choices. Worth a look.
7
Very specific
Tied to their project's names, files or tools. Probably skip.
- Common
- Distinctive
- Very specific
- Very specific: line 16
If $ARGUMENTS is provided, use it.
- Very specific: line 47
Read any README.md, config.json, tokenizer_config.json if present
- Very specific: line 48
If Python is available and there's an .onnx file, inspect it:
- Distinctive: line 61
Use the model card description, file extensions, and config to determine the model type:
- Distinctive: line 161
Use ImageNet normalization values unless model card specifies otherwise.
- Very specific: line 183
Add L2Normalize when the model's modules.json lists a Normalize module, so the vectors have unit length like sentence-transformers' output.
- Very specific: line 195
If the model has voice embeddings (e.g.
- Distinctive: line 215
Before presenting the result, verify:
- Distinctive: line 323
If the user confirms, write the file.
- Distinctive: line 329
If the source is HuggingFace and the model files aren't local yet, offer to download:
- Very specific: line 336
If the user confirms, download each file listed in the files array:
- Very specific: line 361
If /test-model is available (the user has xybrid cloned), suggest running it.
≈ “Common” is an estimate: at least 1 in 5 of the 372,429 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.
Files
Generate a model_metadata.json for any ML model so it works with xybrid.
The user may provide a HuggingFace repo, a local directory, or nothing (in which case ask).
Input: $ARGUMENTS (optional — HuggingFace repo ID, URL, or local path)
Step 1: Determine Source
If $ARGUMENTS is provided, use it. Otherwise ask:
What model do you want to set up?
- Paste a HuggingFace repo (e.g.
hexgrad/Kokoro-82M-v1.0-ONNX)- Or a local directory path (e.g.
./my-model/)
Detect the source type:
- Contains
huggingface.co/or matchesorg/repopattern → HuggingFace - Is a local path that exists → Local directory
- Otherwise → ask the user to clarify
Step 2: Gather Context
If HuggingFace:
Fetch these three resources (use WebFetch for each):
- Model card:
https://huggingface.co/{repo}/raw/main/README.md - File listing:
https://huggingface.co/api/models/{repo}(look at thesiblingsarray for file names and sizes) - Config file (if exists):
https://huggingface.co/{repo}/raw/main/config.json
Also check for these files (fetch if they exist in the file listing):
tokenizer_config.jsontokenizer.jsongeneration_config.json
If Local directory:
- List all files in the directory
- Read any
README.md,config.json,tokenizer_config.jsonif present - If Python is available and there's an
.onnxfile, inspect it:python3 -c "import onnx; m = onnx.load('MODEL.onnx'); print('Inputs:', [(i.name, [d.dim_value for d in i.type.tensor_type.shape.dim]) for i in m.graph.input]); print('Outputs:', [(o.name, [d.dim_value for d in o.type.tensor_type.shape.dim]) for o in m.graph.output])"
Step 3: Analyze and Generate
Using ALL the gathered context (model card, file list, config, ONNX inputs/outputs), generate a valid model_metadata.json.
Decision Tree
Use the model card description, file extensions, and config to determine the model type:
File-based detection:
.gguffile → LLM (Gguf template)- GGML Whisper weights (
ggml-*.bin, or a.binwhose model card says whisper.cpp / GGML) → ASR (GgmlWhisper template) — the default ASR path .safetensors+ whisper architecture → ASR (SafeTensors template, Candle runtime) — opt-in only, needs--features candle; prefer the GGML bundle above.onnxfile → continue to task detection below.mlmodel/.mlpackage→ CoreML template.tflite→ TfLite template
Task detection (from model card + config):
- Model card mentions "text-to-speech", "TTS", "speech synthesis" → TTS
- Model card mentions "speech recognition", "ASR", "transcription" → ASR
- Model card mentions "embedding", "sentence transformer", "similarity" → Embedding
- Model card mentions "classification", "image", "vision", "ImageNet" → Vision
- Model card mentions "language model", "text generation", "chat", "instruct" → LLM
- Model card mentions "object detection", "YOLO", "segmentation" → Vision
Schema Reference
The model_metadata.json must conform to this exact schema:
{
"model_id": "string (required) — kebab-case identifier",
"version": "string (required) — model version",
"description": "string (optional) — human-readable description",
"execution_template": { "type": "...", ... },
"preprocessing": [ ... ],
"postprocessing": [ ... ],
"files": [ "list of all required files" ],
"metadata": { "task": "...", ... },
"voices": { "... (TTS only)" }
}Common optional metadata fields:
tool_calling(boolean, LLMs only): advisory declaration that xybrid's local tool calling works end-to-end for this model — the template accepts atoolscontext AND the model emits a call format xybrid parses (currently LFM2-family pythonic, LFM3<tool_call><function=…>XML, gemma-4-familycall:, and FunctionGemma<start_function_call>notation). Declaretrueonly for those verified families — not Qwen3.5, whose calls use LFM3's notation and parse, but whose tool-result continuations xybrid rejects; omit when unknown (never infer from architecture); a model whose template renders tools but whose emissions xybrid cannot parse must NOT declaretrue— it would produce silent no-call responses.
Execution Templates
Choose ONE:
// ONNX model
{ "type": "Onnx", "model_file": "model.onnx" }
// GGML Whisper (whisper.cpp — the default ASR path, feature `asr-whispercpp`)
// `language`: omit or null to auto-detect. `audio_ctx`: 0 = no encoder
// truncation (the safe default — truncating is the biggest streaming speed
// lever but too much of it makes the decoder loop, so opt in per model after
// a quality check). `translate`: true translates to English instead of
// transcribing in the source language.
{ "type": "GgmlWhisper", "model_file": "model.bin", "language": "en", "audio_ctx": 0, "translate": false }
// SafeTensors (Candle runtime — Whisper only; opt-in `candle` feature, in no platform preset)
{ "type": "SafeTensors", "model_file": "model.safetensors", "architecture": "whisper", "config_file": "config.json", "tokenizer_file": "tokenizer.json" }
// GGUF (local LLMs via llama.cpp)
{ "type": "Gguf", "model_file": "model.gguf", "context_length": 4096 }
// CoreML (Apple platforms)
{ "type": "CoreMl", "model_file": "model.mlpackage" }
// TFLite (mobile)
{ "type": "TfLite", "model_file": "model.tflite" }Preprocessing Steps
Choose the appropriate chain based on task:
TTS (text-to-speech):
[{ "type": "Phonemize", "tokens_file": "tokens.txt", "backend": "MisakiDictionary", "add_padding": true, "normalize_text": true }]Backends: MisakiDictionary (default, pure Rust), EspeakNG (multi-language, needs system install), CmuDictionary (legacy), OpenPhonemizer (hybrid dictionary + neural)
ASR (speech recognition) with ONNX:
[{ "type": "AudioDecode", "sample_rate": 16000, "channels": 1 }]ASR with Whisper SafeTensors: empty [] (Candle handles internally)
Text embedding / NLP:
[{ "type": "Tokenize", "vocab_file": "tokenizer.json", "tokenizer_type": "WordPiece", "max_length": 512, "add_special_tokens": true }]Tokenizer types: WordPiece (BERT), BPE (GPT), SentencePiece (T5)
For a sentence-transformers embedding model, set add_special_tokens: true: the tokenizer's post_processor then adds the tokens the model was trained with ([CLS]/[SEP], <bos>/<eos>). Without them the vectors drift from the reference (all-MiniLM-L6-v2: cosine 0.85–0.95). An ONNX export that also declares image_features, video_features and audio_features inputs (multimodal embedding models such as EmbeddingGemma 2) runs on text alone: those inputs get zero rows.
Image classification / vision:
[
{ "type": "Resize", "width": 224, "height": 224 },
{ "type": "Normalize", "mean": [0.485, 0.456, 0.406], "std": [0.229, 0.224, 0.225] }
]Use ImageNet normalization values unless model card specifies otherwise.
LLM (GGUF): empty [] (llama.cpp handles internally)
Postprocessing Steps
TTS:
[{ "type": "TTSAudioEncode", "sample_rate": 24000, "apply_postprocessing": true }]ASR (CTC-based, e.g. Wav2Vec2):
[{ "type": "CTCDecode", "vocab_file": "vocab.json", "blank_index": 0 }]ASR (Whisper SafeTensors): empty []
Text embedding:
[{ "type": "MeanPool", "dim": 1 }, { "type": "L2Normalize" }]Add L2Normalize when the model's modules.json lists a Normalize module, so the vectors have unit length like sentence-transformers' output.
Image classification:
[{ "type": "Softmax", "dim": 1 }]Or { "type": "Argmax" } if you just need the class index.
LLM: empty []
Voice Config (TTS only)
If the model has voice embeddings (e.g. voices.bin):
{
"voices": {
"format": "embedded",
"file": "voices.bin",
"loader": "binary_f32_256",
"default": "voice_id",
"selection_strategy": "TokenLength",
"catalog": [
{ "id": "voice_id", "name": "Display Name", "index": 0, "gender": "female", "language": "en-US", "style": "neutral" }
]
}
}Step 4: Validate
Before presenting the result, verify:
- All files in
filesarray exist (in the HF repo or local directory) model_filematches an actual file name- Preprocessing/postprocessing steps match the model task
- Step types are valid — only use the types listed above
- The JSON is valid — parseable, no trailing commas
Step 5: Real Examples for Reference
TTS (Kokoro 82M)
{
"model_id": "kokoro-82m",
"version": "1.0",
"description": "Kokoro 82M - High-quality TTS with 24 voices",
"execution_template": { "type": "Onnx", "model_file": "model.onnx" },
"preprocessing": [{ "type": "Phonemize", "tokens_file": "tokens.txt", "backend": "MisakiDictionary", "add_padding": true, "normalize_text": true }],
"postprocessing": [{ "type": "TTSAudioEncode", "sample_rate": 24000, "apply_postprocessing": true }],
"files": ["model.onnx", "voices.bin", "tokens.txt", "misaki/us_gold.json"],
"metadata": { "task": "text-to-speech", "sample_rate": 24000, "parameters": 82000000, "family": "hexgrad", "license": "Apache-2.0" }
}LLM (Qwen 3.5 0.8B)
{
"model_id": "qwen3.5-0.8b",
"version": "1.0",
"description": "Qwen 3.5 0.8B - Lightweight multilingual LLM",
"execution_template": { "type": "Gguf", "model_file": "Qwen3.5-0.8B-Q4_K_M.gguf", "context_length": 4096 },
"preprocessing": [],
"postprocessing": [],
"files": ["Qwen3.5-0.8B-Q4_K_M.gguf"],
"metadata": { "task": "text-generation", "architecture": "qwen35", "backend": "llamacpp", "parameters": 800000000, "license": "Apache-2.0" }
}Text Embedding (all-MiniLM)
{
"model_id": "all-minilm",
"version": "L6-v2",
"execution_template": { "type": "Onnx", "model_file": "model.onnx" },
"preprocessing": [{ "type": "Tokenize", "vocab_file": "tokenizer.json", "tokenizer_type": "WordPiece", "max_length": 512, "add_special_tokens": true }],
"postprocessing": [{ "type": "MeanPool", "dim": 1 }, { "type": "L2Normalize" }],
"files": ["model.onnx", "vocab.txt", "tokenizer.json", "config.json"],
"metadata": { "task": "text-embedding", "embedding_dim": 384 }
}Text Embedding (EmbeddingGemma 2, onnx-community/embeddinggemma-2-ONNX)
List the external weights file: the graph loads it from next to the .onnx. Never pick the fp16 or q4f16 exports (NaN or degraded vectors). Callers prefix queries with task: search result | query: and documents with title: none | text: .
{
"model_id": "embeddinggemma-2",
"version": "q4",
"execution_template": { "type": "Onnx", "model_file": "model_q4.onnx" },
"preprocessing": [{ "type": "Tokenize", "vocab_file": "tokenizer.json", "tokenizer_type": "BPE", "max_length": 8192, "add_special_tokens": true }],
"postprocessing": [{ "type": "MeanPool", "dim": 1 }, { "type": "L2Normalize" }],
"files": ["model_q4.onnx", "model_q4.onnx_data", "tokenizer.json"],
"metadata": { "task": "text-embedding", "embedding_dim": 768 }
}ASR (Whisper Tiny — SafeTensors)
{
"model_id": "whisper-tiny",
"version": "1.0",
"description": "Whisper Tiny - Fast multilingual ASR (Candle runtime)",
"execution_template": { "type": "SafeTensors", "model_file": "model.safetensors", "config_file": "config.json", "tokenizer_file": "tokenizer.json" },
"preprocessing": [],
"postprocessing": [],
"files": ["model.safetensors", "config.json", "tokenizer.json", "melfilters.bytes"],
"metadata": { "task": "speech-recognition", "runtime": "candle", "sample_rate": 16000, "parameters": 39000000 }
}Image Classification (MNIST)
{
"model_id": "mnist-digit-recognition",
"version": "12",
"description": "MNIST handwritten digit recognition",
"execution_template": { "type": "Onnx", "model_file": "model.onnx" },
"preprocessing": [
{ "type": "Reshape", "shape": [1, 1, 28, 28] },
{ "type": "Normalize", "mean": [0.0], "std": [255.0] }
],
"postprocessing": [{ "type": "Softmax", "dim": 1 }],
"files": ["model.onnx"],
"metadata": { "task": "image_classification", "input_shape": [1, 1, 28, 28], "num_classes": 10 }
}Step 6: Present and Save
Show the generated model_metadata.json to the user with a brief explanation of the key decisions made (e.g., "Using MisakiDictionary phonemizer because the model card says it's a Kokoro-based TTS model").
Then ask:
Save to
{directory}/model_metadata.json?
If the user confirms, write the file.
Step 7: Download Model Files (if HuggingFace)
If the source is HuggingFace and the model files aren't local yet, offer to download:
Download model files (~{size})? This will fetch:
model.onnx(150 MB)voices.bin(24 KB)- ...
If the user confirms, download each file listed in the files array:
curl -L "https://huggingface.co/{repo}/resolve/main/{file}" -o "{directory}/{file}"For files in subdirectories (e.g. misaki/us_gold.json), create the subdirectory first.
Step 8: Next Steps
After saving, print:
Your model is ready. Next steps:
# Test it works (use --input-audio <file>.wav for ASR models)
xybrid run --model {model_id} --input-text "test input"
# Or from Rust
cargo run --example your_test -p xybrid-core
# Or use /test-model to validate end-to-endIf /test-model is available (the user has xybrid cloned), suggest running it.
Does it work?
Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.
No reviews yet
People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.
- It worked
- Partly
- It didn't
Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.
Your trail
Files you open will collect here, so you can find them again.
Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.
Now do this for your own.
Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.

