agentleFS
xybrid-aixybrid-ai/xybrid/.agents/skills/xybrid-init/SKILL.md

xybrid-init

Skill524 starsChanged 3 days agoApache-2.0View on GitHub

Generate model metadata for an ML model so it works with xybrid.

The file

We split this skill into the rules it gives an agent

12 rules. About 0 are common, 5 are distinctive, and 7 are very specific to this project.

  • ≈0

    Common · estimate

    Many files like this have a rule of this kind.

  • ≈5

    Distinctive · estimate

    This author's own choices. Worth a look.

  • 7

    Very specific

    Tied to their project's names, files or tools. Probably skip.

  • Common
  • Distinctive
  • Very specific
  1. Very specific:

    If $ARGUMENTS is provided, use it.

    line 16
  2. Very specific:

    Read any README.md, config.json, tokenizer_config.json if present

    line 47
  3. Very specific:

    If Python is available and there's an .onnx file, inspect it:

    line 48
  4. Distinctive:

    Use the model card description, file extensions, and config to determine the model type:

    line 61
  5. Distinctive:

    Use ImageNet normalization values unless model card specifies otherwise.

    line 161
  6. Very specific:

    Add L2Normalize when the model's modules.json lists a Normalize module, so the vectors have unit length like sentence-transformers' output.

    line 183
  7. Very specific:

    If the model has voice embeddings (e.g.

    line 195
  8. Distinctive:

    Before presenting the result, verify:

    line 215
  9. Distinctive:

    If the user confirms, write the file.

    line 323
  10. Distinctive:

    If the source is HuggingFace and the model files aren't local yet, offer to download:

    line 329
  11. Very specific:

    If the user confirms, download each file listed in the files array:

    line 336
  12. Very specific:

    If /test-model is available (the user has xybrid cloned), suggest running it.

    line 361

≈ “Common” is an estimate: at least 1 in 5 of the 372,429 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.

Keep what fits and drop the rest. Your version, ready for your agents.
Files
SKILL.md

Generate a model_metadata.json for any ML model so it works with xybrid.

The user may provide a HuggingFace repo, a local directory, or nothing (in which case ask).

Input: $ARGUMENTS (optional — HuggingFace repo ID, URL, or local path)


Step 1: Determine Source

If $ARGUMENTS is provided, use it. Otherwise ask:

What model do you want to set up?

  • Paste a HuggingFace repo (e.g. hexgrad/Kokoro-82M-v1.0-ONNX)
  • Or a local directory path (e.g. ./my-model/)

Detect the source type:

  • Contains huggingface.co/ or matches org/repo pattern → HuggingFace
  • Is a local path that exists → Local directory
  • Otherwise → ask the user to clarify

Step 2: Gather Context

If HuggingFace:

Fetch these three resources (use WebFetch for each):

  1. Model card: https://huggingface.co/{repo}/raw/main/README.md
  2. File listing: https://huggingface.co/api/models/{repo} (look at the siblings array for file names and sizes)
  3. Config file (if exists): https://huggingface.co/{repo}/raw/main/config.json

Also check for these files (fetch if they exist in the file listing):

  • tokenizer_config.json
  • tokenizer.json
  • generation_config.json

If Local directory:

  1. List all files in the directory
  2. Read any README.md, config.json, tokenizer_config.json if present
  3. If Python is available and there's an .onnx file, inspect it:
    python3 -c "import onnx; m = onnx.load('MODEL.onnx'); print('Inputs:', [(i.name, [d.dim_value for d in i.type.tensor_type.shape.dim]) for i in m.graph.input]); print('Outputs:', [(o.name, [d.dim_value for d in o.type.tensor_type.shape.dim]) for o in m.graph.output])"

Step 3: Analyze and Generate

Using ALL the gathered context (model card, file list, config, ONNX inputs/outputs), generate a valid model_metadata.json.

Decision Tree

Use the model card description, file extensions, and config to determine the model type:

File-based detection:

  • .gguf file → LLM (Gguf template)
  • GGML Whisper weights (ggml-*.bin, or a .bin whose model card says whisper.cpp / GGML) → ASR (GgmlWhisper template) — the default ASR path
  • .safetensors + whisper architecture → ASR (SafeTensors template, Candle runtime) — opt-in only, needs --features candle; prefer the GGML bundle above
  • .onnx file → continue to task detection below
  • .mlmodel / .mlpackage → CoreML template
  • .tflite → TfLite template

Task detection (from model card + config):

  • Model card mentions "text-to-speech", "TTS", "speech synthesis" → TTS
  • Model card mentions "speech recognition", "ASR", "transcription" → ASR
  • Model card mentions "embedding", "sentence transformer", "similarity" → Embedding
  • Model card mentions "classification", "image", "vision", "ImageNet" → Vision
  • Model card mentions "language model", "text generation", "chat", "instruct" → LLM
  • Model card mentions "object detection", "YOLO", "segmentation" → Vision

Schema Reference

The model_metadata.json must conform to this exact schema:

{
  "model_id": "string (required) — kebab-case identifier",
  "version": "string (required) — model version",
  "description": "string (optional) — human-readable description",
  "execution_template": { "type": "...", ... },
  "preprocessing": [ ... ],
  "postprocessing": [ ... ],
  "files": [ "list of all required files" ],
  "metadata": { "task": "...", ... },
  "voices": { "... (TTS only)" }
}

Common optional metadata fields:

  • tool_calling (boolean, LLMs only): advisory declaration that xybrid's local tool calling works end-to-end for this model — the template accepts a tools context AND the model emits a call format xybrid parses (currently LFM2-family pythonic, LFM3 <tool_call><function=…> XML, gemma-4-family call:, and FunctionGemma <start_function_call> notation). Declare true only for those verified families — not Qwen3.5, whose calls use LFM3's notation and parse, but whose tool-result continuations xybrid rejects; omit when unknown (never infer from architecture); a model whose template renders tools but whose emissions xybrid cannot parse must NOT declare true — it would produce silent no-call responses.

Execution Templates

Choose ONE:

// ONNX model
{ "type": "Onnx", "model_file": "model.onnx" }

// GGML Whisper (whisper.cpp — the default ASR path, feature `asr-whispercpp`)
// `language`: omit or null to auto-detect. `audio_ctx`: 0 = no encoder
// truncation (the safe default — truncating is the biggest streaming speed
// lever but too much of it makes the decoder loop, so opt in per model after
// a quality check). `translate`: true translates to English instead of
// transcribing in the source language.
{ "type": "GgmlWhisper", "model_file": "model.bin", "language": "en", "audio_ctx": 0, "translate": false }

// SafeTensors (Candle runtime — Whisper only; opt-in `candle` feature, in no platform preset)
{ "type": "SafeTensors", "model_file": "model.safetensors", "architecture": "whisper", "config_file": "config.json", "tokenizer_file": "tokenizer.json" }

// GGUF (local LLMs via llama.cpp)
{ "type": "Gguf", "model_file": "model.gguf", "context_length": 4096 }

// CoreML (Apple platforms)
{ "type": "CoreMl", "model_file": "model.mlpackage" }

// TFLite (mobile)
{ "type": "TfLite", "model_file": "model.tflite" }

Preprocessing Steps

Choose the appropriate chain based on task:

TTS (text-to-speech):

[{ "type": "Phonemize", "tokens_file": "tokens.txt", "backend": "MisakiDictionary", "add_padding": true, "normalize_text": true }]

Backends: MisakiDictionary (default, pure Rust), EspeakNG (multi-language, needs system install), CmuDictionary (legacy), OpenPhonemizer (hybrid dictionary + neural)

ASR (speech recognition) with ONNX:

[{ "type": "AudioDecode", "sample_rate": 16000, "channels": 1 }]

ASR with Whisper SafeTensors: empty [] (Candle handles internally)

Text embedding / NLP:

[{ "type": "Tokenize", "vocab_file": "tokenizer.json", "tokenizer_type": "WordPiece", "max_length": 512, "add_special_tokens": true }]

Tokenizer types: WordPiece (BERT), BPE (GPT), SentencePiece (T5)

For a sentence-transformers embedding model, set add_special_tokens: true: the tokenizer's post_processor then adds the tokens the model was trained with ([CLS]/[SEP], <bos>/<eos>). Without them the vectors drift from the reference (all-MiniLM-L6-v2: cosine 0.85–0.95). An ONNX export that also declares image_features, video_features and audio_features inputs (multimodal embedding models such as EmbeddingGemma 2) runs on text alone: those inputs get zero rows.

Image classification / vision:

[
  { "type": "Resize", "width": 224, "height": 224 },
  { "type": "Normalize", "mean": [0.485, 0.456, 0.406], "std": [0.229, 0.224, 0.225] }
]

Use ImageNet normalization values unless model card specifies otherwise.

LLM (GGUF): empty [] (llama.cpp handles internally)

Postprocessing Steps

TTS:

[{ "type": "TTSAudioEncode", "sample_rate": 24000, "apply_postprocessing": true }]

ASR (CTC-based, e.g. Wav2Vec2):

[{ "type": "CTCDecode", "vocab_file": "vocab.json", "blank_index": 0 }]

ASR (Whisper SafeTensors): empty []

Text embedding:

[{ "type": "MeanPool", "dim": 1 }, { "type": "L2Normalize" }]

Add L2Normalize when the model's modules.json lists a Normalize module, so the vectors have unit length like sentence-transformers' output.

Image classification:

[{ "type": "Softmax", "dim": 1 }]

Or { "type": "Argmax" } if you just need the class index.

LLM: empty []

Voice Config (TTS only)

If the model has voice embeddings (e.g. voices.bin):

{
  "voices": {
    "format": "embedded",
    "file": "voices.bin",
    "loader": "binary_f32_256",
    "default": "voice_id",
    "selection_strategy": "TokenLength",
    "catalog": [
      { "id": "voice_id", "name": "Display Name", "index": 0, "gender": "female", "language": "en-US", "style": "neutral" }
    ]
  }
}

Step 4: Validate

Before presenting the result, verify:

  1. All files in files array exist (in the HF repo or local directory)
  2. model_file matches an actual file name
  3. Preprocessing/postprocessing steps match the model task
  4. Step types are valid — only use the types listed above
  5. The JSON is valid — parseable, no trailing commas

Step 5: Real Examples for Reference

TTS (Kokoro 82M)

{
  "model_id": "kokoro-82m",
  "version": "1.0",
  "description": "Kokoro 82M - High-quality TTS with 24 voices",
  "execution_template": { "type": "Onnx", "model_file": "model.onnx" },
  "preprocessing": [{ "type": "Phonemize", "tokens_file": "tokens.txt", "backend": "MisakiDictionary", "add_padding": true, "normalize_text": true }],
  "postprocessing": [{ "type": "TTSAudioEncode", "sample_rate": 24000, "apply_postprocessing": true }],
  "files": ["model.onnx", "voices.bin", "tokens.txt", "misaki/us_gold.json"],
  "metadata": { "task": "text-to-speech", "sample_rate": 24000, "parameters": 82000000, "family": "hexgrad", "license": "Apache-2.0" }
}

LLM (Qwen 3.5 0.8B)

{
  "model_id": "qwen3.5-0.8b",
  "version": "1.0",
  "description": "Qwen 3.5 0.8B - Lightweight multilingual LLM",
  "execution_template": { "type": "Gguf", "model_file": "Qwen3.5-0.8B-Q4_K_M.gguf", "context_length": 4096 },
  "preprocessing": [],
  "postprocessing": [],
  "files": ["Qwen3.5-0.8B-Q4_K_M.gguf"],
  "metadata": { "task": "text-generation", "architecture": "qwen35", "backend": "llamacpp", "parameters": 800000000, "license": "Apache-2.0" }
}

Text Embedding (all-MiniLM)

{
  "model_id": "all-minilm",
  "version": "L6-v2",
  "execution_template": { "type": "Onnx", "model_file": "model.onnx" },
  "preprocessing": [{ "type": "Tokenize", "vocab_file": "tokenizer.json", "tokenizer_type": "WordPiece", "max_length": 512, "add_special_tokens": true }],
  "postprocessing": [{ "type": "MeanPool", "dim": 1 }, { "type": "L2Normalize" }],
  "files": ["model.onnx", "vocab.txt", "tokenizer.json", "config.json"],
  "metadata": { "task": "text-embedding", "embedding_dim": 384 }
}

Text Embedding (EmbeddingGemma 2, onnx-community/embeddinggemma-2-ONNX)

List the external weights file: the graph loads it from next to the .onnx. Never pick the fp16 or q4f16 exports (NaN or degraded vectors). Callers prefix queries with task: search result | query: and documents with title: none | text: .

{
  "model_id": "embeddinggemma-2",
  "version": "q4",
  "execution_template": { "type": "Onnx", "model_file": "model_q4.onnx" },
  "preprocessing": [{ "type": "Tokenize", "vocab_file": "tokenizer.json", "tokenizer_type": "BPE", "max_length": 8192, "add_special_tokens": true }],
  "postprocessing": [{ "type": "MeanPool", "dim": 1 }, { "type": "L2Normalize" }],
  "files": ["model_q4.onnx", "model_q4.onnx_data", "tokenizer.json"],
  "metadata": { "task": "text-embedding", "embedding_dim": 768 }
}

ASR (Whisper Tiny — SafeTensors)

{
  "model_id": "whisper-tiny",
  "version": "1.0",
  "description": "Whisper Tiny - Fast multilingual ASR (Candle runtime)",
  "execution_template": { "type": "SafeTensors", "model_file": "model.safetensors", "config_file": "config.json", "tokenizer_file": "tokenizer.json" },
  "preprocessing": [],
  "postprocessing": [],
  "files": ["model.safetensors", "config.json", "tokenizer.json", "melfilters.bytes"],
  "metadata": { "task": "speech-recognition", "runtime": "candle", "sample_rate": 16000, "parameters": 39000000 }
}

Image Classification (MNIST)

{
  "model_id": "mnist-digit-recognition",
  "version": "12",
  "description": "MNIST handwritten digit recognition",
  "execution_template": { "type": "Onnx", "model_file": "model.onnx" },
  "preprocessing": [
    { "type": "Reshape", "shape": [1, 1, 28, 28] },
    { "type": "Normalize", "mean": [0.0], "std": [255.0] }
  ],
  "postprocessing": [{ "type": "Softmax", "dim": 1 }],
  "files": ["model.onnx"],
  "metadata": { "task": "image_classification", "input_shape": [1, 1, 28, 28], "num_classes": 10 }
}

Step 6: Present and Save

Show the generated model_metadata.json to the user with a brief explanation of the key decisions made (e.g., "Using MisakiDictionary phonemizer because the model card says it's a Kokoro-based TTS model").

Then ask:

Save to {directory}/model_metadata.json?

If the user confirms, write the file.


Step 7: Download Model Files (if HuggingFace)

If the source is HuggingFace and the model files aren't local yet, offer to download:

Download model files (~{size})? This will fetch:

  • model.onnx (150 MB)
  • voices.bin (24 KB)
  • ...

If the user confirms, download each file listed in the files array:

curl -L "https://huggingface.co/{repo}/resolve/main/{file}" -o "{directory}/{file}"

For files in subdirectories (e.g. misaki/us_gold.json), create the subdirectory first.


Step 8: Next Steps

After saving, print:

Your model is ready. Next steps:

# Test it works (use --input-audio <file>.wav for ASR models)
xybrid run --model {model_id} --input-text "test input"

# Or from Rust
cargo run --example your_test -p xybrid-core

# Or use /test-model to validate end-to-end

If /test-model is available (the user has xybrid cloned), suggest running it.

Download now (ZIP)

Does it work?

Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.

No reviews yet

People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.

  • It worked
  • Partly
  • It didn't
What did you use it for? e.g. Reviewing pull requests in a Rails app

Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.

Your trail

Files you open will collect here, so you can find them again.

Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.

Now do this for your own.

Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.