mlflow-traces
Skill23 starsChanged 39 days agoMITView on GitHub
Use when working with MLflow traces: debugging via MCP tools, analyzing performance, logging feedback, writing custom scorers/evaluations, or cleaning up trace data
In this repository30 other files it gives its agents.
- AGENTS.mdAGENTS.md
- src/AGENTS.mdAGENTS.md
- tests/AGENTS.mdAGENTS.md
- CLAUDE.mdCLAUDE.md
- av-eventsSkill
- creative-concept-designSkill
- educational-explainerSkill
- ffmpeg-productionSkill
- footage-editSkill
- gemini-visualizeSkill
- gr-advisorSkill
- hardware-evidence-captureSkill
- image-generationSkill
- movie-commentarySkill
- plugin-maintenanceSkill
- qwen-image-integrationSkill
- qwen-video-integrationSkill
- research-brief-builderSkill
- research-visualization-blenderSkill
- research-visualization-freecadSkill
- reverse-search-video-frameSkill
- spatial-video-analysisSkill
- tts-productionSkill
- video-explainerSkill
- video-generationSkill
- video-productionSkill
- video-researchSkill
- video-to-skillSkill
- video-translationSkill
- weaviate-setupSkill
The file
We split this skill into the rules it gives an agent
4 rules. About 0 are common, 2 are distinctive, and 2 are very specific to this project.
≈0
Common · estimate
Many files like this have a rule of this kind.
≈2
Distinctive · estimate
This author's own choices. Worth a look.
2
Very specific
Tied to their project's names, files or tools. Probably skip.
- Common
- Distinctive
- Very specific
- Distinctive: line 12
Core principle: Search first, then act. Always verify before destructive operations.
- Very specific: line 156
Check MLFLOW_TRACKING_URI is set in the server environment
- Very specific: line 157
Verify the experiment name: search with max_results: 1 across experiment IDs
- Distinctive: line 158
Use a separately authorized inference call if needed, then search its retained trace.
≈ “Common” is an estimate: at least 1 in 5 of the 350,495 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.
Files
MLflow Trace Management — video-research-mcp
Overview
Query, tag, evaluate, and manage MLflow traces captured from video-research-mcp Gemini API calls. Uses mcp__mlflow-mcp__* MCP tools — no code writing needed for most operations.
Core principle: Search first, then act. Always verify before destructive operations.
Quick Reference
| Task | Tool | Key Params |
| Find traces | search_traces | experiment_id, filter_string, extract_fields |
| Get details | get_trace | trace_id, extract_fields |
| Tag trace | set_trace_tag | trace_id, key, value |
| Log score | log_feedback | trace_id, name, value, rationale |
| Run scorers | evaluate_traces | experiment_id, trace_ids, scorers |
| List scorers | list_scorers | — |
Canonical Field Paths
CRITICAL — only use fields that actually exist:
| Path | Content | Common mistake |
info.trace_id | Trace identifier | — |
info.state | Status: OK, ERROR | NOT info.status |
info.request_time | Timestamp | NOT info.timestamp_ms |
info.execution_duration_ms | Duration in ms | NOT info.execution_duration |
info.request_preview | First ~100 chars of request | — |
info.response_preview | First ~100 chars of response | — |
info.tags | All tags as object | Use info.tags.* for all |
data.spans.*.name | Span names | Must include data. prefix |
data.spans.*.status_code | Span status | NOT data.spans.*.status |
data.spans.*.inputs | Span inputs | Moderate size |
data.spans.*.outputs | Span outputs | Moderate size |
extract_fields Discipline
Always use extract_fields. Video-research-mcp traces contain video URIs, cached content references, full Gemini prompts/responses. A single get_trace without extract_fields can flood your context window.
// BAD - pulls everything
get_trace({ trace_id: "tr-..." })
search_traces({ experiment_id: "2" })
// GOOD - selective fields
get_trace({ trace_id: "tr-...",
extract_fields: "info.*,data.spans.*.name,data.spans.*.status_code" })
search_traces({ experiment_id: "2", max_results: 10,
extract_fields: "info.trace_id,info.state,info.execution_duration_ms" })Never request data.spans.*.attributes unqualified — it silently drops dotted keys and can contain massive payloads.
Filter String vs Extract Fields — DIFFERENT NAMING!
CRITICAL: filter_string and extract_fields use DIFFERENT field names:
| Data | filter_string syntax | extract_fields syntax |
| Status | status = 'ERROR' | info.state |
| Timestamp | timestamp_ms > 170000... | info.request_time |
| Duration | execution_time_ms > 5000 | info.execution_duration_ms |
| Tags | tags.reviewed = 'true' | info.tags.* |
Common Workflows
Debug failed traces
search_traces({ experiment_id: "<id>", filter_string: "status='ERROR'", max_results: 20,
extract_fields: "info.trace_id,info.state,info.execution_duration_ms,info.request_preview" })
get_trace({ trace_id: "tr-abc123",
extract_fields: "info.*,data.spans.*.name,data.spans.*.status_code" })
set_trace_tag({ trace_id: "tr-abc123", key: "needs_investigation", value: "true" })Find slow traces
search_traces({ experiment_id: "<id>", filter_string: "execution_time_ms > 5000",
max_results: 20, extract_fields: "info.trace_id,info.execution_duration_ms,data.spans.*.name" })Log human feedback
log_feedback({ trace_id: "tr-abc123", name: "response_quality", value: 4.5,
source_type: "human", rationale: "Accurate analysis, good structure" })Run built-in scorers
// List available scorers first
list_scorers()
// Run evaluation
evaluate_traces({ experiment_id: "<id>", trace_ids: "tr-abc,tr-def",
scorers: "Correctness,RelevanceToQuery" })Search before delete
// Step 1: Preview
search_traces({ experiment_id: "<id>", filter_string: "timestamp_ms < 1704067200000",
max_results: 10, extract_fields: "info.trace_id,info.request_time" })
// Step 2: Verify count and IDs, then delete
delete_traces({ experiment_id: "<id>", max_timestamp_millis: 1704067200000 })Field Selection Recipes
"info.trace_id,info.state" // Minimal overview
"info.trace_id,info.execution_duration_ms,data.spans.*.name" // Performance
"info.*,data.spans.*.name,data.spans.*.status_code" // Full context (safe)
"info.trace_id,info.tags.*" // Tags only
"info.trace_id,info.assessments.*.feedback.value" // Feedback scoresvideo-research-mcp Context
| Setting | Value |
| Tracking server | Operator-selected URI; http://127.0.0.1:5001 is a loopback example |
| Experiment name | video-research-mcp |
| Env var | MLFLOW_TRACKING_URI |
| Autolog captures | All GeminiClient generate/generate_structured calls |
| Trace spans | Gemini API calls with model, thinking level, tokens, cost |
Tracing requires the installed tracing extra and a reachable MLFLOW_TRACKING_URI. GEMINI_TRACING_ENABLED=false disables it even with a URI. When enabled, mlflow.gemini.autolog() captures SDK calls; inspect actual retained traces before claiming coverage.
Troubleshooting
MCP tools not available / connection refused
The MLflow tracking server must be running:
MLFLOW_TRACKING_URI=http://127.0.0.1:5001 mlflow server --port 5001Then restart Claude Code to reconnect.
No traces found
- Check
MLFLOW_TRACKING_URIis set in the server environment - Verify the experiment name: search with
max_results: 1across experiment IDs - Use a separately authorized inference call if needed, then search its retained trace; do not send user material solely for diagnostics without authority
Wrong experiment
The default experiment is video-research-mcp. If traces land in Default (experiment 0), the MLFLOW_EXPERIMENT_NAME env var is not set.
Resources
Does it work?
Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.
No reviews yet
People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.
- It worked
- Partly
- It didn't
Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.
Your trail
Files you open will collect here, so you can find them again.
Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.
Now do this for your own.
Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.

