agentleFS
Galbaz1Galbaz1/video-research-mcp/skills/mlflow-traces/SKILL.md

mlflow-traces

Skill23 starsChanged 39 days agoMITView on GitHub

Use when working with MLflow traces: debugging via MCP tools, analyzing performance, logging feedback, writing custom scorers/evaluations, or cleaning up trace data

The file

We split this skill into the rules it gives an agent

4 rules. About 0 are common, 2 are distinctive, and 2 are very specific to this project.

  • ≈0

    Common · estimate

    Many files like this have a rule of this kind.

  • ≈2

    Distinctive · estimate

    This author's own choices. Worth a look.

  • 2

    Very specific

    Tied to their project's names, files or tools. Probably skip.

  • Common
  • Distinctive
  • Very specific
  1. Distinctive:

    Core principle: Search first, then act. Always verify before destructive operations.

    line 12
  2. Very specific:

    Check MLFLOW_TRACKING_URI is set in the server environment

    line 156
  3. Very specific:

    Verify the experiment name: search with max_results: 1 across experiment IDs

    line 157
  4. Distinctive:

    Use a separately authorized inference call if needed, then search its retained trace.

    line 158

≈ “Common” is an estimate: at least 1 in 5 of the 350,495 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.

Keep what fits and drop the rest. Your version, ready for your agents.
Files
SKILL.md

MLflow Trace Management — video-research-mcp

Overview

Query, tag, evaluate, and manage MLflow traces captured from video-research-mcp Gemini API calls. Uses mcp__mlflow-mcp__* MCP tools — no code writing needed for most operations.

Core principle: Search first, then act. Always verify before destructive operations.

Quick Reference

TaskToolKey Params
Find tracessearch_tracesexperiment_id, filter_string, extract_fields
Get detailsget_tracetrace_id, extract_fields
Tag traceset_trace_tagtrace_id, key, value
Log scorelog_feedbacktrace_id, name, value, rationale
Run scorersevaluate_tracesexperiment_id, trace_ids, scorers
List scorerslist_scorers—

Canonical Field Paths

CRITICAL — only use fields that actually exist:

PathContentCommon mistake
info.trace_idTrace identifier—
info.stateStatus: OK, ERRORNOT info.status
info.request_timeTimestampNOT info.timestamp_ms
info.execution_duration_msDuration in msNOT info.execution_duration
info.request_previewFirst ~100 chars of request—
info.response_previewFirst ~100 chars of response—
info.tagsAll tags as objectUse info.tags.* for all
data.spans.*.nameSpan namesMust include data. prefix
data.spans.*.status_codeSpan statusNOT data.spans.*.status
data.spans.*.inputsSpan inputsModerate size
data.spans.*.outputsSpan outputsModerate size

extract_fields Discipline

Always use extract_fields. Video-research-mcp traces contain video URIs, cached content references, full Gemini prompts/responses. A single get_trace without extract_fields can flood your context window.

// BAD - pulls everything
get_trace({ trace_id: "tr-..." })
search_traces({ experiment_id: "2" })

// GOOD - selective fields
get_trace({ trace_id: "tr-...",
  extract_fields: "info.*,data.spans.*.name,data.spans.*.status_code" })
search_traces({ experiment_id: "2", max_results: 10,
  extract_fields: "info.trace_id,info.state,info.execution_duration_ms" })

Never request data.spans.*.attributes unqualified — it silently drops dotted keys and can contain massive payloads.

Filter String vs Extract Fields — DIFFERENT NAMING!

CRITICAL: filter_string and extract_fields use DIFFERENT field names:

Datafilter_string syntaxextract_fields syntax
Statusstatus = 'ERROR'info.state
Timestamptimestamp_ms > 170000...info.request_time
Durationexecution_time_ms > 5000info.execution_duration_ms
Tagstags.reviewed = 'true'info.tags.*

Common Workflows

Debug failed traces

search_traces({ experiment_id: "<id>", filter_string: "status='ERROR'", max_results: 20,
  extract_fields: "info.trace_id,info.state,info.execution_duration_ms,info.request_preview" })

get_trace({ trace_id: "tr-abc123",
  extract_fields: "info.*,data.spans.*.name,data.spans.*.status_code" })

set_trace_tag({ trace_id: "tr-abc123", key: "needs_investigation", value: "true" })

Find slow traces

search_traces({ experiment_id: "<id>", filter_string: "execution_time_ms > 5000",
  max_results: 20, extract_fields: "info.trace_id,info.execution_duration_ms,data.spans.*.name" })

Log human feedback

log_feedback({ trace_id: "tr-abc123", name: "response_quality", value: 4.5,
  source_type: "human", rationale: "Accurate analysis, good structure" })

Run built-in scorers

// List available scorers first
list_scorers()

// Run evaluation
evaluate_traces({ experiment_id: "<id>", trace_ids: "tr-abc,tr-def",
  scorers: "Correctness,RelevanceToQuery" })

Search before delete

// Step 1: Preview
search_traces({ experiment_id: "<id>", filter_string: "timestamp_ms < 1704067200000",
  max_results: 10, extract_fields: "info.trace_id,info.request_time" })

// Step 2: Verify count and IDs, then delete
delete_traces({ experiment_id: "<id>", max_timestamp_millis: 1704067200000 })

Field Selection Recipes

"info.trace_id,info.state"                                    // Minimal overview
"info.trace_id,info.execution_duration_ms,data.spans.*.name"  // Performance
"info.*,data.spans.*.name,data.spans.*.status_code"           // Full context (safe)
"info.trace_id,info.tags.*"                                   // Tags only
"info.trace_id,info.assessments.*.feedback.value"             // Feedback scores

video-research-mcp Context

SettingValue
Tracking serverOperator-selected URI; http://127.0.0.1:5001 is a loopback example
Experiment namevideo-research-mcp
Env varMLFLOW_TRACKING_URI
Autolog capturesAll GeminiClient generate/generate_structured calls
Trace spansGemini API calls with model, thinking level, tokens, cost

Tracing requires the installed tracing extra and a reachable MLFLOW_TRACKING_URI. GEMINI_TRACING_ENABLED=false disables it even with a URI. When enabled, mlflow.gemini.autolog() captures SDK calls; inspect actual retained traces before claiming coverage.

Troubleshooting

MCP tools not available / connection refused

The MLflow tracking server must be running:

MLFLOW_TRACKING_URI=http://127.0.0.1:5001 mlflow server --port 5001

Then restart Claude Code to reconnect.

No traces found

  1. Check MLFLOW_TRACKING_URI is set in the server environment
  2. Verify the experiment name: search with max_results: 1 across experiment IDs
  3. Use a separately authorized inference call if needed, then search its retained trace; do not send user material solely for diagnostics without authority

Wrong experiment

The default experiment is video-research-mcp. If traces land in Default (experiment 0), the MLFLOW_EXPERIMENT_NAME env var is not set.

Resources

Download now (ZIP)

Does it work?

Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.

No reviews yet

People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.

  • It worked
  • Partly
  • It didn't
What did you use it for? e.g. Reviewing pull requests in a Rails app

Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.

Your trail

Files you open will collect here, so you can find them again.

Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.

Now do this for your own.

Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.