agentleFS
NVIDIAKnown publisherNVIDIA/TensorRT-Model-Optimizer/skills/modelopt-accessing-mlflow/SKILL.md

modelopt-accessing-mlflow

Skill5.6k starsChanged todayApache-2.0View on GitHub

Query and browse evaluation results stored in MLflow. Use when the user wants to look up runs by invocation ID, compare metrics across models, fetch artifacts (configs, logs, results), or set up the MLflow MCP server. ALWAYS triggers on mentions of MLflow, experiment results, run comparison, invocation IDs in the context of results, or MLflow MCP setup.

  • Pipes a download into a shell

1 other project uses a version of this.

Adopted by

Steady: 0 new projects took it in the last 4 weeks, none the 4 weeks before.thin data

Week of Sep 20: 0 new projectsWeek of Sep 27: 0 new projectsWeek of Oct 4: 0 new projectsWeek of Oct 11: 0 new projectsWeek of Oct 18: 0 new projectsWeek of Oct 25: 0 new projectsWeek of Nov 1: 0 new projectsWeek of Nov 8: 0 new projects
Sep 2026Oct 2026Nov 2026
New projects taking a version of this file each week, over the 8 weeks our latest run read: only 0 in all, so this is a hint, not a trend.

Most recently taken up by

Who's talking about NVIDIA/TensorRT-Model-Optimizer

The biggest 5 of 45 mentions linking to the repository that our crawl found. Each line is their own headline.

See all 45

Talked about on GitHub (45 times).

On GitHub · 45

40 more on GitHub

The file

We split this skill into the rules it gives an agent

2 rules. About 0 are common, 2 are distinctive, and 0 are very specific to this project.

  • ≈0

    Common · estimate

    Many files like this have a rule of this kind.

  • ≈2

    Distinctive · estimate

    This author's own choices. Worth a look.

  • 0

    Very specific

    Tied to their project's names, files or tools. Probably skip.

  • Common
  • Distinctive
  • Very specific
  1. Distinctive:

    When comparing metrics across runs, fetch the data via MCP, then run the computation in Python for exact results rather than doing math …

    line 43
  2. Distinctive:

    If the MLflow MCP server fails to load or its tools are unavailable:

    line 72

≈ “Common” is an estimate: at least 1 in 5 of the 364,980 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.

Keep what fits and drop the rest. Your version, ready for your agents.
Files
SKILL.md

Accessing MLflow

MCP Server

mlflow-mcp gives agents direct access to MLflow — query runs, compare metrics, browse artifacts, all through natural language.

ID Convention

When the user provides a hex ID (e.g. 71f3f3199ea5e1f0) without specifying what it is, assume it is an invocation_id (not an MLflow run_id). An invocation_id identifies a launcher invocation and is stored as both a tag and a param on MLflow runs. One invocation can produce multiple MLflow runs (one per task). You may need to search across multiple experiments if you don't know which experiment the run belongs to.

Querying Runs

# Find runs by invocation_id
MLflow:search_runs_by_tags(experiment_id, {"invocation_id": "<invocation_id>"})

# Query for example model/task runs
MLflow:query_runs(experiment_id, "tags.model LIKE '%<model>%'")
MLflow:query_runs(experiment_id, "tags.task_name LIKE '%<task_name>%'")

# Get a config from run's artifacts
MLflow:get_artifact_content(run_id, "config.yml")

# Get nested stats from run's artifacts
MLflow:get_artifact_content(run_id, "artifacts/eval_factory_metrics.json")

NOTE: You WILL NOT find PENDING, RUNNING, KILLED, or FAILED runs in MLflow! Only SUCCESSFUL runs are exported to MLflow.

Workflow Tips

When comparing metrics across runs, fetch the data via MCP, then run the computation in Python for exact results rather than doing math in-context:

uv run --with pandas python3 << 'EOF'
import pandas as pd
# ... compute deltas, averages, etc.
EOF

Artifacts Structure

<harness>.<task>/
├── artifacts/
│   ├── config.yml                # Fully resolved config used during the evaluation
│   ├── launcher_unresolved_config.yaml # Unresolved config passed to the launcher
│   ├── results.yml               # All results in YAML format
│   ├── eval_factory_metrics.json # Runtime stats (latency, tokens count, memory)
│   ├── report.html               # Request-Response Pairs samples in HTML format (if enabled)
│   └── report.json               # Request-Response Pairs samples in JSON format (if enabled)
└── logs/
    ├── client-*.log              # Evaluation client
    ├── server-*-N.log            # Deployment per node
    ├── slurm-*.log               # Slurm job
    └── proxy-*.log               # Request proxy

Troubleshooting

If the MLflow MCP server fails to load or its tools are unavailable:

  1. uvx not found — install uv:
    curl -LsSf https://astral.sh/uv/install.sh | sh
  2. MCP server not configured — add the config and restart the agent:

    For Claude Code — add to .claude/settings.json (project or user level), under "mcpServers":

    "MLflow": {
      "command": "uvx",
      "args": ["mlflow-mcp"],
      "env": {
        "MLFLOW_TRACKING_URI": "https://<your-mlflow-server>/"
      }
    }

    For Cursor — edit ~/.cursor/mcp.json (Settings > Tools & MCP > New MCP Server):

    {
      "mcpServers": {
        "MLflow": {
          "command": "uvx",
          "args": ["mlflow-mcp"],
          "env": {
            "MLFLOW_TRACKING_URI": "https://<your-mlflow-server>/"
          }
        }
      }
    }

Download now (ZIP)

Does it work?

Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.

No reviews yet

People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.

  • It worked
  • Partly
  • It didn't
What did you use it for? e.g. Reviewing pull requests in a Rails app

Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.

Your trail

Files you open will collect here, so you can find them again.

Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.

Now do this for your own.

Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.