agentleFS
PatrickGallucciPatrickGallucci/fabric-skills/skills/fabric-data-agent-perf-remediate/SKILL.md

fabric-data-agent-perf-remediate

Skill17 starsChanged 9 months agoMITView on GitHub

Diagnose and resolve Microsoft Fabric Data Agent performance issues including slow query generation, capacity throttling (HTTP 430), Spark session startup delays, KQL/SQL/DAX query timeouts, data source misconfiguration, example query validation failures, resource profile tuning, VOrder optimization, autotune settings, and Lakehouse table maintenance. Use when asked to troubleshoot Fabric Data Agent response times, fix agent query accuracy, debug Operations Agent playbook performance, resolve capacity SKU limits, optimize Spark compute for agents, or diagnose Data Agent data source connection issues.

The file

Files
SKILL.md

Fabric Data Agent Performance remediate

Systematic toolkit for diagnosing and resolving performance issues in Microsoft Fabric Data Agents, Operations Agents, and their underlying Spark compute and data source infrastructure.

When to Use This Skill

  • Data Agent responses are slow or timing out
  • Agent-generated SQL/KQL/DAX queries return errors or produce incorrect results
  • Spark session startup takes longer than expected (>10 seconds)
  • Capacity throttling errors (HTTP 430) during agent workloads
  • Operations Agent playbooks fail to execute or trigger actions
  • Data source connections are unreliable or stale
  • Example queries fail validation against schema
  • Resource profiles need tuning for agent-specific workloads
  • Table maintenance (VOrder, bin-compaction, vacuum) is overdue
  • Autoscale Billing configuration needs optimization

Prerequisites

  • Access: Fabric workspace Contributor or Admin role
  • Tools: PowerShell 7+, Fabric REST API access (Microsoft Entra token)
  • Endpoints: https://api.fabric.microsoft.com/v1/workspaces/{workspaceId}/items
  • Monitoring: Access to Fabric Admin Portal and Azure Cost Analysis
  • Optional: Azure subscription admin for SKU resizing and Autoscale Billing

Diagnostic Decision Tree

Agent performance issue reported
├── Slow response times?
│   ├── First query after idle? → Spark session startup (see Workflow 1)
│   ├── All queries slow? → Resource profile mismatch (see Workflow 2)
│   └── Intermittent slowness? → Capacity throttling (see Workflow 3)
├── Incorrect query results?
│   ├── Wrong tables/columns? → Data source instructions (see Workflow 4)
│   ├── SQL syntax errors? → Example query validation (see Workflow 5)
│   └── Stale data? → Lakehouse table maintenance (see Workflow 6)
├── Agent not responding?
│   ├── HTTP 430 errors? → Concurrency limits (see Workflow 3)
│   ├── Connection failures? → Data source config (see Workflow 4)
│   └── Operations Agent stuck? → Playbook/action config (see Workflow 7)
└── Cost concerns?
    ├── Unexpected charges? → Autoscale Billing audit (see Workflow 8)
    └── Over-provisioned? → SKU right-sizing (see Workflow 8)

Step-by-Step Workflows

Workflow 1: Spark Session Startup Delays

Symptoms: First query takes 2-5 minutes, subsequent queries are fast.

ScenarioExpected Startup
Default, no custom libraries5-10 seconds
Default + library dependencies35 seconds - 5 minutes
High regional traffic2-5 minutes
Private Links / Managed VNet2-5 minutes
Network security + libraries2.5-10 minutes

Resolution steps: See workflow-spark-startup.md

Workflow 2: Resource Profile Optimization

Symptoms: Consistently slow reads or writes across all agent queries.

ProfileUse CaseVOrderOptimizeWrite
writeHeavyHigh-frequency ingestion (default)DisabledDisabled
readHeavyForPBIPower BI dashboard queriesEnabledEnabled (1GB bin)
readHeavyForSparkInteractive Spark analyticsDisabledEnabled (128MB bin)
customUser-defined workload tuningConfigurableConfigurable

Resolution steps: See workflow-resource-profiles.md

Workflow 3: Capacity Throttling (HTTP 430)

Symptoms: TooManyRequestsForCapacity errors, jobs queued or rejected.

SKUSpark VCoresQueue Limit
F244
F8168
F6412864
F128256128
F256512256

Formula: 1 Capacity Unit = 2 Spark VCores

Resolution steps: See workflow-capacity-throttling.md

Workflow 4: Data Source Configuration Issues

Symptoms: Agent queries wrong tables, returns irrelevant results, or fails to connect.

Data Agents use three configuration layers that all affect query quality:

  1. Agent Instructions — Global routing rules (which data source for which topic)
  2. Data Source Instructions — Schema context, table descriptions, column details
  3. Example Queries — Few-shot SQL/KQL/DAX examples for query generation

Resolution steps: See workflow-data-source-config.md

Workflow 5: Example Query Validation

Symptoms: Agent ignores example queries, generates incorrect SQL/KQL syntax.

Key rule: The Fabric Data Agent only uses queries that contain valid SQL/KQL syntax AND match the schema of the selected tables. Queries that fail validation are silently ignored.

Resolution steps: See workflow-example-queries.md

Workflow 6: Lakehouse Table Maintenance

Symptoms: Queries over Delta tables are slow, small file problem, stale statistics.

Three maintenance operations available via REST API:

OperationPurpose
Bin-compactionConsolidate small files into optimal sizes
V-OrderOptimize Parquet layout for read performance
VacuumRemove unreferenced files older than retention

Resolution steps: See workflow-table-maintenance.md

Workflow 7: Operations Agent Debugging

Symptoms: Operations Agent not triggering actions, playbook failures.

Operations Agent definition requires: goals, instructions, at least one KustoDatabase data source, at least one PowerAutomateAction, and shouldRun: true.

Resolution steps: See workflow-operations-agent.md

Workflow 8: Autoscale Billing and SKU Right-Sizing

Symptoms: Unexpected costs, capacity contention between Spark and other workloads.

Resolution steps: See workflow-autoscale-billing.md

Available Scripts

ScriptPurpose
Get-FabricAgentDiagnostics.ps1Collect agent config, capacity state, and Spark metrics
Test-ExampleQueries.ps1Validate example queries against data source schema
Invoke-TableMaintenance.ps1Run bin-compaction, V-Order, and vacuum on Lakehouse tables

remediate Quick Reference

ErrorLikely CauseFirst Action
HTTP 430Capacity VCore limitCheck Monitoring Hub for active jobs
Query timeoutResource profile mismatchSwitch to readHeavyForSpark
Wrong columnsMissing data source instructionsUpdate schema descriptions
Ignored examplesInvalid SQL/KQL syntaxValidate with Test-ExampleQueries.ps1
2-5 min startupPrivate Links or high trafficCheck workspace networking config
Stale resultsMissing table maintenanceRun bin-compaction + V-Order
Agent not runningshouldRun: falseCheck Operations Agent definition
Autotune disabledRuntime > 1.2 or HC modeVerify Fabric Runtime version

References

  • Spark Startup Workflow
  • Resource Profile Workflow
  • Capacity Throttling Workflow
  • Data Source Config Workflow
  • Example Query Workflow
  • Table Maintenance Workflow
  • Operations Agent Workflow
  • Autoscale Billing Workflow

Download now (ZIP)

Does it work?

Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.

No reviews yet

People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.

  • It worked
  • Partly
  • It didn't
What did you use it for? e.g. Reviewing pull requests in a Rails app

Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.

Your trail

Files you open will collect here, so you can find them again.

Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.

Now do this for your own.

Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.