fabric-performance-monitoring
Skill17 starsChanged 9 months agoMITView on GitHub
Monitor and optimize Microsoft Fabric capacity, Spark compute, and workload performance. Use when asked to check capacity utilization, diagnose throttling (HTTP 430), monitor Spark VCore consumption, analyze CU usage, review Monitoring Hub jobs, query Fabric REST APIs for capacity health, generate performance reports, tune Spark resource profiles, investigate concurrency limits, or optimize Fabric SKU sizing. Supports PowerShell, T-SQL, and REST API workflows.
In this repository24 other files it gives its agents.
- fabric-data-agent-perf-remediateSkill
- fabric-data-agent-remediateSkill
- fabric-data-agentSkill
- fabric-data-factory-perf-remediateSkill
- fabric-dataflows-perf-remediationSkill
- fabric-delta-spark-perfSkill
- fabric-iqSkill
- fabric-lakehouse-access-controlSkill
- fabric-lakehouse-perf-remediateSkill
- fabric-lakehouse-views-perf-remediateSkill
- fabric-network-remediateSkill
- fabric-notebook-perf-remediateSkill
- fabric-onelake-perf-remediateSkill
- fabric-pandas-perf-remediateSkill
- fabric-pbi-perf-remediateSkill
- fabric-pbi-security-remediateSkill
- fabric-pyspark-perf-remediateSkill
- fabric-rest-api-perf-remediateSkill
- fabric-rest-api-remediateSkill
- fabric-rti-perf-remediateSkill
- fabric-spark-compute-perf-remediateSkill
- fabric-spark-compute-remediateSkill
- fabric-spark-perf-remediateSkill
- fabric-udf-perf-remediateSkill
The file
We split this skill into the rules it gives an agent
5 rules. About 0 are common, 4 are distinctive, and 1 is very specific to this project.
≈0
Common · estimate
Many files like this have a rule of this kind.
≈4
Distinctive · estimate
This author's own choices. Worth a look.
1
Very specific
Tied to their project's names, files or tools. Probably skip.
- Common
- Distinctive
- Very specific
- Very specific: line 43
When capacity is fully utilized, new Spark jobs receive HTTP 430 with TooManyRequestsForCapacity.
- Distinctive: line 67
Run the capacity health check script to retrieve current capacity status, SKU details, and state.
- Distinctive: line 77
Run the Spark concurrency analyzer to check active sessions, queued jobs, and throttling status.
- Distinctive: line 85
Run the job audit script to retrieve recent job executions, durations, and failure details.
- Distinctive: line 93
Use the performance report template to query the SQL analytics endpoint for Lakehouse operation metrics, then generate a summary with the …
≈ “Common” is an estimate: at least 1 in 5 of the 364,980 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.
Files
Microsoft Fabric Performance Monitoring
Toolkit for monitoring, diagnosing, and optimizing Microsoft Fabric capacity and workload performance across Spark, Data Warehouse, Lakehouse, and Pipeline workloads.
When to Use This Skill
- Checking Fabric capacity utilization or CU consumption
- Diagnosing throttling errors (HTTP 430 / TooManyRequestsForCapacity)
- Monitoring Spark VCore usage and concurrency limits
- Querying Fabric REST APIs for capacity and workspace health
- Generating capacity performance reports
- Tuning Spark resource profiles (readHeavy, writeHeavy, balanced)
- Investigating job failures in the Monitoring Hub
- Analyzing autoscale billing vs capacity-based billing
- Reviewing background vs interactive operation patterns
- Planning capacity SKU sizing or rightsizing
Prerequisites
- PowerShell 7+ with Az.Fabric module installed
- Microsoft Entra ID app registration with Fabric API permissions
- Fabric Capacity Admin or Workspace Admin role
- Fabric Capacity Metrics app installed (for visual monitoring)
Core Concepts
Capacity Units and Spark VCores
One Capacity Unit (CU) equals two Apache Spark VCores. Fabric capacity is shared across all workspaces assigned to it, and Spark VCores are shared among notebooks, Spark job definitions, and lakehouses within those workspaces.
Operation Types
Fabric classifies operations as interactive (on-demand, like DAX queries) or background (scheduled, like refreshes and Spark jobs). Background operations are smoothed over a 24-hour period. All Spark operations are background operations.
Throttling Behavior
When capacity is fully utilized, new Spark jobs receive HTTP 430 with TooManyRequestsForCapacity. With queueing enabled, pipeline-triggered and scheduled jobs enter a FIFO queue and retry automatically when capacity becomes available.
Capacity SKU Limits
| SKU | Spark VCores | Queue Limit |
| F2 | 4 | 4 |
| F4 | 8 | 4 |
| F8 | 16 | 8 |
| F16 | 32 | 16 |
| F32 | 64 | 32 |
| F64 | 128 | 64 |
| F128 | 256 | 128 |
| F256 | 512 | 256 |
| F512 | 1024 | 512 |
Spark Resource Profiles
Fabric supports predefined Spark resource profiles for workload optimization. New workspaces default to writeHeavy. Available profiles: readHeavy, writeHeavy, balanced. When writeHeavy is used, VOrder is disabled by default and must be manually enabled.
Step-by-Step Workflows
Workflow 1: Capacity Health Check
Run the capacity health check script to retrieve current capacity status, SKU details, and state.
./scripts/Get-FabricCapacityHealth.ps1 -SubscriptionId "<sub-id>" -ResourceGroupName "<rg>" -CapacityName "<name>"See capacity-health-reference.md for detailed API response schemas and interpretation guidance.
Workflow 2: Spark Concurrency Analysis
Run the Spark concurrency analyzer to check active sessions, queued jobs, and throttling status.
./scripts/Get-FabricSparkConcurrency.ps1 -WorkspaceId "<workspace-id>"Workflow 3: Monitoring Hub Job Audit
Run the job audit script to retrieve recent job executions, durations, and failure details.
./scripts/Get-FabricJobHistory.ps1 -WorkspaceId "<workspace-id>" -HoursBack 24Workflow 4: Generate Performance Report
Use the performance report template to query the SQL analytics endpoint for Lakehouse operation metrics, then generate a summary with the report generator.
Workflow 5: Autoscale vs Capacity Cost Analysis
See cost-analysis-reference.md for guidance on comparing autoscale billing vs capacity-based models using Azure Cost Management.
remediate
| Symptom | Likely Cause | Resolution |
| HTTP 430 errors | Capacity fully utilized | Scale SKU, cancel idle sessions, enable queueing |
| Jobs stuck in queue | All VCores consumed | Check Monitoring Hub, stop idle notebooks |
| Slow Spark startup | Using custom pool with cold start | Switch to starter pool for quick sessions |
| High CU consumption | Inefficient queries or unoptimized code | Review Capacity Metrics app, optimize DAX/Spark |
| Autoscale charges unexpected | Spark jobs billed independently | Check Azure Cost Analysis with Autoscale meter |
| VOrder disabled | writeHeavy profile active | Manually enable VOrder if read performance needed |
References
- Capacity Health Reference - REST API schemas and interpretation
- Cost Analysis Reference - Autoscale vs capacity billing comparison
- Fabric Capacity Metrics App
- Monitor Spark Capacity Consumption
- Fabric REST API Documentation
- Concurrency Limits and Queueing
Does it work?
Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.
No reviews yet
People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.
- It worked
- Partly
- It didn't
Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.
Your trail
Files you open will collect here, so you can find them again.
Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.
Now do this for your own.
Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.

