eventstream-consumption-cli
Skill3 starsChanged 4 months agoMITView on GitHub
List, inspect, and monitor Fabric Eventstream real-time ingestion pipelines via the Items REST API. Discover Eventstreams across workspaces, decode base64 graph topologies tracing event flow from source through operators to destination nodes. Validate connection IDs, wiring, retention policies (1-90 days), and throughput levels. Retrieve Custom Endpoint Kafka credentials via Topology API. Use to: (1) list Eventstreams, (2) inspect Eventstream topology showing sources and destinations, (3) validate Eventstream configurations, (4) check Eventstream retention policy and throughput level, (5) get connection strings. Triggers: "list eventstreams", "inspect eventstream", "inspect eventstream topology", "eventstream sources and destinations", "eventstream health", "eventstream configuration", "eventstream retention", "eventstream retention policy", "eventstream throughput level", "eventstream connection string", "custom endpoint credentials", "kafka connection", "check eventstream".
1 other project uses a version of this, 1 of them edited.
Adopted by
Most recently taken up by
- kimtth/ms-fabric-skills-dev-starteredited, first seen May 18, 2026
Other versions1 project uses a version of this.
Word for word · 0
None found.
Edited · 1
- kimtth/ms-fabric-skills-dev-starter★ 289% same
Wrote their own version · 0
None found.
How matches are found. Word for word: the exact same text. Edited: shares passages with this file (“99% same” means 99% of its five-word phrases also appear here; 10% or more counts). Their own version: similar meaning, or, weakest, the same name. We can’t tell which project had it first. Projects are counted among the 200 most-starred candidates, and mentions are links to the repository we found, up to 50 per site, so both counts are a floor.
In this repository36 other files it gives its agents.
- AGENTS.mdAGENTS.md
- CLAUDE.mdCLAUDE.md
- activator-authoring-cliSkill
- activator-consumption-cliSkill
- check-updatesSkill
- databricks-migrationSkill
- dataflows-authoring-cliSkill
- dataflows-consumption-cliSkill
- dataflows-save-as-authoring-cliSkill
- e2e-medallion-architectureSkill
- eventhouse-authoring-cliSkill
- eventhouse-consumption-cliSkill
- eventstream-authoring-cliSkill
- fabriciq-ontology-authoring-cliSkill
- fabriciq-ontology-consumption-cliSkill
- fabriciqSkill
- hdinsight-migrationSkill
- mlv-operations-cliSkill
- pipeline-migrationSkill
- powerbi-report-authoringSkill
- powerbi-report-designSkill
- powerbi-report-managementSkill
- powerbi-report-planningSkill
- search-consumption-cliSkill
- semantic-model-authoringSkill
- semantic-model-consumptionSkill
- spark-authoring-cliSkill
- spark-consumption-cliSkill
- spark-operations-cliSkill
- sqldb-authoring-cliSkill
- sqldb-consumption-cliSkill
- sqldb-operations-cliSkill
- sqldw-authoring-cliSkill
- sqldw-consumption-cliSkill
- sqldw-operations-cliSkill
- synapse-migrationSkill
The file
We split this skill into the rules it gives an agent
9 rules. About 0 are common, 7 are distinctive, and 2 are very specific to this project.
≈0
Common · estimate
Many files like this have a rule of this kind.
≈7
Distinctive · estimate
This author's own choices. Worth a look.
2
Very specific
Tied to their project's names, files or tools. Probably skip.
- Common
- Distinctive
- Very specific
- Distinctive: line 134
After decoding, count and list each node type:
- Distinctive: line 216
Use the response to configure a Kafka producer:
- Distinctive: line 233
Check key configuration aspects of a decoded Eventstream topology:
- Very specific: line 276
Use jq (bash) or ConvertFrom-Json (PowerShell) for parsing
- Distinctive: line 282
Do NOT confuse Eventstream with Eventhouse — they are separate Fabric workloads
- Distinctive: line 283
Do NOT hardcode workspace or item IDs — always discover them via the API
- Distinctive: line 284
Do NOT assume all source types appear in API enums — preview sources exist only in the UI
- Very specific: line 285
Do NOT modify Eventstream topology with this consumption skill — use eventstream-authoring-cli for writes
- Distinctive: line 286
Do NOT attempt to query event data through the Eventstream API — use downstream skills (eventhouse-consumption-cli, sqldw-consumption-cli) …
≈ “Common” is an estimate: at least 1 in 5 of the 364,980 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.
Files
Update Check — ONCE PER SESSION (mandatory) The first time this skill is used in a session, run the check-updates skill before proceeding.
- GitHub Copilot CLI / VS Code: invoke the
check-updatesskill.- Claude Code / Cowork / Cursor / Windsurf / Codex: compare local vs remote package.json version.
- Skip if the check was already performed earlier in this session.
CRITICAL NOTES
- To find the workspace details (including its ID) from workspace name: list all workspaces and, then, use JMESPath filtering
- To find the item details (including its ID) from workspace ID, item type, and item name: list all items of that type in that workspace and, then, use JMESPath filtering
- Eventstream ≠ Eventhouse. Eventstream is a real-time event ingestion and routing pipeline. For KQL queries, use
eventhouse-consumption-cli.
Eventstream Consumption — CLI Skill
Table of Contents
| Task | Reference | Notes |
| Finding Workspaces and Items in Fabric | COMMON-CLI.md § Finding Workspaces and Items in Fabric | Mandatory — READ link first [needed for finding workspace id by its name or item id by its name, item type, and workspace id] |
| Fabric Topology & Key Concepts | COMMON-CORE.md § Fabric Topology & Key Concepts | |
| Environment URLs | COMMON-CORE.md § Environment URLs | |
| Authentication & Token Acquisition | COMMON-CORE.md § Authentication & Token Acquisition | Wrong audience = 401; read before any auth issue |
| Core Control-Plane REST APIs | COMMON-CORE.md § Core Control-Plane REST APIs | Includes pagination, LRO polling, and rate-limiting patterns |
| Gotchas, Best Practices & Troubleshooting | COMMON-CORE.md § Gotchas, Best Practices & Troubleshooting | |
| Tool Selection Rationale | COMMON-CLI.md § Tool Selection Rationale | |
| Authentication Recipes | COMMON-CLI.md § Authentication Recipes | az login flows and token acquisition |
Fabric Control-Plane API via az rest | COMMON-CLI.md § Fabric Control-Plane API via az rest | Always pass --resource; includes pagination and LRO helpers |
| Gotchas & Troubleshooting (CLI-Specific) | COMMON-CLI.md § Gotchas & Troubleshooting (CLI-Specific) | az rest audience, shell escaping, token expiry |
| Quick Reference | COMMON-CLI.md § Quick Reference | az rest template + token audience/tool matrix |
| Listing and Discovering Eventstreams | EVENTSTREAM-CONSUMPTION-CORE.md § Listing and Discovering Eventstreams | List, Get, Search across workspaces |
| Inspecting Eventstream Topology | EVENTSTREAM-CONSUMPTION-CORE.md § Inspecting Eventstream Topology | Decode base64 definition → trace graph flow |
| Monitoring Eventstream Health | EVENTSTREAM-CONSUMPTION-CORE.md § Monitoring Eventstream Health | Retention and throughput checks |
| Source and Destination Status | EVENTSTREAM-CONSUMPTION-CORE.md § Source and Destination Status | Validation checklist for sources and destinations |
| Integration with Downstream Analytics | EVENTSTREAM-CONSUMPTION-CORE.md § Integration with Downstream Analytics | Eventhouse, Lakehouse, Activator, Real-Time Hub |
| Gotchas and Troubleshooting Reference | EVENTSTREAM-CONSUMPTION-CORE.md § Gotchas and Troubleshooting Reference | 10 common issues with causes and fixes |
| List Eventstreams | SKILL.md § List Eventstreams | |
| Inspect Eventstream Topology | SKILL.md § Inspect Eventstream Topology | Decode and explore the graph |
| Get Custom Endpoint Connection String | SKILL.md § Get Custom Endpoint Connection String | Retrieve Kafka/EH connection via Topology API |
| Validate Eventstream Configuration | SKILL.md § Validate Eventstream Configuration | |
| Gotchas, Rules, Troubleshooting | SKILL.md § Gotchas, Rules, Troubleshooting | MUST DO / AVOID / PREFER checklists |
List Eventstreams
List All Eventstreams in a Workspace
az rest --method GET \
--url "https://api.fabric.microsoft.com/v1/workspaces/${WORKSPACE_ID}/eventstreams" \
--resource "https://api.fabric.microsoft.com"Returns an array of Eventstream items. Use JMESPath to filter by name:
az rest --method GET \
--url "https://api.fabric.microsoft.com/v1/workspaces/${WORKSPACE_ID}/eventstreams" \
--resource "https://api.fabric.microsoft.com" \
--query "value[?displayName=='my-eventstream']"Get Eventstream Details
az rest --method GET \
--url "https://api.fabric.microsoft.com/v1/workspaces/${WORKSPACE_ID}/eventstreams/${EVENTSTREAM_ID}" \
--resource "https://api.fabric.microsoft.com"Inspect Eventstream Topology
Tip: The Topology API (
GET .../eventstreams/{id}/topology) returns runtime status, error info, and node IDs without base64 decoding. Prefer it for operational inspection (health checks, connection retrieval). UsePOST .../getDefinition(below) when you need the full authoring-time graph structure for topology modification.
Retrieve the Eventstream definition and decode it to inspect the full graph topology.
Step 1: Get the Definition
API Note: The Eventstream Definition API uses
POST .../getDefinition, notGET .../definition. This follows the Fabric Items Definition pattern. See official docs.
az rest --method POST \
--url "https://api.fabric.microsoft.com/v1/workspaces/${WORKSPACE_ID}/eventstreams/${EVENTSTREAM_ID}/getDefinition" \
--resource "https://api.fabric.microsoft.com" \
--body '{}'Step 2: Decode the Topology
Extract the eventstream.json part's payload field and base64-decode it:
# Using jq + base64 (Linux; on macOS use base64 -D instead of -d)
az rest --method POST \
--url "https://api.fabric.microsoft.com/v1/workspaces/${WORKSPACE_ID}/eventstreams/${EVENTSTREAM_ID}/getDefinition" \
--resource "https://api.fabric.microsoft.com" \
--body '{}' \
| jq -r '.definition.parts[] | select(.path=="eventstream.json") | .payload' \
| base64 -d | jq .# PowerShell (Windows)
$def = az rest --method POST `
--url "https://api.fabric.microsoft.com/v1/workspaces/$WORKSPACE_ID/eventstreams/$EVENTSTREAM_ID/getDefinition" `
--resource "https://api.fabric.microsoft.com" `
--body '{}' | ConvertFrom-Json
$payload = ($def.definition.parts | Where-Object { $_.path -eq 'eventstream.json' }).payload
[Text.Encoding]::UTF8.GetString([Convert]::FromBase64String($payload)) | ConvertFrom-Json | ConvertTo-Json -Depth 10Step 3: Summarize the Topology
After decoding, count and list each node type:
| Metric | Path in decoded JSON |
| Sources | .sources[] | .name, .type |
| Destinations | .destinations[] | .name, .type |
| Operators | .operators[] | .name, .type |
| Streams | .streams[] | .name, .type |
Get Custom Endpoint Connection String
The POST .../getDefinition endpoint returns empty properties for Custom Endpoint sources. To retrieve the Kafka/Event Hub connection info, use the Topology API /connection endpoint.
Important: This endpoint requires
Eventstream.ReadWrite.Allpermission scope (not just Read).
Step 1: Get the Topology to Find the Source ID
az rest --method GET \
--url "https://api.fabric.microsoft.com/v1/workspaces/${WORKSPACE_ID}/eventstreams/${EVENTSTREAM_ID}/topology" \
--resource "https://api.fabric.microsoft.com"From the response, find the Custom Endpoint source node and extract its id:
# Extract the sourceId for a Custom Endpoint source (use name filter if multiple exist)
SOURCE_ID=$(az rest --method GET \
--url "https://api.fabric.microsoft.com/v1/workspaces/${WORKSPACE_ID}/eventstreams/${EVENTSTREAM_ID}/topology" \
--resource "https://api.fabric.microsoft.com" \
| jq -r '[.sources[] | select(.type=="CustomEndpoint")] | if length == 0 then error("No Custom Endpoint sources found in this Eventstream") elif length > 1 then error("Multiple Custom Endpoint sources found — filter by .name") else .[0].id end') \
|| { echo "Failed to resolve Custom Endpoint source ID"; exit 1; }
if [ -z "$SOURCE_ID" ]; then echo "SOURCE_ID is empty — check topology output"; exit 1; fi# PowerShell — extract sourceId for Custom Endpoint (fails clearly if multiple exist)
$topology = az rest --method GET `
--url "https://api.fabric.microsoft.com/v1/workspaces/$WORKSPACE_ID/eventstreams/$EVENTSTREAM_ID/topology" `
--resource "https://api.fabric.microsoft.com" | ConvertFrom-Json
$customSources = @($topology.sources | Where-Object { $_.type -eq 'CustomEndpoint' })
if ($customSources.Count -eq 0) { throw "No Custom Endpoint sources found in this Eventstream" }
if ($customSources.Count -gt 1) { throw "Multiple Custom Endpoint sources found. Filter by name: $($customSources.name -join ', ')" }
$sourceId = $customSources[0].idStep 2: Get the Connection Details
⚠️ Security: This endpoint returns access keys and connection strings. Get explicit user confirmation before calling it. Redact
primaryKey,secondaryKey,primaryConnectionString, andsecondaryConnectionStringfrom any displayed output unless the user explicitly asks for secret values in a secure context. Avoid logging raw credentials; store securely and rotate as needed.
az rest --method GET \
--url "https://api.fabric.microsoft.com/v1/workspaces/${WORKSPACE_ID}/eventstreams/${EVENTSTREAM_ID}/sources/${SOURCE_ID}/connection" \
--resource "https://api.fabric.microsoft.com"az rest --method GET `
--url "https://api.fabric.microsoft.com/v1/workspaces/$WORKSPACE_ID/eventstreams/$EVENTSTREAM_ID/sources/$sourceId/connection" `
--resource "https://api.fabric.microsoft.com" | ConvertFrom-JsonExpected Response
{
"fullyQualifiedNamespace": "namespace.servicebus.windows.net",
"eventHubName": "es_<guid>",
"accessKeys": {
"primaryKey": "...",
"secondaryKey": "...",
"primaryConnectionString": "Endpoint=sb://namespace.servicebus.windows.net/;...",
"secondaryConnectionString": "..."
}
}Kafka Producer Configuration
Use the response to configure a Kafka producer:
| Setting | Value |
bootstrap_servers | {fullyQualifiedNamespace}:9093 |
topic | {eventHubName} |
security_protocol | SASL_SSL |
sasl_mechanism | PLAIN |
sasl_plain_username | $ConnectionString (fixed literal — not a variable) |
sasl_plain_password | {primaryConnectionString} |
Limitation: The
/connectionendpoint is only supported for Custom Endpoint sources (returns Kafka/Event Hub credentials). Other source types (Event Hub, IoT Hub, etc.) store their connection configuration (e.g.,dataConnectionId,consumerGroup) directly in the decoded definition properties.
Validate Eventstream Configuration
Check key configuration aspects of a decoded Eventstream topology:
Source Validation Checklist
| Check | How |
| Source type is API-supported | Compare against 25 known type enums |
| Cloud connection exists | Verify dataConnectionId GUID resolves |
| Consumer group set | Required for Event Hub, IoT Hub, Kafka sources |
| Serialization matches source | inputSerialization.type = Json, Csv, or Avro |
Destination Validation Checklist
| Check | How |
| Destination type is valid | Must be Lakehouse, Eventhouse, Activator, or CustomEndpoint |
| Target item accessible | Verify workspaceId + itemId resolve via GET |
| Input wired | inputNodes array must not be empty |
| Eventhouse direct ingestion | connectionName and mappingRuleName set |
EventstreamProperties Validation
Decode eventstreamProperties.json and check:
retentionTimeInDaysis within 1–90eventThroughputLevelisLow,Medium, orHigh
Gotchas, Rules, Troubleshooting
MUST DO
- Always pass
--resource https://api.fabric.microsoft.comwithaz restcalls - Always use JMESPath filtering to resolve workspace name → ID and item name → ID
- Always base64-decode the definition payload before inspecting topology (not needed for the Topology API — that returns JSON directly)
- For Custom Endpoint connection details, use the Topology API —
POST .../getDefinitionreturns empty properties; callGET .../topologyto get the sourceId, thenGET .../sources/{sourceId}/connection - Use POST for definition endpoints —
POST .../getDefinition(not GET),POST .../updateDefinition(not PUT). See official docs. - Handle pagination — check for
continuationUriin list responses - Poll LRO responses — Get Definition may return
202 Accepted
PREFER
- Decode topology JSON into structured output for readable summaries
- Use
jq(bash) orConvertFrom-Json(PowerShell) for parsing - Validate configurations before reporting issues to users
- Cross-reference destinations with downstream skills (eventhouse, sqldw, spark)
AVOID
- Do NOT confuse Eventstream with Eventhouse — they are separate Fabric workloads
- Do NOT hardcode workspace or item IDs — always discover them via the API
- Do NOT assume all source types appear in API enums — preview sources exist only in the UI
- Do NOT modify Eventstream topology with this consumption skill — use
eventstream-authoring-clifor writes - Do NOT attempt to query event data through the Eventstream API — use downstream skills (eventhouse-consumption-cli, sqldw-consumption-cli) for querying landed data
Does it work?
Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.
No reviews yet
People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.
- It worked
- Partly
- It didn't
Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.
Your trail
Files you open will collect here, so you can find them again.
Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.
Now do this for your own.
Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.

