Agent 365 Skills — Coding Agent Context

This repository is a Claude Code / GitHub Copilot CLI plugin marketplace containing skills for the Microsoft Agent 365 platform. Read this file before making any changes.

What's in this repo

agent365-skills/
├── .claude-plugin/marketplace.json    # Marketplace manifest — lists all plugins
├── plugins/
│   └── agent365/                      # The core plugin
│       ├── .claude-plugin/plugin.json # Skill declarations and triggers
│       ├── skills/
│       │   ├── a365-setup/SKILL.md                   # Entry point — CLI install, Azure prereqs, delegates
│       │   ├── make-a365-agent/SKILL.md               # Blueprint provisioning (Register / Observability paths)
│       │   ├── make-ai-teammate/SKILL.md              # AI Teammate transformation + publish pipeline
│       │   ├── instrument-observability/SKILL.md      # OTel + A365 tracing exporter instrumentation
│       │   ├── a365-code-validator/SKILL.md           # Observability/MAC Activity validation + guided fixes
│       │   ├── add-workiq-tools/SKILL.md              # WorkIQ MCP server wiring
│       │   ├── purview-dlp-integration/SKILL.md       # Purview DLP guard + gate wiring (Node.js / Python / .NET)
│       │   └── test-local/SKILL.md                    # Local testing with AgentsPlayground
│       ├── hooks/
│       │   ├── preToolUse/path-guard.js               # Blocks writes outside project root
│       │   └── stop/                                  # Stop hook validators (plain Node.js, no deps)
│       │       ├── validate-a365-setup.js
│       │       ├── validate-make-a365-agent.js
│       │       ├── validate-make-ai-teammate.js
│       │       ├── validate-instrument-observability.js
│       │       ├── validate-add-workiq-tools.js
│       │       ├── validate-purview-dlp-integration.js
│       │       └── validate-test-local.js
│       └── shared/agent-detection.md  # Shared heuristics for detecting agent type and authMode
├── tests/                             # Unit tests for stop hook validators
│   ├── helpers.js
│   ├── validate-a365-setup.test.js
│   ├── validate-make-a365-agent.test.js
│   ├── validate-observability.test.js
│   ├── validate-workiq.test.js
│   └── validate-purview-dlp-integration.test.js
├── evals/
│   └── agent365/                      # Evaluation test cases (one per skill)
│       ├── a365-setup/evals.json
│       ├── make-a365-agent/evals.json
│       ├── make-ai-teammate/evals.json
│       ├── instrument-observability/evals.json
│       ├── add-workiq-tools/evals.json
│       ├── purview-dlp-integration/evals.json
│       └── test-local/evals.json
├── scripts/install.js                 # One-liner installer for Claude Code + Copilot CLI
├── AGENTS.md                          # Top-level contributor guidelines
└── README.md                          # User-facing documentation

Key rules

1. SKILL.md files are instructions for the AI, not code. They are read at skill invocation time. Keep them precise, phase-by-phase, and unambiguous.
2. Validator scripts are plain Node.js. No dependencies allowed. They run in 15 seconds.
3. Reference docs are the source of truth for code patterns. When the A365 SDK updates, update references/ — not SKILL.md.
4. Never commit secrets. The .gitignore excludes a365.generated.config.json, .env, and appsettings.*.json.
5. Skills are additive. They never delete or restructure existing agent code.
6. authMode canonical values are obo, s2s, and agentic-user. Read case-insensitively, write lowercase.
7. has_obs and has_workiq are composite signals — entry-point symbol alone is insufficient. has_obs = true requires entry-point call AND token resolver AND handler-side BaggageBuilder / InvokeAgentScope. has_workiq = true requires non-empty ToolingManifest.json AND the framework's MCP wiring symbol (addToolServersToAgent / GetMcpToolsAsync / AddToolServersToAgentAsync / add_tool_servers_to_agent) AND — when mcp_WordServer + Node.js LangChain — the Word @mention wiring (WpxComment + proactive + userKeyToConversationId). Anything less is has_obs_partial / has_workiq_partial (read-time only, not cached). make-ai-teammate Phase 9.5 / 9.6 must re-enter the sub-skill on partial, not silently skip. add-workiq-tools Phase 4 must preserve obs anchors when editing files that overlap with the obs wrapping.
8. .a365-workspace-detection.local.json MUST exist before any code-edit phase. Every skill's Phase 0A Step 1 triage is responsible for writing this file (via a365-setup when missing on a project with code). Phase 0A Step 2 begins with a hard STOP guard that refuses to proceed when the file is absent. The stop-hook validators (validate-make-ai-teammate.js, validate-instrument-observability.js, validate-add-workiq-tools.js) fail the session at end if the cache wasn't written. Never invent default cache values — re-run a365-setup instead.
9. WorkIQ is not available for authMode = s2s — never offer or invoke add-workiq-tools for S2S agents. The guard exists at three layers: a365-setup, make-a365-agent Phase 4, and add-workiq-tools Phase 0B.
10. Run task lists to completion in one turn. When a skill creates a task list, execute every task and mark each complete (TaskUpdate or - [ ] → - [x]) the moment its phase finishes. Only pause at the explicit interaction points each SKILL.md documents (capabilities menu, run-target, blueprint Reuse/Re-run/Fresh, WorkIQ offer, MCP server selection, Word @mention offer when mcp_WordServer is selected on Node.js LangChain, launch confirmation). CLI permission prompts and manual browser steps (Teams Dev Portal, M365 Admin Center, GA consent) are not stopping conditions — surface them with URL + action and continue.
11. a365-code-validator is report-first. It diagnoses exporter activation, agent-id binding, semantic span coverage, delegated (OBO-route) telemetry or a missing S2S route flag, and live Blueprint grants / effective inheritance through read-only a365 query-entra checks when login is already available. It then asks whether to apply safe fixes, create a fix plan, or stop. It must not provision, install packages, mutate Graph, grant permissions, or run a365 publish.
12. purview-dlp-integration is additive and report-first-validated. It copies one generic env-driven guard and wires an INPUT gate before the LLM + optional output auditing; the supplied policy does not filter sensitive responses. Delegated guards (purview.ts / purview.py / purview.cs) use the agent's token at /me with Content.Process.User. Node.js client-secret FMI S2S uses purview-s2s.ts, the agent identity token at /users/{sponsor}/..., and Content.Process.All on the agent identity SP. Never use a blueprint app-only token, run az ad app permission admin-consent on the blueprint, or switch managed-identity-only agents to client-secret authentication. Manual IDs replace config discovery, not authentication; stop before editing unsupported plain bots and route to a365-setup. Guards always set contentEntry.name and fail closed by default. Its validator (validate-purview-dlp-integration.js) is report-first (advisory findings, always ok: true) because "start disabled / skip policy" is a valid bring-up state.
13. Telemetry always uses the S2S route with an app-only token, in every authMode. The S2S route rejects delegated (scp) tokens, so OBO / Agentic User tokens are only for workload calls (MCP / Graph). obo / agentic-user agents get an app-only resolver that reuses the hosting connection's blueprint credential (AgentAppTokenResolver.cs / app-token-resolver.ts / app_token_resolver.py); s2s agents get the FMI token-service scaffold. Always set the S2S route flag (o.Agent365.UseS2SEndpoint = true, useS2SEndpoint: true, a365_use_s2s_endpoint=True). Never generate a per-turn delegated telemetry token (RegisterObservability(..., AgenticTokenStruct), refreshObservabilityToken(...), exchange_token(...) for the observability scope), and never add the delegated OtelWrite scope for telemetry. Registered blueprint agent instances need no Agent365.Observability.OtelWrite permission or admin consent, so never make an OBS grant a required step for them. Newer a365 setup all versions skip OtelWrite for blueprint agents and fail when registration fails or cannot be verified; older versions may still grant OtelWrite (harmless) and may exit 0 after a failed registration, so check setup output and rerun a365 setup all --agent-registration-only if needed. For blueprint agents, a 403 insufficient_scope means the instance isn't registered. AI Teammates are authorized by their Blueprint's registration (subject to service policy), so newer CLI versions skip OtelWrite for them too; older versions still print an OtelWrite application-role step, which is harmless to complete. The application role on the Blueprint is always an accepted fallback on the S2S route.

Testing

# Run all validator unit tests
npm test

# Run a specific validator test suite
node --test tests/validate-a365-setup.test.js
node --test tests/validate-make-a365-agent.test.js
node --test tests/validate-observability.test.js
node --test tests/validate-workiq.test.js

# Validate stop hooks directly against the current directory
node plugins/agent365/hooks/stop/validate-a365-setup.js
node plugins/agent365/hooks/stop/validate-instrument-observability.js
node plugins/agent365/hooks/stop/validate-make-a365-agent.js

# Test skill against a real agent project
cd /path/to/dotnet-agent
claude --plugin-dir /path/to/agent365-skills/plugins/agent365
# Say: "instrument observability for this agent"

# Run evals manually
# See evals/README.md for detailed testing instructions

For comprehensive eval test cases, see evals/README.md.

Allowed commands

dotnet *, npm *, node *, python *, python3 *, pip *, pip3 *, uv *, a365 *, az *, pwsh *, devtunnel *, agentsplayground *, git *, grep *, find *, cat *, ls *