MicrosoftKnown publishermicrosoft-foundry/foundry-agent-webapp/.github/skills/testing-cli-compatibility/SKILL.mdtesting-cli-compatibility
Skill124 starsChanged 7 months agoMITView on GitHub
Validate that Copilot CLI can see all repo skills, MCP servers, and custom instructions. Use when setting up CLI for the first time, after adding skills, or to diagnose CLI issues.
- Installs packages
In this repository20 other files it gives its agents.
- .github/copilot-instructions.mdCopilot instructions
- committing-codeSkill
- deploying-to-azureSkill
- implementing-chat-streamingSkill
- planning-featuresSkill
- researching-azure-ai-sdkSkill
- reviewing-documentationSkill
- syncing-mcp-serversSkill
- testing-with-playwrightSkill
- triaging-issuesSkill
- troubleshooting-authenticationSkill
- understanding-architectureSkill
- validating-deploymentsSkill
- validating-local-setupSkill
- validating-ui-featuresSkill
- writing-bicep-templatesSkill
- writing-csharp-codeSkill
- writing-typescript-codeSkill
- writing-unit-tests-csharpSkill
- writing-unit-tests-typescriptSkill
The file
We split this skill into the rules it gives an agent
6 rules. About 0 are common, 2 are distinctive, and 4 are very specific to this project.
≈0
Common · estimate
Many files like this have a rule of this kind.
≈2
Distinctive · estimate
This author's own choices. Worth a look.
4
Very specific
Tied to their project's names, files or tools. Probably skip.
- Common
- Distinctive
- Very specific
- Very specific: line 32
Authenticated: Run copilot login if not already logged in
- Very specific: line 33
MCP servers synced: Run the syncing-mcp-servers skill first if MCP tests fail
- Very specific: line 48
After adding or modifying skills in .github/skills/
- Very specific: line 49
After syncing MCP servers with the syncing-mcp-servers skill
- Distinctive: line 50
When a user reports CLI issues (missing agents, skills, or tools)
- Distinctive: line 55
If the script isn't available, you can test manually:
≈ “Common” is an estimate: at least 1 in 5 of the 350,495 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.
Files
Testing CLI Compatibility
Validates that the standalone Copilot CLI works correctly with this repo's skills and MCP servers.
Quick Start
# Run all tests (including MCP)
./deployment/scripts/test-cli-compatibility.ps1
# Skip MCP server tests (faster, no tool approval needed)
./deployment/scripts/test-cli-compatibility.ps1 -SkipMcpWhat It Tests
| # | Test | What It Checks |
| 1 | CLI Version | copilot command is installed and responds |
| 2 | Custom Instructions | .github/copilot-instructions.md is loaded (identifies project name) |
| 3 | Skills | All .github/skills/*/SKILL.md files are visible (count auto-detected) |
| 4 | MCP Servers | playwright and microsoftdocs servers are connected |
Prerequisites
- Copilot CLI installed:
npm install -g @anthropic-ai/copilot-clior via GitHub - Authenticated: Run
copilot loginif not already logged in - MCP servers synced: Run the
syncing-mcp-serversskill first if MCP tests fail
How It Works
The script uses copilot -p "<prompt>" -s --no-auto-update to invoke the CLI in non-interactive mode:
-p— Single prompt, exits after response-s— Silent mode, output only the agent response (no stats)--no-auto-update— Skip update check for faster execution--allow-all-tools— Used only for MCP test (needs tool access)
Each test sends a prompt asking the CLI to list what it can see, then validates the response against expected values.
When to Run
- After adding or modifying skills in
.github/skills/ - After syncing MCP servers with the
syncing-mcp-serversskill - When a user reports CLI issues (missing agents, skills, or tools)
- As a smoke test after CLI updates (
copilot update)
Manual Testing
If the script isn't available, you can test manually:
# Check version
copilot --version
# Test custom instructions
copilot -p "What project is this repo for?" -s
# Test skills
copilot -p "List all custom skills" -s
# Test MCP (needs --allow-all-tools)
copilot -p "List all MCP servers" -s --allow-all-toolsTroubleshooting
| Issue | Fix |
copilot not found | Install CLI or add to PATH |
| Custom instructions not loaded | Ensure you're in the repo directory |
| Agents missing | No custom agents expected — this repo uses skills + copilot-instructions only |
| Skills missing | Check .github/skills/*/SKILL.md files exist with valid frontmatter |
| MCP servers not connected | Run syncing-mcp-servers skill to sync .vscode/mcp.json → ~/.copilot/mcp-config.json |
Does it work?
Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.
No reviews yet
People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.
- It worked
- Partly
- It didn't
Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.
Your trail
Files you open will collect here, so you can find them again.
Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.
Now do this for your own.
Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.